To Pot with That: The Quest to Clarify San Marino’s Televote

27/05/2021

Since 2016, jury and televote results have been presented separately, requiring a country to be able to provide data for both. San Marino, simply cannot fulfil this criteria, as it shares its phone lines with neighbouring Italy. To get around this problem the EBU employs a mathematical formula to work out what San Marino’s televote points ‘should be’.

San Marino gave in the televote:

1 point to Russia

2 points to Bulgaria

3 points to Switzerland

4 points to Finland

5 points to Ukraine

6 points to Moldova

7 points to Greece

8 points to Cyprus

10 points to France

And the maximum 12 points to Italy

So how is this done?

Well, the European Broadcasting Union (EBU) Reference Group is in charge of overseeing the calculating of this sum. They are a body of both elected and appointed industry professionals, including of course our Executive Supervisor, the one… the only… Martin Österdahl. Now we’ve clarified that I think we’re ‘good to go’!

The first decision the EBU Reference Group have to make is to decide which countries to include in the model. The first port of call for these countries is likely to be the remainder of the ‘pot’ in which San Marino (or whatever country couldn’t provide a valid televote fell into). Pots are simply groupings of countries the EBU has put together based off of shared voting patterns in the past 15 years. In 2021, the semi-final allocation draw and the pots that were used to create it were carried over from the cancelled 2020 contest. This left San Marino in pot 4 with Bulgaria, Cyprus, Greece, Malta, Portugal and Romania. So the first likely assumption is that these were the countries used in the creation of San Marino’s 2021 televote.

Another factor that then has to be considered is the exact model to be used for the results. The simplest would of course be to take an average of the rankings of all the countries. This would create the following results:

1 point to Russia

2 points to Bulgaria

3 points to Sweden

4 points to Lithuania

5 points to Iceland

6 points to Switzerland

7 points to Ukraine

8 points to Finland

10 points to France

And the 12 points to Italy

	Bulgaria	Cyprus	Greece	Malta	Portugal	Romania	Mean
Italy	1	2	2	1	4	2	2
France	2	3	3	8	1	3	3.333333
Finland	3	7	5	4	6	5	5
Ukraine	5	5	6	10	2	4	5.333333
Switzerland	8	11	7	13	5	6	8.333333
Iceland	11	15	11	6	8	12	10.5
Lithuania	17	6	13	5	14	8	10.5
Sweden	10	13	15	3	7	15	10.5
Bulgaria	N/A	4	12	11	13	14	10.8
Russia	4	8	10	16	10	18	11
Norway	12	16	14	2	11	13	11.33333
Cyprus	13	N/A	1	9	19	16	11.6
Azerbaijan	7	14	9	12	21	7	11.66667
Serbia	6	12	16	7	20	11	12
Malta	20	10	8	N/A	15	9	12.4
Greece	9	1	N/A	17	19	21	13.4
Moldova	15	25	21	24	3	1	14.83333
Israel	14	9	22	21	16	10	15.33333
Albania	19	21	4	18	24	20	17.66667
Portugal	18	17	19	15	N/A	22	18.2
Germany	16	18	20	19	17	23	18.83333
San Marino	23	19	17	20	23	17	19.83333
Spain	22	22	23	23	12	19	20.16667
Belgium	21	20	18	22	18	24	20.5
United Kingdom	25	24	24	14	25	25	22.83333
Netherlands	24	23	25	25	22	26	24.16667

However, the EBU introduced an exponential system to create a system where songs at the top of rankings carry more weight than songs at the bottom. So unsurprisingly those weren’t the results given. We have a slight problem here though, as the EBU has never revealed the exact formula they used to calculate these points on an exponential scale. Instead, all we have is a graphic which displays roughly the conversion used in the formula.

So what we have is a graph but what we need to work out the formula is the equation of the line. Since the shape is clearly an exponential graph, we simply require two points lying on the line to work out the formula. The issue is, the scale on the graph is not inherently clear. We can state with a lot of confidence that a point roughly at (1,12) lies on the line but any second point will be a little more inaccurate. The clearest second point on the line lies at x=26, the factored used for the last place country. It lies somewhere between y=0 and y=2. With this information, to create a similar formula to the EBU and get closer to the results, I took the mid-point of those to values and used that as my rough second point.

Having the equation of an exponential graph y=be^-ax and the two points (1,12) and (26,1) we can use simultaneous equations to create an equation which is similar to that used by the EBU.

Skipping the working out, this comes to y=32.29e^-0.99x

But does this roughly follow with the televotes calculated for San Marino?

Using this method San Marino would give:

1 point to Bulgaria

2 points to Sweden

3 points to Finland

4 points to Norway

5 points to Ukraine

6 points to Moldova

7 points to Greece

8 points to Cyprus

10 points to France

And the 12 points to Italy

Country	Bulgaria	Cyprus	Greece	Malta	Portugal	Romania	Mean
Italy	11.99821	4.458256	4.458256	11.99821	0.615548	4.458256	6.331123
France	4.458256	1.656584	1.656584	0.011734	11.99821	1.656584	3.572992
Cyprus	8.31E-05	N/A	11.99821	0.00436	2.19E-07	4.26E-06	2.400532
Greece	0.00436	11.99821	N/A	1.58E-06	2.19E-07	3.02E-08	2.400515
Moldova	1.15E-05	5.76E-10	3.02E-08	1.55E-09	1.656584	11.99821	2.275801
Ukraine	0.228723	0.228723	0.084988	0.00162	4.458256	0.615548	0.93631
Norway	0.000224	4.26E-06	3.09E-05	4.458256	0.000602	8.31E-05	0.7432
Finland	1.656584	0.03158	0.228723	0.615548	0.084988	0.228723	0.474358
Sweden	0.00162	8.31E-05	1.15E-05	1.656584	0.03158	1.15E-05	0.281648
Bulgaria	N/A	0.615548	0.000224	0.000602	8.31E-05	3.09E-05	0.123298
Russia	0.615548	0.011734	0.00162	4.26E-06	0.00162	5.89E-07	0.105088
Albania	2.19E-07	3.02E-08	0.615548	5.89E-07	1.55E-09	8.13E-08	0.102591
Switzerland	0.011734	0.000602	0.03158	8.31E-05	0.228723	0.084988	0.059618

Ultimately, this comes a lot closer – bringing Cyprus, Greece and Moldova up in the rankings to a level closer to what was given. However it still isn’t as close as you would expect, Sweden and Norway are nowhere to be seen in the actual result. Instead Switzerland and Russia should be in the top 10 but currently they are sitting in 11^th and 13^th place. So we clearly aren’t far off but we aren’t there yet.

The next question is why? The first reason could be the inaccuracies in the formula we created. Because one of the points was not precise – this could be the factor changing the results. However I believe this is unlikely to be the only reason as Norway is surprisingly comfortably high in the rankings to completely drop off.

There is however a second reason! The EBU reference committee picks the countries being used for this televote. Could it be possible that the EBU used a slight variation of these countries?

To test this the first thing to check was which country was increasing the standing of Norway and Sweden within the group. As Malta ranked Norway 2^nd and Sweden 3^rd – they are the likely candidate to have been excluded from the results. So what happens to the mean values when Malta is excluded?:

1 point to Albania

2 points to Russia

3 points to Bulgaria

4 points to Finland

5 points to Ukraine

6 points to Moldova

7 points to Cyprus

8 points to Greece

10 points to France

And the 12 points to Italy

Country	Bulgaria	Cyprus	Greece	Portugal	Romania	Mean
Italy	11.99821	4.458256	4.458256	0.615548	4.458256	5.197705
France	4.458256	1.656584	1.656584	11.99821	1.656584	4.285244
Greece	0.00436	11.99821	N/A	2.19E-07	3.02E-08	3.000643
Cyprus	8.31E-05	N/A	11.99821	2.19E-07	4.26E-06	2.999575
Moldova	1.15E-05	5.76E-10	3.02E-08	1.656584	11.99821	2.730961
Ukraine	0.228723	0.228723	0.084988	4.458256	0.615548	1.123248
Finland	1.656584	0.03158	0.228723	0.084988	0.228723	0.44612
Bulgaria	N/A	0.615548	0.000224	8.31E-05	3.09E-05	0.153971
Russia	0.615548	0.011734	0.00162	0.00162	5.89E-07	0.126105
Albania	2.19E-07	3.02E-08	0.615548	1.55E-09	8.13E-08	0.12311
Switzerland	0.011734	0.000602	0.03158	0.228723	0.084988	0.071525
Lithuania	1.58E-06	0.084988	8.31E-05	3.09E-05	0.011734	0.019368

Fascinatingly, dropping Malta gives us all the correct countries apart from 11^th place Switzerland being pushed out for Albania.

Using this model we therefore have two errors:

1. Switzerland is 3 places too low
2. Greece and Cyprus have now swapped places.

However, we are a lot closer to the actual results now.

OK ESC Essence, but this still isn’t the correct model. To that I have to say, you are absolutely right! The most likely reason for this now though is probably that slightly inaccurate data point from our initial assumptions.

If anyone else has looked at this in more detail and can add onto what we have uncovered – or wants to use it as a starting point, please go for it! I for one would be really curious to hear what you found out. This brings in the question of whether other people have looked at this matter and whether these theories hold across the years. That is a massive question that maybe in time we will come back with more answers to. In the meantime, I can only recommend fellow fansite ESC Insight’s analysis of the 2018 results where they tried various models on that dataset. Once again, they came close to finding the formula used but did not have a precise answer – it’s definitely worth a read and still relevant to this day!

Without the exact formula what can we learn from today’s analysis though?

1. The exponential system being used means that country pairings such as Greece: Cyprus and Romania: Moldova will be emphasised in San Marino’s points – causing initially peculiar high results to said countries.
2. The biggest losers from the complex methodologies being used to work out San Marino’s points were Norway, Sweden and Albania – who dropped off completely after all the procedures took place
3. Moldova, Greece and Cyprus gained the most from this system
4. San Marino’s televote is a complex yet important aspect of the Eurovision vote – in which it would be beneficial for the EBU to reveal more steps involved
5. San Marino’s televotes do not appear suspicious at all when analysing the countries in their pot

I hope this article has inspired you to dive into the Eurovision data! There is also data from the semi-finals which should be analysed, as well as previous years. Beyond that I hope despite all the stats and maths it was an enjoyable read! Comments are always open if you wish to discuss this topic and there is more amazing content to be found on our YouTube channel – linked here!

Special thanks to Nick for picking up my curiosity on this topic.