December 5, 2016

Committee Members in action? Empirical Study on the Influence of Referee Committee Members on the Referee Categories in UEFA

Referee Advent Calendar - Door 5: On next Thursday, UEFA's Referee Committee are going to define the new Referee Categories for the 2nd half of the 2016/17 season. In former times, the category allocation (promotions and demotions) were partly criticized for being politically motivated or at least influenced by the members of the Referee Committee - by us as independent observers, by you as our readers and commentators, and by some international referees who frequently express their displeasure about that topic when talking to them ("Some of us are more equal than the others - it depends on whether you have a committee member behind you" is only one quote).


Is that true? We analyzed objective data to investigate the relationship between a federation's representation in the UEFA Referee Committee and the same nation's representation in the UEFA Referee Categories. A study based on a scientific and statistical approach tends to support the intuitive feeling some officials have. How we proceeded and how the results look like:


Method

Data Basis and Calculation

As the data basis, we used the current UEFA Referee Categories (version 2016-17, 1st half). Each nation was awarded points that varied across the different categories following an intuitive scheme:

Elite Group: 4 points; First Group: 3 points; Second Group: 2 points; Third Group: 1 point.

On the basis of that, three values were calculated:

1) The respective nation's number of FIFA Referees included in the Category list ("Refs in total").

2) The respective nation's number of weighted Category Points following the scheme above.

3) A category-height-ratio for each nation by dividing the two previous variables (Category Points divided by Refs in total). We call that "Category Ratio". This tells us how highly the referees of a certain nation are located in the categories on average.

An example: Italy

1) 4 referees in Elite + 2 referees in First Group + 4 referees in Second Group = 10 referees

2) 4*4 + 2*3 + 4*2 + 0*1 = 30

3) 30 Category Points / 10 Refs in Total = 3.0 

Compare it with Ireland:

1) 0 referees in Elite + 0 referees in First Group + 2 referees in Second Group + 2 referees in Third Group

2) 0*4 + 0*3 + 2*2 + 2*1 = 6

3) 6 Category Points / 4 Refs in total = 1.5

So Italy is clearly higher represented in the categories than Ireland - their officials reach higher referee categories on average!


Nations

In a second step, we identified those nations represented by a Refereeing Officer or Chairman (Italy, France, Scotland and Spain), those with other members of the UEFA Referee Committee (Czech Republic, England, Germany, Netherlands, Romania*, Russia, Slovenia, Sweden) and those without a Committee member (the other 42 UEFA member nations except Gibraltar, Kosovo and Liechtenstein).

*We counted Kyros Vassaras as representant of the Romanian FA, for which he primarily works.


Variables and Statistical Analysis Methods

First of all: We want to investigate a possible effect of a certain input variable ("independent variable") on a certain output variable ("dependent variable"). In this case, our independent variable is a nation's belonging to one of the three groups ("Officer group", "Committee member group" and "No Committee member group"). Our dependent variable consists of the nations' Category Ratio we calculated in steps ) above.

What we could now do is simply calculating means and having a look for differences between the mean category ratios of the nations in the three groups. If the suspected political influences exist, the means should be significantly higher for those nations who have a Committee member or even a Referee Officer inside UEFA.

However, this would stay speculation. There would be no limit at which we could say "Yes, there is an influence" or "No, there is no influence". That's why statisticians have developed statistical tests to test such assumptions reliably. The aim of such tests is to check whether there are statistically significant differences, e.g. between mean scores of two or more different groups in a certain variable.

The statistical significance is usually mentioned as a p-value. If the p-value is p < .05 (or p < 5%), this means that a potential difference between two or more groups is unlikely to be random: The likelihood, that such an effect is mere coincidence and a random result, would be smaller than 5%. Such significance levels can even be smaller, e.g. 1%, 0,1% or even 0,01%. The smaller the significance level is, the better - because we can say that the effect we found is robust and not just coincidence. And as claims of political influences in refereeing are often speculation, it is important for me to conduct objective tests and not just have a short view on some rough data.


Alternative Explanations

Up to here, most of you will probably think "Well, these nations with committee members are widely from Europe's top football nations. In these nations, there are better or more experienced referees, used to bigger stadiums, faster football and of course they are therefore higher ranked in the Referee Categories!".

This is probably correct. That's why we conducted our analysis under consideration of a possible influence of the nations' football quality. The most objective data we found to operationalize that are the UEFA Association Coefficients (2012 - 2016).

If referees benefit from the good footballing level of the teams in their domestic division, then the last 3 or 4 seasons' coefficients should be a good parameter to measure that.

What we conducted is called "Analysis of Covariance" (ANCOVA). This ANCOVA does exactly what we want and need. It tests whether a possible difference between the three groups of nations (Officer, Committee member, No committee member) with regard to the Category Ratios is statistically significant and even considers the influence of the nations' UEFA coefficients. So if we find a significant effect, we would be able to say "The effect found is not only due to the different football quality in the committee member's nations, but also because of something else.".


Results

An overview with the raw scores of the nations can be found in the following table:

Nation Ratio
Nation Ratio
Nation Ratio
Nation Ratio
Spain** 3,00
Hungary 2,17
Montenegro 1,67
Bosnia&Herzeg. 1,33
Italy** 3,00
Israel 2,17
Portugal 1,67
Estonia 1,33
Netherlands* 3,00
Austria 2,14
Wales 1,67
Latvia 1,33
England* 2,88
Belgium 2,00
Croatia 1,60
Iceland 1,25
Scotland** 2,86
Belarus 2,00
Greece 1,57
Kazakhstan 1,25
Germany* 2,80
Macedonia 2,00
Switzerland 1,57
Northern Ireland 1,25
Slovenia* 2,75
Serbia 2,00
Albania 1,50
Andorra 1,00
Romania* 2,57
Norway 1,86
Bulgaria 1,50
Armenia 1,00
France** 2,50
Ukraine 1,86
Cyprus 1,50
Faroe Islands 1,00
Poland 2,43
Czech Rep.* 1,83
Ireland 1,50
Georgia 1,00
Sweden* 2,29
Finland 1,80
Denmark 1,43
Luxembourg 1,00
Turkey 2,29
Malta 1,75
Slovakia 1,40
Moldova 1,00
Russia* 2,22
Lithuania 1,67
Azerbaijan 1,33
San Marino 1,00

*nation with Committee member, **nation with Referee Officer or Chairman

The two diagrams below illustrate the main result: With regard to the Category Ratios, which point out how a nation's refereeing is relatively represented and how highly a nation's referees have got within the category system on average, the nations belonging to the Officer group scored highest (M = 2.84, SD = 0.24), followed by the Committee member group (M = 2.54, SD = 0.40) and the No Committee member group (M = 1.57, SD = 0.40).

*M = mean, SD = standard deviation



The ANCOVA yielded highly significant main effects (p < .001). This means that there is a certain significant difference between the means of the three groups, for which the likelihood of being random is smaller than 0,1%. This does not tell us between which exact groups the significant differences are, though.

Pairwise comparisons showed that there was no significant difference between the Officer and Committee member groups, but there were highly significant differences between the respective means of nations from the Officer group and the No Committee member group as well as between the Committee member group and the No Committee member group. These significance levels are illustrated by the green stars in the diagram!

To measure how much variance or, in other words, how much of a nation's representation in the categories can be explained by whether this nation is or is not represented by an officer or member in the UEFA Referee Committee, the so-called Partial Eta² was calculated.

For the Category Ratios, it was Partial Eta² = .35. This means that 35% of how highly a nation's referees are located in the category system on average can be explained by having or not having a committee member or referee officer from the same nation. In contrast, only 13% could be explained by the quality of football mirrored by the UEFA association coefficients.


Discussion

What do these findings mean? And how to interpret them?

Nations represented by an Officer or a Committee member inside UEFA are more represented in the higher Referee Categories than nations without a Committee member or Officer. The mean differences between the two groups with committee representation and the group without a Committee member are both significant. The chance, that these differences and results are just based on coincidence, have been found to be smaller than 0,1% and 1% respectively.

35% (Partial Eta² = .35) of a nation's relative representation in the UEFA Category System were found to be explained by the presence or absence of a Referee Committee member or Officer from the same nation.

We have therefore found robust evidence for an effect of whether a nation is represented in the UEFA Referee Committee on how highly this nation is represented in the UEFA Referee Categories.

We used objective data to minimize subjective sentiments and to avoid speculation.

We have included the possibility that Referee Committee members are usually from the bigger footballing nations with an equally good level of refereeing. By intergrating the UEFA association coefficients into the statistical model, we can say that the effect of the Committee presence still exists. The nations' football level operationalized by the association coefficients obviously has an effect on the categories, but only a smaller one.


Implications and Limitations

Of course you can question whether the distribution of points to the categories (4, 3, 2 and 1) is reasonable. You can equally question whether someone like Mark Clattenburg really needed David Elleray to become a world-class referee. Most likely, the real meaningful effects depicted in the results of our calculation are rather located in the lower area of the Elite Group and particularly in the two or three lower categories.

A criticism on the procedure might be that some nations such as England, France, Italy, Germany and Spain are de facto not able to have a referee in Third Group as their officials usually start in Second Group as soon as they become international officials. It is however unprobable that this majorly influenced the effect found.

In addition it might be that those nations with a committee member are generally characterized by a higher level of refereeing and referee education which maybe even enabled certain people to enter the UEFA committee.

What we did not consider in our study are the roles some Referee Committee members fulfill in foreign referee departments (e.g. Marc Batta in Belgium, Pierluigi Collina in the Ukraine, Hugh Dallas in Israel, Jaap Uilenberg in Turkey until 2016 etc.). Unfortunately, we do not have enough information to consider this as an additional variable. But the ranking posted above partly suggests that some of the mentioned nations are not that badly represented in the categories.

UEFA can nonetheless use these findings to critically reflect themselves in terms of whether they (un)consciously favour referees from certain nations - and in this concrete case, from the commitee members' nations.

More focus should be shifted on officials from countries that are not represented in the committee. UEFA has the chance to do that on Thursday. Szymon Marciniak from Poland or Milorad Mažić from Serbia are good examples that this might pay off in the long run. UEFA's focus on referees from "smaller" nations like Azerbaijan (Aliyar Aghayev), Lithuania (Gediminas Mazeika) or Latvia (Andris Treimanis) is a good first sign, but rather the exception.

It should not be doubted that those referees who reached the highest referee categories mostly are excellent officials and widely deserved to get to where they are! However, it can be assumed that there are numerous match officials from other nations who are underrepresented in both the referee committee and - maybe even for this reason - are equally underrepresented in the category system and would never get sufficient chances to develop and climb the career ladder.

In a future article, we are going to focus on a similar question taking into account influences on referee appointments as another dependent variable.

25 Comments:

  1. National assocciation have 4 fifa referee and send uefa 5 referees surname.can iefa make some rotation?for exapmle 5th became 3rd and 3rd bacome 5th and etc. Tnahk you

    ReplyDelete
  2. My prediction-demoted:Tomson,Moen,Tagliavento,Borbalan,Nijhuis.
    Promoted:Oliver,Kulbakov,Mekkelie.

    ReplyDelete

  3. Grand season of Borbalán. Not possible if he is demoted.

    ReplyDelete
  4. Imo,
    Promotions: 2-3 countries represented in the referee committee + Kulbakov

    ReplyDelete
  5. Replies
    1. I don't understand your question.

      Delete
    2. National federation has 4(A;B;X;Y) referees in fifa 2016 list.for 2017 year and send 5 referees (A;B;X;Y;Z).question.can fifa or uefa made rotation for 2017 and tell national federation that in 2017 fifa referees will be : A;B;Z;Y (x referee put out by fifa or uefa)

      Delete
    3. I understood the question.

      No, UEFA/FIFA cannot change the order the referees are nominated. That decision stays on the National FA.

      Delete
  6. Democtions:Moen,Thomson
    Promotions:Gil Manzano,Oliver or Taylor,Kulbakov

    ReplyDelete
    Replies
    1. Gil (and Manzano), did not have any CL games
      Unlikely!

      Delete
    2. Why does quite many here predict Thomson to be demoted? I think he has a season without big mistakes until now. There are some other referees who would "deserve" a demotion instead of Thomson. I am thinking about Tagliavento, who showed some weaknesses in the past seasons and doesn't get really important appointments by now or Borbalan, who is a good referee for sure but he wasn't able to make the last big step to be a top class official. And, of course, Moen...

      So I think that they won't demote more than one, and this one will be Moen and nobody else. It wouldn't be fair if anyone else would get demoted while he stays in elite group.

      It is difficult to predict rhe referees to be promoted, but I think that Makkelie, Kulbakov, Zwayer, de Sousa (or Soares) and Oliver (or Taylor) have quite good chances. However, it is clear that they won't promote all of them.

      Delete
  7. Number of games by referees and their categories:

    ELITE GROUP

    3 CL + 1 EL
    Mark Clattenburg (ENG), Milorad Mažić (SRB)

    3 CL
    Deniz Aytekin (GER), Felix Brych (GER), Cüneyt Çakır (TUR), William Collum (SCO), Ovidiu Alin Hațegan (ROU), Sergey Karasev (RUS), Pavel Královec (CZE), Björn Kuipers (NED), Szymon Marciniak (POL), Daniele Orsato (ITA), Nicola Rizzoli (ITA), Gianluca Rocchi (ITA), Anastassios Sidiropoulos (GRE), Damir Skomina (SVN), Craig Thomson (SCO), Clément Turpin (FRA)

    2 CL + 1 EL
    Martin Atkinson (ENG), Jonas Eriksson (SWE), David Fernández Borbalán (ESP), Viktor Kassai (HUN), Bas Nijhuis (NED), Paolo Tagliavento (ITA), Alberto Undiano Mallenco (ESP)

    2 CL
    Antonio Miguel Mateu Lahoz (ESP)

    2 EL
    Svein Oddvar Moen (NOR)


    FIRST GROUP

    3 CL
    Michael Oliver (ENG), Manuel Jorge Neves Moreira de Sousa (POR), Martin Strömbergsson (SWE), Felix Zwayer (GER)

    2 CL + 1 EL
    Ruddy Buquet (FRA), Matej Jug (SVN), Alaksey Kulbakou (BLR), Artur Manuel Ribeiro Soares Dias (POR), Anthony Taylor (ENG)

    2 CL
    Danny Makkelie (NED)

    1 CL + 2 EL
    Gediminas Mažeika (LTU), Slavko Vinčič (SVN)

    3 EL
    Yevhen Aranovsky (UKR), Benoît Bastien (FRA), Ivan Bebek (CRO), Vladislav Bezborodov (RUS), Tamás Bognár (HUN), Serhiy Boyko (UKR), Aleksey Eskov (RUS), Xavier Estrada Fernández (ESP), Jesús Gil Manzano (ESP), Hüseyin Göçek (TUR), Serdar Gözübüyük (NED), Manuel Gräfe (GER), Stefan Johannesson (SWE), Jakob Kehlet (DEN), István Kovács (ROU), Ivan Kružliak (SVK), Harald Lechner (AUT), Liran Liany (ISR), Paweł Raczkowski (POL), Robert Schörgenhofer (AUT), Aleksandar Stavrev (MKD), Tobias Stieler (GER), István Vad (HUN), Miroslav Zelinka (CZE)

    2 EL
    Luca Banti (ITA), John Beaton (SCO), Kevin Blom (NED), Sébastien Delferière (BEL), Simon Lee Evans (WAL), Paweł Gil (POL), André Marriner (ENG), Davide Massa (ITA), Paolo Silvio Mazzoleni (ITA), Clayton Pisani (MLT), Tobias Welz (GER), Alon Yefet (ISR)

    1 EL
    Tony Chapron (FRA), Oliver Drachta (AUT), Robert Madden (SCO), Alexandru Dan Tudor (ROU)

    0 CL & EL
    Mattias Gestranius (FIN), Serge Gumienny (BEL), Halis Özkahya (TUR)


    SECOND GROUP

    3 EL
    Əliyar Ağayev (AZE), Orel Grinfeld (ISR), Andris Treimanis (LTU)

    2 EL
    Carlos del Cerro Grande (ESP), Andreas Ekberg (SWE), Stephan Klossner (SUI), Ali Palabıyık (TUR)

    1 EL
    Kevin Clancy (SCO), Hugo Filipe Ferreira Campos Moreira Miguel (POR), Craig Pawson (ENG), Radu Marian Petrescu (ROU), Sandro Schärer (SUI), Daniel Stefański (POL), Bart Vertenten (BEL)

    ReplyDelete
    Replies
    1. Thank you so much, a small correction: Andris Treimanis (LVA).

      Delete
    2. Great job!! Thank you... now, it´s time for predictions!! ;)

      Delete
    3. There will be a post on that by tomorrow. If possible, don't shoot your munition for now and wait 1 day :) Thanks!

      Delete
    4. Davide Massa (ITA) with 2 EL is still currently Second Group.
      For me it is candidate to promote First Categorie and more late probably Elite

      Delete
  8. Bravo and thank you RayHD!

    ReplyDelete
  9. IMO Mr. A.Kulbakov will be promoted for the Elite group. Also in my opinion it is very big possibility that Latvian referee Treimanis will be promoted for the 1st group.

    ReplyDelete
  10. Off topic: Three controversies from today's Ruch Chorzów - Arka Gdynia game in Polish Ekstraklasa (refereed by Zbigniew Dobrynin, his 2nd game in Ekstraklasa)

    https://streamable.com/vmui

    ReplyDelete
    Replies
    1. IMO only the last one situation could be discused as the penalty kick.

      Delete
  11. Thank you,,Niclas, for a very interesting article. I like that kind of articles where hard data and statisticial methodologies are used. I hope you will provide us with some new insightful articles in the future :-).

    Well, I'd like to point out to the end of the article where you mention that more focus should be shifted on officials from countries that are not represented in the committee and you give Szymon Marciniak from Poland and Milorad Mažić from Serbia as two examples. I would add to this list also Viktor Kassai from Hungary. Although there is no UEFA Committee Member from Hungary, Kassai has got some high profile appointments in recent years or this year particularly. I presume that some existing members of the Committee have got quite an influence also on the appointments of previously mentioned referees. Of course, it's a way better to have both, a Committee member and a referee from the same country, since this increases their influence. But in my opinion some of the existing Committee members act also as the ''godfathers'' of well renowned referee names (and their appointments) coming from ''smaller/less important country'' or to say that each member of the Committee ''covers'' the Referee Associations in the geographical proximity of the country he comes from. What the other members of this blog think in this respect?

    As regards the new UEFA President Mr. Ceferin, who comes from Slovenia, he emphasized at the beginning of his new position that during his Presidency the football associations from smaller countries will get more influence in decision making within UEFA. So, the proposal at the end of the article to appoint more referees from other (smaller or with not so strong football league) countries comes at the right moment and we can only hope that Mr. Ceferin would keep and fulfil his promises.

    ReplyDelete
  12. National federation has 4(A;B;X;Y) referees in fifa 2016 list.for 2017 year and send 5 referees (A;B;X;Y;Z).question.can fifa or uefa made rotation for 2017 and tell national federation that in 2017 fifa referees will be : A;B;Z;Y (x referee put out by fifa or uefa)

    ReplyDelete
    Replies
    1. I really don't know whether FIFA has to approve the new FIFA place first before an association can send an additional proposal. But it is correct that FIFA or maybe even UEFA have to approve the proposals made by the associations, so maybe such rotations are not impossible.

      Delete
  13. can anybody tell me when will be new fifa referees list announced ?

    ReplyDelete

Copyright © . The 3rd Team
Theme Template by BTDesigner · Powered by Blogger