Is that true? We analyzed objective data to investigate the relationship between a federation's representation in the UEFA Referee Committee and the same nation's representation in the UEFA Referee Categories. A study based on a scientific and statistical approach tends to support the intuitive feeling some officials have. How we proceeded and how the results look like:
Data Basis and Calculation
As the data basis, we used the current UEFA Referee Categories (version 2016-17, 1st half). Each nation was awarded points that varied across the different categories following an intuitive scheme:
Elite Group: 4 points; First Group: 3 points; Second Group: 2 points; Third Group: 1 point.
On the basis of that, three values were calculated:
1) The respective nation's number of FIFA Referees included in the Category list ("Refs in total").
2) The respective nation's number of weighted Category Points following the scheme above.
3) A category-height-ratio for each nation by dividing the two previous variables (Category Points divided by Refs in total). We call that "Category Ratio". This tells us how highly the referees of a certain nation are located in the categories on average.
An example: Italy
1) 4 referees in Elite + 2 referees in First Group + 4 referees in Second Group = 10 referees
2) 4*4 + 2*3 + 4*2 + 0*1 = 30
3) 30 Category Points / 10 Refs in Total = 3.0
Compare it with Ireland:
1) 0 referees in Elite + 0 referees in First Group + 2 referees in Second Group + 2 referees in Third Group
2) 0*4 + 0*3 + 2*2 + 2*1 = 6
3) 6 Category Points / 4 Refs in total = 1.5
So Italy is clearly higher represented in the categories than Ireland - their officials reach higher referee categories on average!
In a second step, we identified those nations represented by a Refereeing Officer or Chairman (Italy, France, Scotland and Spain), those with other members of the UEFA Referee Committee (Czech Republic, England, Germany, Netherlands, Romania*, Russia, Slovenia, Sweden) and those without a Committee member (the other 42 UEFA member nations except Gibraltar, Kosovo and Liechtenstein).
*We counted Kyros Vassaras as representant of the Romanian FA, for which he primarily works.
Variables and Statistical Analysis Methods
First of all: We want to investigate a possible effect of a certain input variable ("independent variable") on a certain output variable ("dependent variable"). In this case, our independent variable is a nation's belonging to one of the three groups ("Officer group", "Committee member group" and "No Committee member group"). Our dependent variable consists of the nations' Category Ratio we calculated in steps ) above.
What we could now do is simply calculating means and having a look for differences between the mean category ratios of the nations in the three groups. If the suspected political influences exist, the means should be significantly higher for those nations who have a Committee member or even a Referee Officer inside UEFA.
However, this would stay speculation. There would be no limit at which we could say "Yes, there is an influence" or "No, there is no influence". That's why statisticians have developed statistical tests to test such assumptions reliably. The aim of such tests is to check whether there are statistically significant differences, e.g. between mean scores of two or more different groups in a certain variable.
The statistical significance is usually mentioned as a p-value. If the p-value is p < .05 (or p < 5%), this means that a potential difference between two or more groups is unlikely to be random: The likelihood, that such an effect is mere coincidence and a random result, would be smaller than 5%. Such significance levels can even be smaller, e.g. 1%, 0,1% or even 0,01%. The smaller the significance level is, the better - because we can say that the effect we found is robust and not just coincidence. And as claims of political influences in refereeing are often speculation, it is important for me to conduct objective tests and not just have a short view on some rough data.
Up to here, most of you will probably think "Well, these nations with committee members are widely from Europe's top football nations. In these nations, there are better or more experienced referees, used to bigger stadiums, faster football and of course they are therefore higher ranked in the Referee Categories!".
This is probably correct. That's why we conducted our analysis under consideration of a possible influence of the nations' football quality. The most objective data we found to operationalize that are the UEFA Association Coefficients (2012 - 2016).
If referees benefit from the good footballing level of the teams in their domestic division, then the last 3 or 4 seasons' coefficients should be a good parameter to measure that.
What we conducted is called "Analysis of Covariance" (ANCOVA). This ANCOVA does exactly what we want and need. It tests whether a possible difference between the three groups of nations (Officer, Committee member, No committee member) with regard to the Category Ratios is statistically significant and even considers the influence of the nations' UEFA coefficients. So if we find a significant effect, we would be able to say "The effect found is not only due to the different football quality in the committee member's nations, but also because of something else.".
An overview with the raw scores of the nations can be found in the following table:
*nation with Committee member, **nation with Referee Officer or Chairman
The two diagrams below illustrate the main result: With regard to the Category Ratios, which point out how a nation's refereeing is relatively represented and how highly a nation's referees have got within the category system on average, the nations belonging to the Officer group scored highest (M = 2.84, SD = 0.24), followed by the Committee member group (M = 2.54, SD = 0.40) and the No Committee member group (M = 1.57, SD = 0.40).
*M = mean, SD = standard deviation
The ANCOVA yielded highly significant main effects (p < .001). This means that there is a certain significant difference between the means of the three groups, for which the likelihood of being random is smaller than 0,1%. This does not tell us between which exact groups the significant differences are, though.
Pairwise comparisons showed that there was no significant difference between the Officer and Committee member groups, but there were highly significant differences between the respective means of nations from the Officer group and the No Committee member group as well as between the Committee member group and the No Committee member group. These significance levels are illustrated by the green stars in the diagram!
To measure how much variance or, in other words, how much of a nation's representation in the categories can be explained by whether this nation is or is not represented by an officer or member in the UEFA Referee Committee, the so-called Partial Eta² was calculated.
For the Category Ratios, it was Partial Eta² = .35. This means that 35% of how highly a nation's referees are located in the category system on average can be explained by having or not having a committee member or referee officer from the same nation. In contrast, only 13% could be explained by the quality of football mirrored by the UEFA association coefficients.
What do these findings mean? And how to interpret them?
Nations represented by an Officer or a Committee member inside UEFA are more represented in the higher Referee Categories than nations without a Committee member or Officer. The mean differences between the two groups with committee representation and the group without a Committee member are both significant. The chance, that these differences and results are just based on coincidence, have been found to be smaller than 0,1% and 1% respectively.
35% (Partial Eta² = .35) of a nation's relative representation in the UEFA Category System were found to be explained by the presence or absence of a Referee Committee member or Officer from the same nation.
We have therefore found robust evidence for an effect of whether a nation is represented in the UEFA Referee Committee on how highly this nation is represented in the UEFA Referee Categories.
We used objective data to minimize subjective sentiments and to avoid speculation.
We have included the possibility that Referee Committee members are usually from the bigger footballing nations with an equally good level of refereeing. By intergrating the UEFA association coefficients into the statistical model, we can say that the effect of the Committee presence still exists. The nations' football level operationalized by the association coefficients obviously has an effect on the categories, but only a smaller one.
Implications and Limitations
Of course you can question whether the distribution of points to the categories (4, 3, 2 and 1) is reasonable. You can equally question whether someone like Mark Clattenburg really needed David Elleray to become a world-class referee. Most likely, the real meaningful effects depicted in the results of our calculation are rather located in the lower area of the Elite Group and particularly in the two or three lower categories.
A criticism on the procedure might be that some nations such as England, France, Italy, Germany and Spain are de facto not able to have a referee in Third Group as their officials usually start in Second Group as soon as they become international officials. It is however unprobable that this majorly influenced the effect found.
In addition it might be that those nations with a committee member are generally characterized by a higher level of refereeing and referee education which maybe even enabled certain people to enter the UEFA committee.
What we did not consider in our study are the roles some Referee Committee members fulfill in foreign referee departments (e.g. Marc Batta in Belgium, Pierluigi Collina in the Ukraine, Hugh Dallas in Israel, Jaap Uilenberg in Turkey until 2016 etc.). Unfortunately, we do not have enough information to consider this as an additional variable. But the ranking posted above partly suggests that some of the mentioned nations are not that badly represented in the categories.
UEFA can nonetheless use these findings to critically reflect themselves in terms of whether they (un)consciously favour referees from certain nations - and in this concrete case, from the commitee members' nations.
More focus should be shifted on officials from countries that are not represented in the committee. UEFA has the chance to do that on Thursday. Szymon Marciniak from Poland or Milorad Mažić from Serbia are good examples that this might pay off in the long run. UEFA's focus on referees from "smaller" nations like Azerbaijan (Aliyar Aghayev), Lithuania (Gediminas Mazeika) or Latvia (Andris Treimanis) is a good first sign, but rather the exception.
It should not be doubted that those referees who reached the highest referee categories mostly are excellent officials and widely deserved to get to where they are! However, it can be assumed that there are numerous match officials from other nations who are underrepresented in both the referee committee and - maybe even for this reason - are equally underrepresented in the category system and would never get sufficient chances to develop and climb the career ladder.
In a future article, we are going to focus on a similar question taking into account influences on referee appointments as another dependent variable.