The rating system has been a fixture since the beginning of this website. However, it had several limitations; for example, the database it used only went back to 2012, it didn’t factor in score margins, and it failed to adjust for the extra uncertainty following a long gap between matches. In short, changes needed to be made, from this simple, slightly modified Elo system, to something better. After lots of work, the new Take Your Datapoints rating system is finally ready to be revealed: BAINISTEOIR (BAINISTEOIR’s Acronym Is Needlessly Intricate; Statistically Trained Estimator Of Iománaíocht Ratings)
BAINISTEOIR is the new rating system for the site, and it involves a total overhaul of how each team is rated. The previous iteration of the rating system only checked whether a team had won, lost or drawn, the rating of its opponent, and whether they were playing at home, away or at a neutral venue. How much weight it gave to both these variables into its equation for a new rating was based, for the most part, on what ‘felt right’.
Like the previous version, BAINISTEOIR factors in whether a team won, lost or drew, how good its opponent was, and where they were playing. However, it also factors in the expected scoring margin versus the actual result, and how long it’s been since the teams played. Additionally, it will revert a team’s score towards the mean for its tier at the end of each year, in order to prevent teams from maintaining excessively high ratings from past performances, thereby getting more accurate ratings for the current moment.All these factors were weighted through the use of a genetic algorithm, to ensure that it produces the most accurate results possible.
The most important change, however, was to the database.
The new database
The old database was fairly comprehensive for the time period it covered, but that time period was extremely limited. The new database contains records of matches going all the way back to 1887, the year of the first All-Ireland championship. This includes games from every division of the league, Munster, Leinster, Ulster (including the Ulster shield), and the All-Ireland, as well as the now defunct All-Ireland B Championship, Connacht championship and the Oireachtas cup. It includes not only every currently active team, but also the records for teams that competed in the past, such as New York or Glasgow. While the old database covered roughly 800 matches, this new version contains, at the time of writing, 5,705 games, taken from a variety of sources. This includes the date of the match, the teams involved, whether the match was at one team’s home grounds or at a neutral venue (if known), and, of course, the score. As well as improving the rating system, it is hoped that this new database will make a variety of future articles possible, as there is now much better access to historical data.
Unfortunately, one drawback is that the database is not complete for all of these years: While it’s easy to find comprehensive records for recent games, you only need to go back a few years before it becomes difficult to find the results for lower tier games, such as the lower divisions of the league. Before long, even more significant games begin to go missing. The chart below indicates how many records there are for each year, and it’s easy to see how difficult it becomes to find details of older matches, especially once you get any further back than the 70s.
This, combined with the smaller number of inter-county games per year, means that ratings are often closer in the early decades than in more recent years. This means that a team who wins an All-Ireland in recent years is more likely to have a better rating than one who did the same thing in the 50's. At the very least, attempts were made to include the results of finals in a given year. However, many early games are extremely difficult to pin down: even if a score can be found, sometimes the exact date, or venue, can’t be. As such, I’d like to remind any readers who may have corrections, additions or suggestions for the database that they can get in touch at firstname.lastname@example.org
As with the old version of the rating system, teams have additional ratings for how they perform at home, away or in neutral venues. Though this improved the ratings, some teams ended up with certain ratings getting very extreme values. Galway, for example, played very few home games, due to playing in Leinster. As such, their neutral rating got much larger than it perhaps should have. Though they performed well on neutral venues, especially this year, the system was often overconfident in their abilities. You also ended up with teams, typically those playing against either much stronger or weaker opponents, who would end up with positive or negative ratings in each category. This went against the aim of the home advantage system, as it was only meant to account for their relative improvement or disimprovement based on where they play. As such, the new system has modified this, so the home/away/neutral ratings are relative to one another. An improvement in one will result in a disimprovement to the others, and the sum of all three should add up to zero. Therefore, a team that wins or loses all the time, regardless of venue, will have all three ratings close to zero. A team with a runaway home rating now actually means what it’s meant to: this team wins more at home than would be expected, and loses other games more than would be expected.
A previous article on this website regarding scoring margins finished with inconclusive results. It was found at the time that there was no definite correlation between a team’s winning margin and how much better they were than their opponents. However, with the massively expanded match database, a more definite trend could be found. This investigation also took into account how scoring itself has changed, as the value of a goal varied in the early years of the sport.
On average, when the system predicted the expected odds for a team, for roughly every 3-4 percentage points above 50%, a team would increase their winning margin by a point. Obviously, there is a lot of variation in this, but overall the correlation was found to be strong enough that it was included in BAINISTEOIR. If a team had a better winning margin than expected, they were given a slight boost in their rating, relative to how much they exceeded expectation (or, for a losing team that kept the game surprisingly close, their rating wasn’t reduced by as much as normal). Similarly, a team that wins by less than expected will have a smaller than normal rating increase, and a team than loses by more than expected will see a bigger hit to its rating.
As well as improving the predictive ability of the ratings, this change has other, interesting implications: rather than just giving odds of how much a team will win by, BAINISTEOIR will also be able to provide a projected point spread for a given match, as well as a better idea of the likelihood of a draw. There may also be the potential for live updates on a team’s odds of winning as a match progresses.
An issue faced by the old rating system was how slowly it would adjust to a team improving or disimproving, especially if their previous rating was very high or low. Kilkenny, for example, performed very poorly this year, exiting the league at the quarter final stage and the All-Ireland in round 2, and netted a losing record over the course of the year. However, the old rating system still had them as the second best team around. A solution to this, adopted by BAINISTEOIR, and which is already used by many other Elo-based systems, is to slightly revert scores back to the mean at the end of the year. This system also reduces the inflation or deflation of ratings suffered by the system as teams are added or removed. Each team has its rating slightly increased or decreased towards a mean value before its first match each year. This reflects the increased uncertainty created by how a team has changed in the off-season. Though teams should still have the same ranking, their odds of winning or losing against other teams will decrease. This has the added benefit of kick-starting the system in adapting to a team’s change in rating.
Hurling presented an extra issue for this, however: there is a huge gap between those at the top and the bottom. Having every team revert to the same value resulted in the rating system giving odds which were too high for very poor teams, or too low for very good ones. There is less of a steady gradient in quality in hurling than many other sports: currently, any of the top 8-10 teams could beat one another on a given day, but would almost never lose to anybody below that range. Similarly, the next 8-10 teams would be competitive within themselves, but almost never lose against those below them. In order to reflect this, it was decided that each team would revert to a score within its own tier.
Tier one teams are those who most recently played in division 1A or 1B of the league, the All-Ireland, Leinster, Munster or, when it was played, the Oireachtas cup. Their scores trend towards a rating of 1750. Examples include Galway, Wexford, Limerick and Offaly.
Tier two teams are those who most recently played in division 2A or 2B of the league, the Christy Ring championship, the Ulster championship or, when they was played, the Connacht championship and All-Ireland B championship. Their scores trend towards a rating of 1500. Examples include Antrim and Wicklow (though Antrim will begin playing in tier one in 2018 with their promotion to division 1B of the league).
Tier three teams are those who most recently played in division 3A or 3B of the league, the Nicky Rackard championship or the Ulster Shield. Examples include Tyrone and Louth.
Tier four teams are those who most recently played in division 4 of the league (when it existed) or the Lory Meagher championship. Their scores trend towards a rating of 1000. Examples include Cavan and Lancashire.
The average team from one tier would be expected to beat the average team from the tier below more than 80% of the time. They would be expected to beat a team two tiers below roughly 95% of the time, and a tier one team should beat a tier four team almost 99% of the time.
This tiered system is an attempt to strike a balance between preventing teams’ scores getting too extreme or having them all trend towards unrealistically similar values. It also sets up the system to work more on a system of ‘how good are these teams relative to their peers?’, rather than ‘how good are these teams relative to all-time?’. It was felt that this was important, as hurling has become such a different sport over the decades, that trying to compare present teams to those of the past would be futile.
One further question is what to do about teams who haven’t played in a long time. The database has gaps, especially in the early years, and even recently teams like Cavan have re-entered senior hurling after several years away. If a team like Glasgow decided to compete again, should they have a tier one rating because their last recorded match was in the All-Ireland, even though it was more than 100 years ago? This would result in very inflated ratings for any them, or team who beat them, while their rating readjusted. To combat this, any team whose last fixture in the database was from more than a year ago has their rating reset. A team enters at the value for the tier of the competition it plays in, with no home, away or neutral advantage. Cavan, for example started playing again this year for the first time since 2012. When they stopped, they had a rating of 832. This year, upon entering the Lory Meagher competition, they had their rating reset back to 1000 before their opening match.
One final change was made to reflect a team’s rustiness. The GAA calendar is highly irregular, and teams can vary between playing week on week at the height of the league, to having month-long gaps in the later stages of the All-Ireland. These long gaps introduce uncertainty about how a team will perform, and so the predicted odds are narrowed the longer the teams have gone without playing. This adjustment is slight, and has upper and lower limits (in other words, a team that hasn’t played in two months has the same adjustment as one who hasn’t played for three), but it helps to prevent the system getting overly confident, or overly pessimistic, about teams who haven’t played in a long time.
Finally, the important business: how will the website change? The Ratings section of the site has undergone an overhaul. As before, you can see a table of how each team is rated, and their odds for the current league or championship, once those get underway. Another feature which will be available when the new season starts is a page on upcoming match predictions, giving odds for each team to win, and by how much. In order to make use of the expanded database, historical ratings can also now be viewed for each team by clicking on them in the rating table. You can see the highs and lows of every team, when they did or didn’t win their leagues or All-Ireland championships, and see which teams under- or over-performed in terms of silverware. Hopefully you’ll enjoy exploring these new features, but here are some interesting points related to the rating histories to get you started:
I’m sure that you’ll be able to find plenty more notable pieces of information browsing through the website’s new rating system. No new entries will be added to its database from now until the beginning of next year’s hurling league, though once the 2018 fixtures have been announced, then work will begin on adding predictions for that year’s competitions. Following the 2018 All-Ireland, it’s hoped that further updates to the algorithm, and possibly to the database, will be made, and the BAINISTEOIR system will continue to improve.