New year, new rating system

For the last year or so, this website has been using its own BAINISTEOIR system to rate the inter-county hurling teams. BAINISTEOIR is a modified Elo rating system; the Elo system updates a rating following a game based on the opponent’s rating, and the result of the match. The better the team you beat, the more your rating improves, and the worse the team you lose to, the more your rating drops. BAINISTEOIR factored in opponent quality, but also adjusted based on home advantage, how long it had been since both teams played, and how much a team won by. All this produced a solid rating system that outperformed a basic Elo rater, in terms of how often it correctly predicted the winner of a match. However, no system is without issues, Therefore, for 2019, we are unveiling BAINISTEOIR, version 2.

The new system

BAINISTEOIR v2 works in a fundamentally different manner to the previous system. Whereas before the Elo rating was the basis, and adjustments were made based on how much teams won or lost by, the new system has the team’s expected score as the basis, with adjustments based on Elo rating. Odds of winning matches are based on Poisson distributions of how many goals and points each team are likely to score and concede.

To determine the expected scores, BAINISTEOIR v2 assigns each team an offensive and defensive rating. This is based on a weighted average of the team’s goals scored, points scored, goals conceded and points conceded, with more recent games being weighted more heavily. These values are also adjusted based on the team’s Elo rating, the opposing team’s Elo rating and home advantage.

Whereas, with the previous system, most of the how good or bad a team is was based on a single value, the BAINISTEOIR rating, there are now three numbers of note: The Elo rating, the Offensive rating, and the Defensive rating. These different values help to capture more information about each team, and by using them in combination, we get an idea of which teams may be overrated or underrated according to Elo, as well as which teams might be in the middle of an uncharacteristic winning or losing streak, but might return to their normal form soon. The chart below displays each division 1 team’s offensive and defensive ratings, with the bubble sized according to their Elo rating:


chart.png

Elo rating

The Elo rating is based on the traditional Elo formula. It’s a unitless number, but the higher the rating, the better the team. Behind the scenes, this is also modified based on how the team performs when playing at home, away or at neutral grounds. For BAINISTEOIR v2, an average team has a rating of 1500. For some idea of scale, at the beginning of 2019, the average rating for each division in the league are roughly as follows:

  • Division 1A: 1750

  • Division 1B: 1640

  • Division 2A: 1490

  • Division 2B: 1450

  • Division 3A: 1385

  • Division 3B: 1290

Offensive rating

The offensive rating is how much you’d expect this team to score against an average (Elo rating = 1500) team. This is a combination of the team’s expected goals and points scored. The average team at the beginning of the 2019 season would be expected to score 22 points against the average opponent. When calculating the odds of one team beating another, it is adjusted for the relative Elo ratings, and  is averaged with the opponent’s expected goals and points conceded, with each value weighted based on how much it varies from game to game. Again, for scale, at the beginning of 2019, the average offensive rating for the different league divisions are as follows:

  • Division 1A: 33.3

  • Division 1B: 27.8

  • Division 2A: 20.6

  • Division 2B: 20.8

  • Division 3A: 15.6

  • Division 3B: 12.1

Defensive rating

The defensive rating is how much you’d expect this team to concede against an average (Elo rating = 1500) team. This is a combination of the team’s expected goals and points conceded. The average defensive rating across all teams is equal to the average offensive rating across all teams, and is roughly 22 points at the start of 2019. When calculating the odds of one team beating another, it is adjusted for the relative Elo ratings, and  is averaged with the opponent’s expected goals and points scored, with each value weighted based on how much it varies from game to game. At the beginning of 2019, the average defensive rating for the different league divisions are as follows:

  • Division 1A: 16.1

  • Division 1B: 19.0

  • Division 2A: 22.0

  • Division 2B: 22.4

  • Division 3A: 25.8

  • Division 3B: 28.7

Old versus New

There are a number of advantages to the new system over the old. Over the entire database, the old system was able to correctly identify the winner of the match about 68% of the time. The new system predicted the winner in 73% of matches.

A consistent issue with the previous BAINISTEOIR system was handling the relatively small number of games per year. It sometimes took a very long time for the rating to catch up with the actual performance of the team. In 2018, for example, the BAINISTEOIR system had Galway as overwhelming favourites to win, because Limerick’s rating was still catching up after several years of under performance. Teams that rise quickly, such as Limerick in 2018, Clare in 2013 or Cork in 1999, cause big issues for the old rating system, as it simply can’t tell the difference between a genuine improvement and a flukey win until lots of games have been played. Similarly, when teams have big drop offs in quality, such as the injury-plagued Waterford panel last year, it can overrate them for a long time. Another issue is teams underperforming in 50/50 situations. A perfect example is Dublin in the 2018 Leinster championship. They only won one game, but had a positive score difference at the end of it, and with the smallest change in fortunes could have easily come away winning all four matches. The previous rating system had no choice but to keep lowering their rating, despite the fact that it was clear that they were capable of going toe-to-toe with top tier teams like Galway, Kilkenny and Wexford.

The new system, however, deals with this in a much better way. Because it weights recent performances more heavily, it can respond to a team suddenly putting up better numbers against strong opposition. Though their Elo rating may not catch up as quickly, when predicting match results it can overrule which team has the higher Elo rating, and favour the up and coming team. Going back to the example of Dublin above; at the end of 2018, though their Elo rating is lower than most of the other teams playing in the top tier, the new system gives them an offensive rating of 29 and a defensive rating of 18, clearly displaying that they’re not necessarily heavy underdogs against top opponents. On the other side of things, even though Kilkenny still have the highest Elo rating after years of consistent performance, their expected scoring margin is only the fourth best, indicating that they’re not quite as dominant as their Elo rating implies.

Just as the offensive and defensive ratings help to reign in Elo, the Elo rating helps to prevent getting too carried away with scoring margins. Even though teams like Donegal put up some monstrous scorelines in last year’s Nicky Rackard cup, the new system doesn’t expect these dominant wins over teams like Warwickshire and Tyrone to translate into big wins over the likes of Kilkenny or Tipperary.

Another advantage of the new system is that, because it’s based around scoring, it can give a better prediction of not only who will win but how they’ll do it; the old system could give a crude prediction of the expected winning margin for a match, but the new system can also predict how many goals and points each team should score. Obviously, hurling is too unpredictable for these to be very precise predictions, and a lot of variance is expected, but over the course of the season, these errors should average out and it should prove reasonably robust.

Going forward

As before, following each weekend of hurling action in the league and All-Ireland, each team’s ratings will be updated on the ratings page. Over time, the rating history for each team will also be updated to the new system. Whereas before this only displayed the teams’ BAINISTEOIR ratings, it will now display their Elo, offensive and defensive ratings. In advance of each round of the league and championship, it is also intended that the system’s predictions for each match will be displayed, as well as how this affects its overall prediction for the championship as a whole.