1 What is this Analysis?

This notebook is for examining current ratings from boardgamegeek.com with the aim of computing different versions of these ratings. The main goal is to compute alternative ranks to adjust for a tendency of the BGG ratings to skew towards complex and recently released games. This is not a novel contribution, but this notebook aims to explain some of the considerations behind calculating different ratings for games as well as refresh these ratings frequently off a live connection to the BGG database.

For further analysis and projects related to boardgamegeek data, see github.com/phenrickson/bgg

2 Examining BGG Ratings

We’ll load in data previously pulled from BGG and begin our analysis. There are a couple of different variables we’re interested in exploring: the average rating, the geek rating, and the average weight, or complexity. For the average and geek ratings, votes can range from 1 to 10.

The geek rating is the game’s average plus around 1000 baseline 5.5 votes added, which serve as a prior to prevent games with few ratings from moving to the top of the geek list.

Additionally, users can rate the complexity of games on BGG, where games with a complexity of 1 are very simple where games with a complexity of 5 are extremely difficult to learn.

If we look at the average rating of games on vs their complexity, we can see that a game’s rating is heavily correlated with its complexity.

We can also plot this with labels to pick out particular games.

The average rating is also a function of time, as we’ve seen higher ratings for newer games.

We can add the number of user ratings to size to see how that affects things.

2.1 Geek Ratings vs Average Ratings

Now, so far we’ve just looked at the average rating, which is simply the avearge of community ratings for a given game. Most people care about the Geek ratings, which uses Bayesian averaging to account for the fact that games with relatively few votes will tend to have high averages. The Geek ratings start every game off with 1000 votes at 5.5, which means that it takes a decent number of users rating the game to actually move the baseline rating.

If we size by the number of users rated, we see how games with good or bad Geek ratings require a significant number of user ratings. This is the intent of the Bayesian averaging, where it takes a lot of people having an opinion on a game to shift it away from the average (the prior).

We can also break this down by decade released to see how time drives this as well.

3 Complexity Adjusted Ratings

The geek ratings on BGG are highly influential, but they are heavily skewed towards complex games. This means that “normal people” will have a hard time using the games that are highly rated on BGG. If we look at the top 100 games on BGG, we can see that they tend to fall on the heavier side in terms of game complexity.

We want a list of games that isn’t so heavily skewed towards complexity. We want to “control for” the influence of complexity on the rating. That is, if we take the variation in the average ratings that isn’t explained by complexity, which games still have high ratings?

3.1 Fitting a Simple Regression

To do this, wwe need the residuals from a regression of average rating on complexity - these will be the variation in game ratings that are not explained by complexity. We will fit the model and inspect the results.

The intercept indicates the average rating of a game with a complexity of 0, which is kind of nonsensical. The coefficient indicates the effect of a unit increase in the complexity of a game on the average rating. Putting these two together tells us that a game with a complexity rating of 1 would have a rating of 5.81. A game with a complexity rating of 5, meanwhile, would have a weight of 8.18.

The model indicates that the complexity of a game explains about 26% of the variation in average ratings (R-squared, which simply is the correlation coefficient (R) we saw earlier squared). So it’s not the only thing that matters, but it has a pretty sizeable impact on the average rating and the corresponding geek rating.

We don’t really care about the model per se, we just want to get the residuals.

A positive residual in this case is a game that has a higher than expected average given its complexity. But, we can’t just adjust the ratings alone because it skews heavily towards games that have a highly inflated average rating due to a only having a handful of users.

We will adjust this using an approach similar to their Bayesian averaging methodology, adding 1000 ratings at the average of 5.5

3.2 Examining the Top Complexity-Adjusted Games

We can put this all together to now look at games that are rated highly after adjusting for the effect of complexity. We’ll look at the top 250 to keep it simple.

This gets us a list of games that is wildly different than before, and it’s a list of games that are much more palatable to “normal people”. In my mind, Crokinole taking the top spot makes a ton of sense and I will die on this hill.

3.3 Comparing Geek and Complexity Adjusted Ratings

Which games have moved up and down the most? We can look at the difference between the BGG Rank and the Complexity adjusted ranks.

We want to focus in on the games that see a positive and negative shift in particular. Let’s look at the games that are penalized the most.

This list makes a lot of sense to me - the ratings themselves are still pretty good for some of these games, but these are all very, very heavy games. A lot of Vital Lacerda showing up in here, which to me epitomizes the disconnect between Geek Ratings and ‘ratings for people who fun games’. The former is heavily slanted towards people who relish complexity, whereas the latter is slanted towards games that provide heavy bang for their buck in terms of their weight.

The list of games that go up the most is on the other hand very light, party games. Even with the boost from being simple, many of these games still aren’t rated that highly, though some notable ones (Monikers, MicroMacro, KLASK, Just One) end up near the top of the overall list.

3.4 Movement within BGG Top 100

Let’s restrict to games inside the BGG Top 100 and see how these games are affected.

It’s important to remember that this is meant to be a ranking list for someone who sees complexity as a negative moreso than a positive. Terra Mystica and Gaia Project might be great games for someone like me who is into the hobby, but would it rank highly for someone who isn’t that keen on games? Probably not.

4 Complexity and BGG Ratings over Time

How has the relationship been complexity and average rating changed over time?

4.1 Complexity and the Average Rating

We can plot complexity versus the average BGG rating by every year since 1970.

Or, alternatively, let’s make one plot but then color the year.

We can fit the same model to each individual year to see the relationship between complexity and rating has changed by the year the game is published. We’ll plot the coefficient from a linear model fit to each year.

We can similarly examine the strength of this relationship over time by plotting the variation explained by complexity in each year (the R squared)

If I’m thinking about this correctly, this would mean that complexity has mostly had the same impact on the Average Rating over time, though post 2000 we might be seeing a decrease in the effect of complexity. Now I’d imagine that that is because we see newer games with relatively few votes that get a pretty high average rating even if they aren’t all that complex.

4.2 Complexity and the Geek Rating over Time

What if we look at the effect of complexity on the geek rating?

We’ll follow the same methodology as before, now fitting a regression of the average and geek rating as a function of game complexity by publishing year. But we’ll also fit a model for the effect of complexity on the number of user ratings. We’ll then plot the coefficients over time.

Okay so this pretty much explains what’s going on with the skew towards complex and recently published games in the geek ratings.

It’s not that complexity games are getting higher ratings over time, it’s that complex games are attracting a larger share of the user ratings, particularly since the early 2000s. I would imagine that as the marketplace has become more and more crowded with games, the boardgamegeek community has started to consolidate around more complex games that emerge as the cream of the crop. Complex games get purchased more, attract more user ratings, and end up shooting up the Geek list.

5 Adjusting for “The Hotness” in BGG Ratings

The list of complexity adjusted games is pretty great for recommending games to a beginner, but it’s not as helpful for people who have been in the hobby longer. If you do like more complex games, what list should you look at? The Geek list as it currently stands is still probably good, but its not without its problems.

The Geek Ratings are heavily skewed towards games that have been released in recent years: all of the top 10 games were released after 2015, and only 5 of the top 50 were released prior to 2010. Either we truly seeing the pinnacle of game design in the last 10 years (this is possible), or it means that the BGG users have a tendency to seek out and rate the hotness, which skews the list towards recent games.

Why do newer games so quickly climb the geek list? This is basically due to the fact that BGG uses 1000 votes to start its Bayesian average. This was probably a good prior for when board games had a much smaller audience, but as BGG and the hobby has grown, newer games rapidly attract enough user ratings that they manage to quickly overtake this prior.

If we plot the average and median user ratings for games published in each year, we can see how more recently published games draw lots of user ratings (though this does start to taper off for games in the last 1-2 years, though we would expect this to go up as these games accumulate user ratings).

This isn’t a problem per se, but it does mean that two games might actually be pretty similar in quality, but the one released more recently draws lots of user ratings, and so it gets a big boost to its rating on the Geek list.

5.1 Varying the Number of Baseline Votes

Fortunately for us, we can adjust for this in a pretty simple way by upping the number of baseline 5.5 votes for every single game. Rather than using the 1000 or so that BGG uses, we’ll toggle the number of baseline ratings at various thresholds up to 100,000 votes and see how the rankings start to change. Games that maintain a high ranking at each threshold will be games that have both a large number of user ratings and a high average. Games that are sensitive to the number of baseline user ratings will be ones that are ‘inflated’ in the current ranking system.

What do we find? There’s no getting around the fact that Gloomhaven is going to be a top game, it has an extremely high average with a lot of votes. The same goes for Terraforming Mars, which has an extremely high number of user ratings with a high average - if we set the number of baseline votes over 25k Terraforming Mars becomes the top game of all time.

There’s no correct number of votes to add, but it is interesting to see how games are affected by having more votes. If we add 100k votes we basically end up with a list that penalizes recent games to enter the top 100 (Gloomhaven Jaws of the Lion, Marvel Champions, Rising Sun, Underwater Cities, Dune Imperium) and provides a boost to some of the pillars of the board game renaissance Pandemic, Ticket to Ride, Pandemic, 7 Wonders, Puerto Rico, Agricola. Man, evidently Terraforming Mars is really good?

5.2 Top Board Games with 100k Votes Added

Adding 100k votes gives us the list of games that more or less maps to the evergreen games of the last two decades, and actually shows a decent balance of complex and simple games. I would argue that is is a stronger list to look at for people who are interested in the hobby but aren’t necessarily going to be interested in chasing the hotness.

We’ll store these lists on Github for everyone else to use.