Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

First off, I want to thank Peter for all his work curating the database at 40kstats.com

I wanted to share some of the analyses i've started of the data covering the entire 2018 tournament season (you can find the data here). Peter and the folks at Chapter Tactics have mainly talked about means and trends in these data, but have not (to my knowledge) discussed any formal statistical analyses.

My first slice at these data is looking at primary faction win rates. Obviously, list building is much more complex than this, and looking at just primary faction is a limitation of these analyses.

You can see the technical methods details for these analyses here:




Here you can see the model estimated means of win percentages by faction, plotted alongside the observed means from the raw data. As you can see, the model recreates the observed means closely - except the observed means are likely overestimated, probably because the simple means aren't accounting for particular players driving the means up or down in contrast to the model. Also plotted on all the means are the confidence intervals - this gives us an indication of the precision of our estimate. Our best guess about the true mean out there in the world falls within that confidence interval. This is useful because the means themselves might be misleading alone. Genestealer Cults, for instance, have a high win percentage, but the 95% CI is actually quite large, so we can't be as confident about those results as we might be about a faction like Knights. After all, we have less data for Genestealer Cults than other factions.

There was a highly significant effect of Faction, F (23, 290) = 10.52, p < .001, indicating that some factions had significantly higher win percentages than others.


Here you can see the results of each individual mean comparison between factions. I corrected for multiple comparisons by using false-discovery rate (FDR). This table summarizes the significant differences (accounting for FDR) between each factions' win rates. You can read the table by finding your faction on the vertical axis, and comparing it to a faction on the horizontal axis. A negative value means your faction (on the y-axis) performs significantly worse generally than the faction (on the horizontal axis) and a positive value means your faction performs significantly better than that faction. The values themselves are the model-estimated difference in win-rate. Do note, this is not the how well your faction performs against the different factions. This is a comparison on mean win rates overall. The results are generally in agreement with what you might expect from eye-balling the means. However, there were also a lot of non-significant differences between factions - meaning we don't have evidence either way to know which faction is better than which. Non-significant comparisons are left as blank squares in the matrix.

Finally, I conducted a k-means clustering of the model estimated means of win rate by primary faction to see whether we can summarize these results based on statistical evidence for "rankings" of different factions. A 3 cluster (tier) solution was the best solution for the data. That is, adding additional "tiers" didn't explain a whole lot more variance over lower cluster solutions. The 4 cluster solution, for instance, just explains a little more variance than the 3 cluster solution, and splits Drukhari and Ynnari into their own top-tier ranking.


I interpret the clustering results as pretty hopeful, honestly. It means that armies generally fall into just 3 ranks. Poor, average, and good armies. Furthermore, an examination of the model estimated means shows that the vast majority of armies has a 95% CI that overlaps with a 50% win rate - which is the ideal win rate for an army if there is nice balance between factions.

Let me know if anyone has any questions, or ideas for additional analyses. This is really just a first pass at these data - as it's a really rich dataset.

I'm glad to have some statistical backing regarding how garbage my Grey Knights are. And now I can show my TSons friends that no, they are not even CLOSE to the same boat as me!

Yes! Virtually every faction performs better on average than Grey Knights at tournaments.

At the risk of complicating your calculations, did you adjust the data to exclude Mirror Matches (games where both players have the same Primary Faction)? Those will get an equal win/loss rate and can warp the numbers if there are enough in the sample.

No, this is an analysis of a players overall win rate over the course of tournament. It could very well have happened that mirror matches popped up, and this will have affected the overall win rates. Something like what you are suggesting could be done.

I figured as much. I imagine it would be a lot of work to pry those out of the numbers since they warp them to an extent when trying to do a faction to faction comparison. Not important on the player to player comparison.

Primary faction is 50% of the army's points value.

For any army capable of allying, the data for mixes armies versus mono faction is convolutes together.

It would be more interested to see data for only 100% mono faction added. Would allow to see which factions are/have Ally options that skew their results.

Currently that information is buried in the results you posted.

blaktoof wrote:
Primary faction is 50% of the army's points value.

For any army capable of allying, the data for mixes armies versus mono faction is convolutes together.

It would be more interested to see data for only 100% mono faction added. Would allow to see which factions are/have Ally options that skew their results.

Currently that information is buried in the results you posted.

Very few people who have the option to take allies play mono-faction in tournaments. Sure, it happens, but its certainly not the majority.

Of course it's not the majority, and that is the point.

Let's take drukhari for example, they are basically in second place in the graph above but that is not reflective of how powerful they are, but rather how powerful the Ally system is. If you can see the difference between the mono faction and primary faction result you can judge how much of that is based on some synergy combination outside of the primary faction.

Much like knights, mono knights doesn't do as hot as knights+AM

It's even more significant when 3 factions are involved since the primary faction is under 50% many of the times, so it's more representative of the Ally issue than the primary faction being powerful.

Problem is thus because not many do that there's not enough statistics for it...

Now see how many of those were soup. I'm getting a bit tired of this "Look how diverse everything else!" ITC propaganda when it's soup almost all the way down with the same components cropping up. That's not actually diversity.

Will the stats change for GK post CA2018?

 _SeeD_ wrote:
Will the stats change for GK post CA2018?

GK's are fundamentally flawed. More expensive marines that die like any other marine is just not good.

 Ordana wrote:

 _SeeD_ wrote:
Will the stats change for GK post CA2018?

GK's are fundamentally flawed. More expensive marines that die like any other marine is just not good.

That's what I'm afraid of too. I'm hoping for a reduction in CP costs within the next year. Maybe I should post a cool picture of GK on the warhammer40k subreddit every day until they change it