Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

Hey all,

I just finished doing some statistics on the latest Warzone Atlanta. People have expressed interest in this kind of thing in the past, so here it is:

http://variancehammer.com/2017/11/16/number-crunching-warzone-atlanta-2017/

TLDR: Good players are good, Chaos is quite strong right now, Ynnari are rare but solid, Grey Knights are trash, and Codex > Index.

Certainly a VERY interesting read – thanks for spending the time to put it together!

I will definitely be interested in see this develop as more major tournament data is added in (where possible).

yeah it was a good write up.

Excellent analysis. Thanks for putting in the effort for this.

Great work, well worth a read.

Thanks!

I'm statistically illiterate so I don't understand a lot of the information gathered but I have a couple of questions.

1) How do you deal with "soup" armies in terms of where you factor them into faction statistics? (For example I see lots of GK armies with only a GK warlord and no other or little actual GK units/models.)

2) If I'm reading your score chart properly GK players have to fight their way to 0 let alone a positive score. Is this correct and could you explain in simple terms what the point value statistic reflects?

Thank you for your answers and patience.

Leo_the_Rat wrote:
I'm statistically illiterate so I don't understand a lot of the information gathered but I have a couple of questions.

1) How do you deal with "soup" armies in terms of where you factor them into faction statistics? (For example I see lots of GK armies with only a GK warlord and no other or little actual GK units/models.)

2) If I'm reading your score chart properly GK players have to fight their way to 0 let alone a positive score. Is this correct and could you explain in simple terms what the point value statistic reflects?

Thank you for your answers and patience.

Regarding 1), I was also wondering that. Wasn't the Blood Angels army predominantly Guard, for example?

Good analysis and an enjoyable read.

There isn't an easy way to score "soup" armies as that would entail understand how the player utilized each of the different armies in comparison to how many points worth of each army was in the composition.

Far too complex to put into simple math.

Usually all you can do is nominate which codex the Warlord is from or which codex has the highest % of points in the army.

This was a good analysis. However, I'd be interested in knowing how Marine armies without Guilliman fared, as there has been a lot of talk of him being the sole thing that keeps that codex afloat.

Leo_the_Rat wrote:I'm statistically illiterate so I don't understand a lot of the information gathered but I have a couple of questions.

1) How do you deal with "soup" armies in terms of where you factor them into faction statistics? (For example I see lots of GK armies with only a GK warlord and no other or little actual GK units/models.)

2) If I'm reading your score chart properly GK players have to fight their way to 0 let alone a positive score. Is this correct and could you explain in simple terms what the point value statistic reflects?

Thank you for your answers and patience.

1. "Soup" armies will be a problem, and always will be. For the purposes of this, we're using the ITC faction definitions, which are:

"Your ITC faction is determined as follows: find the the most specific Faction Keyword (which is typically the last Faction Keyword listed on each unit's’ datasheet) shared by every model in the detachment with the most points in it in your army. Consult column B in the chart above. Your ITC faction is the corresponding ITC Faction Keyword in column C.

Example: you have 3 detachments in your army. The 1st is 400 points, the 2nd is 1,000 points and the 3rd is 600 points. The largest detachment is the 2nd. The most specific Faction Keyword shared by all models in that detachment is Ultramarines. Your ITC Faction for your army is Adeptus Astartes and your ITC points will be awarded to that faction and you will compete with all other players whose army corresponds to that ITC faction keyword."

It's complex, admittedly. But it tries to get at the "thrust" of the army a little bit, and ignores the "Technically the GK is my Warlord in a sea of conscripts" problem.

2. That's how much they're predicted to add or subtract from the average score. Basically, the model predicts everyone starts with a score of 76.9 (the average). GKs subtract 1.7 from that, giving them a predicted score of 75.2 just for the army choice. Dark Eldar on the other hand add 22.1, giving them a predicted score of 99 just for the army choice.

Does that make sense?

HuskyWarhammer wrote:

Leo_the_Rat wrote:
I'm statistically illiterate so I don't understand a lot of the information gathered but I have a couple of questions.

1) How do you deal with "soup" armies in terms of where you factor them into faction statistics? (For example I see lots of GK armies with only a GK warlord and no other or little actual GK units/models.)

2) If I'm reading your score chart properly GK players have to fight their way to 0 let alone a positive score. Is this correct and could you explain in simple terms what the point value statistic reflects?

Thank you for your answers and patience.

Regarding 1), I was also wondering that. Wasn't the Blood Angels army predominantly Guard, for example?

See above. The Blood Angels army was Dante, some Stormravens, and lots of Scions. While that "feels" like a Scion army, the largest detachment was the spendy Dante + Flyers, and so it's a Blood Angels army per ITC.

It's *really* hard to drill down past there without immediately running into small numbers problems. I'm working on some ways to capture that, but it would involve:

1. A *ton* of work
2. Full access to army lists
3. Lots and lots of data.

Crimson wrote:This was a good analysis. However, I'd be interested in knowing how Marine armies without Guilliman fared, as there has been a lot of talk of him being the sole thing that keeps that codex afloat.

With only eight armies total, my guess is unless non-Guilliman armies are *disastrously poor* it would be hard to distinguish from noise. This is always the problem with sub-faction analysis of tournament data - there's just not big enough data sets.


Automatically Appended Next Post:

 Eihnlazer wrote:
There isn't an easy way to score "soup" armies as that would entail understand how the player utilized each of the different armies in comparison to how many points worth of each army was in the composition.

Far too complex to put into simple math.

Usually all you can do is nominate which codex the Warlord is from or which codex has the highest % of points in the army.

Yeah - I have some ideas about how to capture it using things like information theory and diversity indexes from ecology, but it rapidly heads into "Yeah, but I have a job already..." territory.

Good read, well thought out. Although, I have to disagree with your conclusions. Saying 8th hasn't brought better balance to the game when more than half the codices haven't been released doesn't make much sense. On top of that, 7th's formations were all across the board in terms of balance, making it virtually impossible for certain armies to even compete. The same can be said for 8th, but only when you are comparing index armies to codex armies. While some codices are better than others (which is inevitable), they are certainly more competitive than the codices in use at the end of 7th. But, you can only work with the data you have so far and it was interesting to see how things are progressing.

 VarianceHammer wrote:
Hey all,

I just finished doing some statistics on the latest Warzone Atlanta. People have expressed interest in this kind of thing in the past, so here it is:

http://variancehammer.com/2017/11/16/number-crunching-warzone-atlanta-2017/

TLDR: Good players are good, Chaos is quite strong right now, Ynnari are rare but solid, Grey Knights are trash, and Codex > Index.

What a shocker Tau over nerfed form 7th. Been saying this since 8th dropped. Man watching people catch up to this stuff is painful. I feel like the world is in slow motion in the 40k scene.

Poor Dark Eldar also Deathwatch I imagine are all bad facitons. All the 40k armies I collect are bad. :(

Chaos and Imperial armies on top like in 7th edition. Even more armies are less viable. 8th sucks man. I haven't played it since I started. I knew it was a bad edition... for comp play. In casual games as usual things seem okay but the same could more or less be said for 7th. The only reason everyone thinks balance okay is because IoM and chaos is dominating and they don't have to fight any other factions seriously with their over powered win armies.

It is strange, I heard so many times that 7th was a mess, but here it seems to be much more balanced when viewing the results.

A good read, but your conclusions seem a little doom and gloom.

Looking at whether or not 8th is balanced should be reserved for when every faction receives their specific codex. You yourself mention the benefit of a codex and the disparity of codex to non-codex wins/losses, so saying 8th isn't as balanced as 7th seems like a VERY knee jerk reaction.

Also, I'm a little perplexed with your two graphs of army representation. You sat that 8th has done little to balance the factions and there will be strong contenders, but your graph of 7th has MUCH less armies than 8th (even though it's only a single instance). Surely this is an example of 8th giving a chance to other armies? I freely admit this could be due to 8th editions newness, but seeing so many different armies is heartening.

Lastly, I applaud your level headed and neutral tone throughout the write up. Even though I disagree with your conclusion, you wrote it well and it allowed further discussion rather than being gospel that cannot be questioned.

Keep up the good work!

HuskyWarhammer wrote:

Leo_the_Rat wrote:
I'm statistically illiterate so I don't understand a lot of the information gathered but I have a couple of questions.

1) How do you deal with "soup" armies in terms of where you factor them into faction statistics? (For example I see lots of GK armies with only a GK warlord and no other or little actual GK units/models.)

2) If I'm reading your score chart properly GK players have to fight their way to 0 let alone a positive score. Is this correct and could you explain in simple terms what the point value statistic reflects?

Thank you for your answers and patience.

Regarding 1), I was also wondering that. Wasn't the Blood Angels army predominantly Guard, for example?

No, iirc it was about 2/3 blood Angels, though of course that's stormravens and assbacks. The only blood angel model was the chapter master fellow.

It's going to be very hard to tease out exactly what's going on with the strong correlation between success this year and success last year. Like, obviously a lot goes into this beyond a player's skill at making in-game decisions. All players are constructing lists so as to have the best chance of winning. Many of them are even choosing their faction based in part on what they think will be most successful. If successful players are successful in part because they're good at identifying the best factions to bring, you're going to be underestimating faction imbalance when you control for players' past success. For example, as far as I can tell Andrew Whittaker brought Blood Angels to last year's WZA. Maybe he did well both years in part because he picked a strong faction each time. Likewise there are obvious issues arising from list composition within factions, since no one thinks that all armies from a faction will be equally good. Though of course you can't do much with this mathematically.

It was an interesting read, and good work, but I want to caution people against the naive assumption that you're controlling for generalship only when you control for past success.

OK, I am a lawyer by trade so means, standard deviation, etc., mean not a jot to me.

Please could someone let me know where DE Covens (mono) and R&H placed/performed.

Also, kudos OP for putting in the effort. I am sure much more knowledgeable guys and gals than myself will get a lot from your work.

Dionysodorus wrote:
It's going to be very hard to tease out exactly what's going on with the strong correlation between success this year and success last year. Like, obviously a lot goes into this beyond a player's skill at making in-game decisions. All players are constructing lists so as to have the best chance of winning. Many of them are even choosing their faction based in part on what they think will be most successful. If successful players are successful in part because they're good at identifying the best factions to bring, you're going to be underestimating faction imbalance when you control for players' past success. For example, as far as I can tell Andrew Whittaker brought Blood Angels to last year's WZA. Maybe he did well both years in part because he picked a strong faction each time. Likewise there are obvious issues arising from list composition within factions, since no one thinks that all armies from a faction will be equally good. Though of course you can't do much with this mathematically.

It was an interesting read, and good work, but I want to caution people against the naive assumption that you're controlling for generalship only when you control for past success.

The problem is that you either have to control for players past successes, or you don't.

If you don't, then whatever army they're playing will appear to be OP, as their skill is better with it, even if the game is perfectly balanced. (So, in a hypothetical game where balance is 100% achieved, then your factions will still appear imbalanced if you don't control for player skill, because the best players play their factions of choice).

If you do, then you get a better picture of balance because the best players won't overinflate an otherwise balanced faction's numbers. But, as you said, people choose factions based on what they think is OP, so... *shrug*

Honestly, if we're already down to the point where the difference between an army appearing balanced or not is whether or not you control for player skill, then I'd say the game is probably pretty darn balanced; after all, if player skill (or not) swings the outcome, then it wasn't the faction that did it.

 Unit1126PLL wrote:

Honestly, if we're already down to the point where the difference between an army appearing balanced or not is whether or not you control for player skill, then I'd say the game is probably pretty darn balanced; after all, if player skill (or not) swings the outcome, then it wasn't the faction that did it.

Part of what I was saying is that we can't actually tell if we're at this point, unless we use an implausibly broad definition of "player skill". We don't actually know what things look like if we control for "player skill" as most people would understand it.

You should probably also keep in mind that the sample size is sufficiently small here that an army "appearing balanced" is not strong evidence that it is actually balanced.

Dionysodorus wrote:

 Unit1126PLL wrote:

Honestly, if we're already down to the point where the difference between an army appearing balanced or not is whether or not you control for player skill, then I'd say the game is probably pretty darn balanced; after all, if player skill (or not) swings the outcome, then it wasn't the faction that did it.

Part of what I was saying is that we can't actually tell if we're at this point, unless we use an implausibly broad definition of "player skill". We don't actually know what things look like if we control for "player skill" as most people would understand it.

You should probably also keep in mind that the sample size is sufficiently small here that an army "appearing balanced" is not strong evidence that it is actually balanced.

True. Also you made me open another thread (you perhaps saw) about whether or not army choice/listbuilding is "player skill" or not. Because if it is then... woo the game is balanced? XD

Thank you for your explanations. I appreciate the effort you're putting into this project.