Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

Hey folks,

Awhile back, I did a statistical analysis of the Throne of Skulls Tournament, for both Warhammer and Warhammer 40K. It was a fairly popular post, and having come back to the game from a long break, I decided to revisit the idea of looking at tournament results with statistics.

Since there's not many 7th Edition tournaments yet, I decided to look at the Las Vegas Open as a "6th Edition Retrospective". There's a series of four posts - an introduction, a look at how common allied armies are, a army-by-army breakdown of tournament performance, and then my closing thoughts:

http://variancehammer.com/2014/06/15/requiem-for-an-edition-an-introduction/
http://variancehammer.com/2014/06/15/requiem-for-an-edition-how-big-of-a-deal-are-allies/
http://variancehammer.com/2014/06/18/requiem-for-an-edition-army-breakdown/
http://variancehammer.com/2014/06/18/requiem-for-an-edition-a-couple-fun-ones-and-closing-thoughts/

That was some very interesting statistics, thanks for a job well done. It was very facinating to see how surprisingly balanced the game seems to be even in a tournament scene with all the complaining done on the internet.

Wow, a real eye opener, amazing just how overblown certain "issues" about 6th were.

How many tournaments was the study based on?

 Dantioch wrote:
That was some very interesting statistics, thanks for a job well done. It was very facinating to see how surprisingly balanced the game seems to be even in a tournament scene with all the complaining done on the internet.

Half the TOP 8 was comprised of some combination of Eldar and you call that balanced?

PhantomViper wrote:

 Dantioch wrote:
That was some very interesting statistics, thanks for a job well done. It was very facinating to see how surprisingly balanced the game seems to be even in a tournament scene with all the complaining done on the internet.

Half the TOP 8 was comprised of some combination of Eldar and you call that balanced?

Yes, I don't deny that eldar are an OP codex and that it certainly performed very well, the eldar faction as a whole didn't place itself markedly better than most other faction that participated. And I didn't actually say that the game was balanced, just look at how poorly space wolves and dark angels performed, it certainly is more balanced than many people on the internet claims it is, at least as far as this still quite limited study shows.

 Archon_Zarbyrn wrote:
Wow, a real eye opener, amazing just how overblown certain "issues" about 6th were.

LVO changed the missions, restricted allies (max 2 sources), banned escalation, lords of war and data slates, and IIRC changed core rules (2+ rerolls). And as PhantomViper says, the top tables were still dominated by certain armies/builds anyway.

So it's less a case for 6th ed balance, and more a case for how a lot of legwork and tweaks by TOs can help smooth out results somewhat - which is a good thing too, and something they'll need to look at more as GW moves to a more Laissez-faire attitude towards army building and balance. GW provides the options, TOs pare them back where needed.

 Maddermax wrote:

 Archon_Zarbyrn wrote:
Wow, a real eye opener, amazing just how overblown certain "issues" about 6th were.

LVO changed the missions, restricted allies (max 2 sources), banned escalation, lords of war and data slates, and IIRC changed core rules (2+ rerolls). And as PhantomViper says, the top tables were still dominated by certain armies/builds anyway.

So it's less a case for 6th ed balance, and more a case for how a lot of legwork and tweaks by TOs can help smooth out results somewhat - which is a good thing too, and something they'll need to look at more as GW moves to a more Laissez-faire attitude towards army building and balance. GW provides the options, TOs pare them back where needed.

Beat me to it.

The internet didn't really complain about Tau/Tau, there was mainly complaints about one Tau/Tau list in particular; O'Vesa star. A list that it's pretty obvious from your article that you are not aware of what that is. This highlights one big issue of your analysis; lack of qualitative data; you seem to lack an in-depth knowledge of what you are analysing. Clearly, Howling Banshee spam allied with a Wych Cult is not the same as a Beastpack, but your data treats it as the same.

The lack of control variables is another issue, and one that's related to the first. A dummy variable on netlists would be very helpful to your analysis to control for spuriousness. Especially when some of the armies in question are represented by so few data points. E.g., Eldar/Dark Eldar with 6 or 7 players. A single guy with a whacky themed list who spent the weekend getting trashed could throw that average right off. Of course, you don't have access to that data, but why not use standard deviation to address the issue?

There are also a few interesting viewpoints I think you should have included with the data you have available; top 16 vs. bottom 16, midfield performance, and analysis of the top 31 (everyone with one loss or less) to see how that contrasts with the entire field.

 Maddermax wrote:

 Archon_Zarbyrn wrote:
Wow, a real eye opener, amazing just how overblown certain "issues" about 6th were.

LVO changed the missions, restricted allies (max 2 sources), banned escalation, lords of war and data slates, and IIRC changed core rules (2+ rerolls). And as PhantomViper says, the top tables were still dominated by certain armies/builds anyway.

So it's less a case for 6th ed balance, and more a case for how a lot of legwork and tweaks by TOs can help smooth out results somewhat - which is a good thing too, and something they'll need to look at more as GW moves to a more Laissez-faire attitude towards army building and balance. GW provides the options, TOs pare them back where needed.

Fair enough, though I was mostly talking about the hysteria around the infamous Tau/Eldar army that was supposedly dominating the entire tournament scene. Though it a shame that TOs have to do so much work just to help the game more enjoyable for everyone participating you think GW who are being paid would do this themselves with balanced rule sets. Seeing so many adjustment makes me wonder if it would be possible for everyone who dissatisfied with the current state of the game to join together and make a rule set that provides both balance and variance between each of the armies.

Dantioch wrote:That was some very interesting statistics, thanks for a job well done. It was very facinating to see how surprisingly balanced the game seems to be even in a tournament scene with all the complaining done on the internet.

Archon_Zarbyrn wrote:Wow, a real eye opener, amazing just how overblown certain "issues" about 6th were.

Thank you!

Makumba wrote:How many tournaments was the study based on?

Just one. The hope for 7th Edition is to do a rolling analysis as things get published. I tried to get a hold of the Adeptacon 2014 data, but no dice.

PhantomViper wrote:

 Dantioch wrote:
That was some very interesting statistics, thanks for a job well done. It was very facinating to see how surprisingly balanced the game seems to be even in a tournament scene with all the complaining done on the internet.

Half the TOP 8 was comprised of some combination of Eldar and you call that balanced?

In fairness, there were a fair number of Eldar armies, and some of them were pretty bad too. But the Eldar are definitely a strong army. But the good armies were strong, not Snap Game Balance Over Your Knee.

Maddermax wrote:

 Archon_Zarbyrn wrote:
Wow, a real eye opener, amazing just how overblown certain "issues" about 6th were.

LVO changed the missions, restricted allies (max 2 sources), banned escalation, lords of war and data slates, and IIRC changed core rules (2+ rerolls). And as PhantomViper says, the top tables were still dominated by certain armies/builds anyway.

So it's less a case for 6th ed balance, and more a case for how a lot of legwork and tweaks by TOs can help smooth out results somewhat - which is a good thing too, and something they'll need to look at more as GW moves to a more Laissez-faire attitude towards army building and balance. GW provides the options, TOs pare them back where needed.

It was indeed some tuning from the LVO - I actually wanted to include that, as they were a major late 6th Edition tournament, and "the meta" over all is an impossible question, because it's so dependent on local-level factors.

Thud wrote:The internet didn't really complain about Tau/Tau, there was mainly complaints about one Tau/Tau list in particular; O'Vesa star. A list that it's pretty obvious from your article that you are not aware of what that is. This highlights one big issue of your analysis; lack of qualitative data; you seem to lack an in-depth knowledge of what you are analysing. Clearly, Howling Banshee spam allied with a Wych Cult is not the same as a Beastpack, but your data treats it as the same.

The Tau/Tau thing was mostly actually addressing that when people near my when nuts about All Riptides All The Time in unbound lists, and noting you could already kind of do this, and that having access to a second detachment of the same Tau army didn't really seem to be pushing things over the edge. O'Vesa star is another explanation, but regardless of the makeup of the list, I think "Double Tau armies don't seem to perform all that much better" is, in and of itself, an interesting thing to note.

As for the Eldar/DE lists, I do note that these are "potential Seer Star" lists, rather than out and out Seer Star lists because I myself run a Eldar/DE Corsair-themed army. So again, it was mostly "Does merging these lists together clearly make them better?" and in the case of those two books, the answer is yes, but it's not guaranteed.

The lack of control variables is another issue, and one that's related to the first. A dummy variable on netlists would be very helpful to your analysis to control for spuriousness. Especially when some of the armies in question are represented by so few data points. E.g., Eldar/Dark Eldar with 6 or 7 players. A single guy with a whacky themed list who spent the weekend getting trashed could throw that average right off. Of course, you don't have access to that data, but why not use standard deviation to address the issue?

If I could get at that data, I'd be happy to do some of that - for example, have a dummy variable for "Baron Present/Absent" in Eldar/DE lists. But, to be blunt, I don't have it.

I essentially *did* use the standard deviation, or more accurately, went beyond it. Each of the violin plots in the posts are smoothed estimates of the entire performance distribution for each force, and I tried to talk about them as a distribution. Hence discussion of floors (wacky, themed lists that did terribly fall in this category) and ceilings (there's just no way for X army to get in the top Y...). That was part of the analysis - for example, that while Daemons *can* be good, they are not inherently good.

There are also a few interesting viewpoints I think you should have included with the data you have available; top 16 vs. bottom 16, midfield performance, and analysis of the top 31 (everyone with one loss or less) to see how that contrasts with the entire field.

I have been considering some of those - butI wanted to get this out before the data got even older, and given I have conference travel for a few weeks...

As I get more data (as I said, hoping to do a rolling analysis of 7th), things will get somewhat more complex as I have more degrees of freedom to work with, and get to play with my favorite toolkit, which is mass simulation, rather than just statistical analysis.

My gaze now turns to 7th Edition: http://variancehammer.com/2014/08/04/announcing-the-7th-edition-rolling-meta-project/

Agreed, a very interesting reading.

Of course apart from the limitation of the small sample size, one aspect that is probably impossible to analyse is how many good players pick a strong army because they are good at picking strong armies, versus players who go just to have a good time with whatever army they happen to own.

 Kilkrazy wrote:
Agreed, a very interesting reading.

Of course apart from the limitation of the small sample size, one aspect that is probably impossible to analyse is how many good players pick a strong army because they are good at picking strong armies, versus players who go just to have a good time with whatever army they happen to own.

Yeah, there's definitely some things like "Really great tournament players are willing to drop the cash on a new hot army, and can pick accordingly" and things like that. I once fantasied about having a "Self Reported Skill" on everyone's registration sheet for a major tournament, but that would involve a ton of cooperation from TOs. Or similarly, whether the Eldar/Dark Eldar player is running some sort of horrific deathstar, or just wants to load his Banshees in a raider.