Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

A large part of this forum seems to be obsessed by ranking stuff.
It is all fun and all. But most of us have a had at least some scientific schooling so why are we so bad at using it.
Sure we can entertain ourselves or (enrage others if you are into that) by posting: Unit X is the best ever, because I think it is.
And others can use the same to call unit X the worst thing in existence. But this doesn't get us any further now does it.

if we are all really that obsessed about talking about our favourite hobby and ranking stuff could we at least try to do a decent job and support it by actual empirical evidence ?
Not just mathammer. Mathammer is fun and all but you can't really discuss on it and it doesn't really give you the same results as actual battle field results.

So here is my claim: CSM isn't the worst army at all ignoring the top tier armies they are the best army out there.
Have fun disproving this with actual empirical evidence.

win rates in tournament games - extracted from torrent of fire during 7th edition

primary army facng, number of games played, win ratio

All opponents 617 42.5%

Space Marines 90 43.3%
Tau Empire 53 41.5%
Imperial Guard 43 52.3%
Tyranids 53 42.5%
Necrons 61 35.2%
Space Wolves 34 54.4%
Chaos Daemons 41 42.7%
Eldar 54 27.8%
Blood Angels 20 57.5%
Orks 28 35.7%

Untestable hypothesis. Science falls flat.

Specifically:
"Top tier codecies" is arbitrary. It can be redefined as needed to make the premise valid. There exists no combination of possible facts that would falsify the hypothesis.

Also,
Gravity can't be "proven". Neither trickle-down nor labor focused economic theories can't be proven. You're moving the goalpost.

Less specifically:
Ranking is difficult.

Comparing ScatterLaser Bikes to Wyches is easy. Its like comparing Einstein or Hawking to a severely mentally handicapped person on intelligence. When the gap is big, its easy.

But what about BA Tacs to IH Tacs? I have an opinion, but its certainly debatable. Similarly, when you start comparing two human intelligences within a few percentiles of each other, identifying which is actually smarter is nearly impossible to say with any real certainty.

That doesn't mean its neither fun nor worthwhile to try.

For fun, I simply enjoy the mental exercise.

For worthwhile, there are a lot of troops I know more about, and more importantly, there are a lot of viewpoints I am now familiar with.

So a scientific, provable answer is virtually impossible. It makes having the "one true answer" impossible. But that doesn't make pursuing it pointless or a waste.

 RFHolloway wrote:
win rates in tournament games - extracted from torrent of fire during 7th edition

primary army facng, number of games played, win ratio

All opponents 617 42.5%

Space Marines 90 43.3%
Tau Empire 53 41.5%
Imperial Guard 43 52.3%
Tyranids 53 42.5%
Necrons 61 35.2%
Space Wolves 34 54.4%
Chaos Daemons 41 42.7%
Eldar 54 27.8%
Blood Angels 20 57.5%
Orks 28 35.7%

Something seems really off there. Eldar only had 27.8% win rates? Necrons only 35.2? Not to mention the fact that CSM isn't even on there!

http://variancehammer.com/2016/02/19/number-crunching-the-lvo/

Done.

 ChazSexington wrote:
http://variancehammer.com/2016/02/19/number-crunching-the-lvo/

Done.

Thats more like it : D

Now do we have the same kind of things based on other large scale events or about individual units / formations ?

Buy dark vengeance, have a battle with the models included, watch Chaos lose.

Therefore games workshop will always purposely gimp Chaos marines because it is in their *supposed* best interest as a company.

 Lord Corellia wrote:

Something seems really off there. Eldar only had 27.8% win rates? Necrons only 35.2? Not to mention the fact that CSM isn't even on there!

In the statistically improbable case that you're serious here, that's the percentage of games CSM won, broken down by army.

if we are all really that obsessed about talking about our favourite hobby and ranking stuff could we at least try to do a decent job and support it by actual empirical evidence ?
Not just mathammer. Mathammer is fun and all but you can't really discuss on it and it doesn't really give you the same results as actual battle field results.

So you're asking for "science", but NOT math. Got it.

So here is my claim: CSM isn't the worst army at all ignoring the top tier armies they are the best army out there.
Have fun disproving this with actual empirical evidence.

So you're saying they're the "best of the worst"? Snark aside, I could probably get behind that. Do YOU have evidence? Additionally, to support your claim, we would also need to have empirical evidence as to whom the "best" armies are. I'm really not sure that any of this is truly possible. Tourney results and math hammer tend to support the assertion that CSM suck though.

http://variancehammer.com/2016/02/19/number-crunching-the-lvo/

Done.


Thats more like it : D

Now do we have the same kind of things based on other large scale events or about individual units / formations ?

I'm not sure how valid you can really consider that data in terms of the specific purpose of this thread. They did not take allies into account. That makes a huge difference with CSM and would probably cause a pretty significant shift. It's a well done report with some good data, but like I said, I feel for the purpose of this specific thread, the data would need to take allies into account if/when they were used. It doesn't. If you're looking for "empirical" evidence, allies is a heck of huge variable to leave out.

 oldzoggy wrote:

Not just mathammer. Mathammer is fun and all but you can't really discuss on it and it doesn't really give you the same results as actual battle field results.

So here is my claim: CSM isn't the worst army at all ignoring the top tier armies they are the best army out there.
Have fun disproving this with actual empirical evidence.

Measuring something's performance in a Vacuum (ie math-hammer) is exactly how much of science is done. Actual battlefield results are anecdotes, because if I turn my Leman Russ demolisher tanks to face my edge, then stay 30 inches away from the chaos units, Chaos Obliterators will have a field day shooting their rear armour. Once my heavy support is all dead, his whole army can concentrate fire on my remaining units and he will win with his army, but that has no bearing on how 'top tier' it is or isn't.

Comparing survivability, maneuverability and damage output with points cost is the only real way of determining how good or bad something is. The metric can be comparing like units to like (Thousand sons v Sternguard v Legion of the Damned) or prevalent opponents in the meta (how many space marines will I kill in one round of shooting)

 oldzoggy wrote:

 ChazSexington wrote:
http://variancehammer.com/2016/02/19/number-crunching-the-lvo/

Done.

Thats more like it : D

Now do we have the same kind of things based on other large scale events or about individual units / formations ?

You can ask tournament organisers for them, though they generally don't bother tracking all things.

However, you can see from the spread of how CSMs did, they could score decently, but no chance of winning a tournament. I am presuming that's the Sicarans/Heldrake/Fire Raptor etc. spams.

Sure, CSMs can beat anyone head to head, but that's very specific builds. Three Sicarans will wreak havoc with Eldar scatterbike spam or DA Darkshrouds, 3 Heldrakes vs. a Gladius Strike Force etc.

 =Angel= wrote:

 oldzoggy wrote:

Not just mathammer. Mathammer is fun and all but you can't really discuss on it and it doesn't really give you the same results as actual battle field results.

So here is my claim: CSM isn't the worst army at all ignoring the top tier armies they are the best army out there.
Have fun disproving this with actual empirical evidence.

Measuring something's performance in a Vacuum (ie math-hammer) is exactly how much of science is done. Actual battlefield results are anecdotes, because if I turn my Leman Russ demolisher tanks to face my edge, then stay 30 inches away from the chaos units, Chaos Obliterators will have a field day shooting their rear armour. Once my heavy support is all dead, his whole army can concentrate fire on my remaining units and he will win with his army, but that has no bearing on how 'top tier' it is or isn't.

Comparing survivability, maneuverability and damage output with points cost is the only real way of determining how good or bad something is. The metric can be comparing like units to like (Thousand sons v Sternguard v Legion of the Damned) or prevalent opponents in the meta (how many space marines will I kill in one round of shooting)

Lets pretend it is an organism.

I dont want in silico or in vitro experiments i am only interested in invivo experiments

You're asking for more rigorous scientific methods while completely ignoring statistics and creating an unfalsifiable hypothesis. Furthermore, the onus is on you to prove your claim that CSM is the best army, not on us to disprove it. I'm not sure how this is ever going to work.

 oldzoggy wrote:

 =Angel= wrote:

 oldzoggy wrote:

Not just mathammer. Mathammer is fun and all but you can't really discuss on it and it doesn't really give you the same results as actual battle field results.

So here is my claim: CSM isn't the worst army at all ignoring the top tier armies they are the best army out there.
Have fun disproving this with actual empirical evidence.

Measuring something's performance in a Vacuum (ie math-hammer) is exactly how much of science is done. Actual battlefield results are anecdotes, because if I turn my Leman Russ demolisher tanks to face my edge, then stay 30 inches away from the chaos units, Chaos Obliterators will have a field day shooting their rear armour. Once my heavy support is all dead, his whole army can concentrate fire on my remaining units and he will win with his army, but that has no bearing on how 'top tier' it is or isn't.

Comparing survivability, maneuverability and damage output with points cost is the only real way of determining how good or bad something is. The metric can be comparing like units to like (Thousand sons v Sternguard v Legion of the Damned) or prevalent opponents in the meta (how many space marines will I kill in one round of shooting)

Lets pretend it is an organism.

I dont want in silico or in vitroexperiments i am only interested in invivo experiments

I'm not even sure how you would make an in vitro experiment here. But what I showed was an in vivo (if you wish) statistical analysis of the Las Vegas Open, with almost 500 participants and how the separate codices ranked.

 Lord Corellia wrote:

 RFHolloway wrote:
win rates in tournament games - extracted from torrent of fire during 7th edition

primary army facng, number of games played, win ratio

All opponents 617 42.5%

Space Marines 90 43.3%
Tau Empire 53 41.5%
Imperial Guard 43 52.3%
Tyranids 53 42.5%
Necrons 61 35.2%
Space Wolves 34 54.4%
Chaos Daemons 41 42.7%
Eldar 54 27.8%
Blood Angels 20 57.5%
Orks 28 35.7%

Something seems really off there. Eldar only had 27.8% win rates? Necrons only 35.2? Not to mention the fact that CSM isn't even on there!

That is the win rate for CSM vs Other armies so when CSM faces off against elder they won 27.8% of the time.

These are the total figures for all games played with each army (win percentage, number of games played)

Chaos Daemons 54.3% 698
Chaos Space Marines 42.5% 617
Eldar 58.4% 1401
Imperial Knights 65.4% 172
Necrons 55.0% 943
Orks 43.5% 497
Space Marines 49.1% 1724
Tau Empire 54.1% 928
Tyranids 50.1% 777

I have included the number of games as it makes a difference - the sample size foe imperial knights is quite small, so there may not be that much difference between them and Eldar which should be more stable estimate given the larger number of games.

The whole premise of the OP is just plain wrong. When testing a hypothesis(Or claim as it were) the person making that hypothesis has to provide evidence to prove it. Not to dare others to disprove it. That is not how science works.


If we were using such a methodology I could make any claim I wanted to. I think Chaos Space Marines are the worst army ever. Disprove it with actual, empirical evidence.

 TheCustomLime wrote:
The whole premise of the OP is just plain wrong. When testing a hypothesis(Or claim as it were) the person making that hypothesis has to provide evidence to prove it. Not to dare others to disprove it. That is not how science works.


If we were using such a methodology I could make any claim I wanted to. I think Chaos Space Marines are the worst army ever. Disprove it with actual, empirical evidence.

True, but I think the intent was is there any evidence for the claims one way or the other. the data I have supplied shows that your hypothesis is more likely

There is one key issue which I think magnifies the effect shown in the data - better players will tend to take stronger codexes, That means that the Eldar are more likely to win their games both through the strength of the codex, and because the better players are more likely to take elder. It is quite difficult to split the effects between player impact and codex impact, but as they will typically both move in the same direction it is easier to see which codexes are stronger, but difficult to estimate by how much.

 TheCustomLime wrote:
The whole premise of the OP is just plain wrong. When testing a hypothesis(Or claim as it were) the person making that hypothesis has to provide evidence to prove it. Not to dare others to disprove it. That is not how science works.

Not exactly. Technically science does both. Once the hypothesis is tested and supporting evidence is found, then other people try to disprove it. A lot of the best work on Quantum mechanics comes from Einstein's furious attempts to disprove it. In a sense, the whole point of a hypothesis is a statement that you DO try to disprove, and inability to do so at all is taken as verification. In our specific case, the fact that CSM have a positive winrate against only 3 codexes, all of which are themselves considered terrible, falsifies the hypothesis immediately. In addition, their winrates against poor codexes are less extreme variations from the supposedly balanced 50% compared to their loss rates against even other mediocre codexes. Their best variance is +7.5 percentage points, but they have multiple matchups at about -7.5 and even ignoring Eldar/Necrons, they swing as far as -14.3 against a 'mid tier' codex, Orks. Of course, it's very difficult to extract player skill from this data, which could potentially imply that only weaker players take chaos marines. But then who knows, weaker players might love eldar and be trashing their winrates. Because we can't determine player skill as easily out of the data we have, all we can say is that multiple different codexes has been observed to make top8s, but CSM is not one of them, which implies it's not just a skill issue.

Incidentally, the OP demonstrates the classic misunderstanding of the relationship between mathhammer and observed results. All possible battlefield results are found in the set of possible results described in mathhammer, and mathhammer describes the relative frequency of those results across infinite games. It's like taking a fair die, rolling six 6s in a row, and proclaiming probability must be wrong because that 'shouldn't happen.' We've got a fundamental misunderstanding of probability there. In fact, it SHOULD happen around once per 46656 throws of six dice. Throw six dice a million times and you ought to see six 6s more than a few times. If a matchup is 95% favored for one codex, the other book should still take a win once in a while. A data set as small as, say, a player playing once a week for a year is basically the same thing. Taking years of tournament results and distilling them down gives us a better picture, but still not an exact one.

As an arbitrary example, if you threw a fair die 6 million times, you wouldn't get exactly 1 million of each result, but you'd get close to a million of each. A poor understanding of probability (or mathhammer) would then lead to the conclusion that probability must be wrong. But the actual prediction made by probability is that you will be very close to the result of 1 million per, with a very small chance to actually vary further away. This is why small sample sizes are bad. The more samples you take, the closer you will come to the 'true' values, but the math actually suggests you will rarely, if ever, get the exact 'truth' until you run infinitely many trials.

@GreyHamster

To clarify myself I meant that science doesn't use a system where the onus is on his peers to disprove his claim. Ex. "There is a white fluffy dog at the center of the galaxy. Try to disprove that!". Obviously in science a hypothesis can be disproven and a lot of science is trying to disprove it but I felt a repeat of the scientific method was besides the point.

 RFHolloway wrote:

 Lord Corellia wrote:

 RFHolloway wrote:
win rates in tournament games - extracted from torrent of fire during 7th edition

primary army facng, number of games played, win ratio

All opponents 617 42.5%

Space Marines 90 43.3%
Tau Empire 53 41.5%
Imperial Guard 43 52.3%
Tyranids 53 42.5%
Necrons 61 35.2%
Space Wolves 34 54.4%
Chaos Daemons 41 42.7%
Eldar 54 27.8%
Blood Angels 20 57.5%
Orks 28 35.7%

Something seems really off there. Eldar only had 27.8% win rates? Necrons only 35.2? Not to mention the fact that CSM isn't even on there!

That is the win rate for CSM vs Other armies so when CSM faces off against elder they won 27.8% of the time.

These are the total figures for all games played with each army (win percentage, number of games played)

Chaos Daemons 54.3% 698
Chaos Space Marines 42.5% 617
Eldar 58.4% 1401
Imperial Knights 65.4% 172
Necrons 55.0% 943
Orks 43.5% 497
Space Marines 49.1% 1724
Tau Empire 54.1% 928
Tyranids 50.1% 777

I have included the number of games as it makes a difference - the sample size foe imperial knights is quite small, so there may not be that much difference between them and Eldar which should be more stable estimate given the larger number of games.

also keep in mind with this that a lot of times a CSM primary detachment is little more than a rump detachment to Daemon allies (and often include few or even no actual Chaos Marines). The allies and detachment rules further muddle such comparisons

Welll there is such a thing but only after the original claim is proven to a certain extent.

I'd be curious how many of those chaos marine lists have an power armor guys in them as opposed to two cultist, belakor and a socceror

Here is the issue:

In science what you have here is a hypothesis.

An idea you can test.

Your idea is 'CSM isn't the worst army at all ignoring the top tier armies they are the best army out there'

So, how can we test this?

I'd say the best way to test it would be to make a CSM army and play it against 1 of each other army.

But...it would be impossible to test every variation of a CSM army against every variation of every other army.

So, I think your hypothesis would need to be more focused or defined.

Something like:

This CSM list (enter list here) is better than these lists: (enter lists here). This could then be tested.

 oldzoggy wrote:
A large part of this forum seems to be obsessed by ranking stuff.
It is all fun and all. But most of us have a had at least some scientific schooling so why are we so bad at using it.
Sure we can entertain ourselves or (enrage others if you are into that) by posting: Unit X is the best ever, because I think it is.
And others can use the same to call unit X the worst thing in existence. But this doesn't get us any further now does it.

if we are all really that obsessed about talking about our favourite hobby and ranking stuff could we at least try to do a decent job and support it by actual empirical evidence ?
Not just mathammer. Mathammer is fun and all but you can't really discuss on it and it doesn't really give you the same results as actual battle field results.

So here is my claim: CSM isn't the worst army at all ignoring the top tier armies they are the best army out there.
Have fun disproving this with actual empirical evidence.

Troll-tastic! Lol.

For what it's worth, the reality of infinite battlefields, enormous variety in your own build and that of your opponents, coupled with the variety of missions... I mean... One becomes a good general not so much by doing the math (no, I did not just suggest that you ignore math) as making sure that the number of times the math can matter against you is minimized and the times it can help you are maximized.

Good Generals know how to do this, which makes the math itself a LOT less important. Poor Generals do not, which makes it a lot MORE important to know the math. Bad generals survive on forums and on battlefields by hoping they can Statistics you to death. And if the opponent allows you the freedom to fully express this advantage then they have a good chance. Big if, but it happens.

My encouragement to anyone tackling the issue is to remember that figures don't lie..but liers can figure. A fancy way of saying that you can make numbers look good any time you want but they are more often presented to explain plausible scenarios, not probable ones.

"There are three kinds of lies: lies, damned lies, and statistics."

-Mark Twain

 oldzoggy wrote:
A large part of this forum seems to be obsessed by ranking stuff.
It is all fun and all. But most of us have a had at least some scientific schooling so why are we so bad at using it.
Sure we can entertain ourselves or (enrage others if you are into that) by posting: Unit X is the best ever, because I think it is.
And others can use the same to call unit X the worst thing in existence. But this doesn't get us any further now does it.

if we are all really that obsessed about talking about our favourite hobby and ranking stuff could we at least try to do a decent job and support it by actual empirical evidence ?
Not just mathammer. Mathammer is fun and all but you can't really discuss on it and it doesn't really give you the same results as actual battle field results.

So here is my claim: CSM isn't the worst army at all ignoring the top tier armies they are the best army out there.
Have fun disproving this with actual empirical evidence.

If you are a hard core chaos marine player then i feel bad for you as you are in denial phase and soon will be in anger you wasted si much time and money on csm when you could have been playing eldar and winning.

So here is my claim: CSM isn't the worst army at all ignoring the top tier armies they are the best army out there.
Have fun disproving this with actual empirical evidence.

You're basically just saying "If we ignore everyone who's better, then we're not the worst!" Which is true in a way I guess ... although for someone throwing around words like "empirical", and "scientific", your post is tenuous at best.

Incidentally, the OP demonstrates the classic misunderstanding of the relationship between mathhammer and observed results. All possible battlefield results are found in the set of possible results described in mathhammer, and mathhammer describes the relative frequency of those results across infinite games. It's like taking a fair die, rolling six 6s in a row, and proclaiming probability must be wrong because that 'shouldn't happen.' We've got a fundamental misunderstanding of probability there. In fact, it SHOULD happen around once per 46656 throws of six dice. Throw six dice a million times and you ought to see six 6s more than a few times. If a matchup is 95% favored for one codex, the other book should still take a win once in a while. A data set as small as, say, a player playing once a week for a year is basically the same thing. Taking years of tournament results and distilling them down gives us a better picture, but still not an exact one.

One becomes a good general not so much by doing the math (no, I did not just suggest that you ignore math) as making sure that the number of times the math can matter against you is minimized and the times it can help you are maximized.

The above two quotes are hugely important to the discussion IMO. GreyHamster's description of math hammer vs observed results is one of the best breakdowns of why math hammer matters that I've ever seen. Jancoran's quote sort of straddles the line between making the same mistake vs making a good point, but if you look at what he's trying to say, it kind of goes along with what others have said. You really can't ignore the math.

"They're the best if you ignore everything that is better" is some Orwell-level doublethink.

"They're the best if you ignore everything that is better" is some Orwell-level doublethink.

It's also not something anyone in here has said ...

You may have misread my post.

Well in that case i should ignore everyone else and everything and i would have a shot with Natahlie Dormer or Lucy Lu..., heck i can even have a shot with Lisa Ann for the matter.