Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

Orks are getting the "One of the best players in the world is bringing them to the biggest tournament in north america so they must be good" bump.

I thought time was going to be an issue but the people I talk to say the good players aren't having a time even with chess clocks. LVO will be a good test.

Asmodios wrote:
You don't do it because its something called survivorship bias in statistics and is a truly terrible way to analyze data.

Say for example you are creating a new drug to cure a cold. There are 100 trials and in 57% of the time the cold is cured in the other 43% the patient dies. Saying "well looking at the top 57% of the trials this new drug is amazing.... I mean why even pay attention to the ones that didn't make it". Purposely skewing your data to remove failures is going to obviously give a skewed result to what you are looking at.

+1 to this. Statistics isn't magic, you have to pay attention to all the numbers to get a realistic view, and it's very easy and tempting to twist statistics to say something it doesn't say.

 Melissia wrote:

Asmodios wrote:
You don't do it because its something called survivorship bias in statistics and is a truly terrible way to analyze data.

Say for example you are creating a new drug to cure a cold. There are 100 trials and in 57% of the time the cold is cured in the other 43% the patient dies. Saying "well looking at the top 57% of the trials this new drug is amazing.... I mean why even pay attention to the ones that didn't make it". Purposely skewing your data to remove failures is going to obviously give a skewed result to what you are looking at.

+1 to this. Statistics isn't magic, you have to pay attention to all the numbers to get a realistic view, and it's very easy and tempting to twist statistics to say something it doesn't say.

How do you remove data that is caused be a secondary factors.
If someone is 0-6 they can't play the game how they place isn't going to be a realistic representation of the power of any codex.
Their are players who do got to events for lolz with idiotic incoherent lists or knowingly introducing underpowered units.
Yiu need to be able to remove them from the data or even your raw data is flawed.

Ice_can wrote:

 Melissia wrote:

Asmodios wrote:
You don't do it because its something called survivorship bias in statistics and is a truly terrible way to analyze data.

Say for example you are creating a new drug to cure a cold. There are 100 trials and in 57% of the time the cold is cured in the other 43% the patient dies. Saying "well looking at the top 57% of the trials this new drug is amazing.... I mean why even pay attention to the ones that didn't make it". Purposely skewing your data to remove failures is going to obviously give a skewed result to what you are looking at.

+1 to this. Statistics isn't magic, you have to pay attention to all the numbers to get a realistic view, and it's very easy and tempting to twist statistics to say something it doesn't say.

How do you remove data that is caused be a secondary factors.
If someone is 0-6 they can't play the game how they place isn't going to be a realistic representation of the power of any codex.
Their are players who do got to events for lolz with idiotic incoherent lists or knowingly introducing underpowered units.
Yiu need to be able to remove them from the data or even your raw data is flawed.

If someone goes 0-6 though, especially in a game like this, how do you know they don't know how to play? This game is highly based on luck, in both matchups and gameplay. It's like 1/3 luck, 1/3 skill, 1/3 list building. If he got some bad matchups early on, then some bad luck later, its possible a decent player could end up losing a lot of games. You cannot assume someone with a worse record is absolutely better than someone with a winning record, which is why it's important to include every game in the win/loss rates.

Ice_can wrote:

 Melissia wrote:

Asmodios wrote:
You don't do it because its something called survivorship bias in statistics and is a truly terrible way to analyze data.

Say for example you are creating a new drug to cure a cold. There are 100 trials and in 57% of the time the cold is cured in the other 43% the patient dies. Saying "well looking at the top 57% of the trials this new drug is amazing.... I mean why even pay attention to the ones that didn't make it". Purposely skewing your data to remove failures is going to obviously give a skewed result to what you are looking at.

+1 to this. Statistics isn't magic, you have to pay attention to all the numbers to get a realistic view, and it's very easy and tempting to twist statistics to say something it doesn't say.

How do you remove data that is caused be a secondary factors.
If someone is 0-6 they can't play the game how they place isn't going to be a realistic representation of the power of any codex.
Their are players who do got to events for lolz with idiotic incoherent lists or knowingly introducing underpowered units.
Yiu need to be able to remove them from the data or even your raw data is flawed.

The simple answer is that you don't remove it.. Those 0-6/for the luz players are part of the event and data set. They are a realistic representation of the data as they are 1. playing in the same event 2. do not play any 1 specific faction 3. We are still counting wins against these players. Once again you can do something like saying "lets just look at the ITC top 10 players and what they're bringing" this isn't a complete data set like what we are referencing before but clearly some people don't want to look at the data. Now the only mono list a top player is taking is DE so saying comments like "mono IG is so laughably ahead of other mono lists" not only doesn't hold up under a large data set analysis but also even looking at just the top "winners".

The only time you remove data from a data set is when you want to hold a variable constant. So something like (players who have attended 3 or more events)essentially you need something that would apply equally to an entire data pool but you never remove single data points (oh this guy attended 3 events but did poorly we will just ignore him). You also need to take into account what you are holding constant and how this is going to skew the outcome

look at the lists at the bottom of tournament pools. Winning vs them is not 30% luck, the odds are stacked so far in the favor of a good list in the hands of a good player, that you simply have to play badly to lose to it. That's my opinion.

 Horst wrote:

Ice_can wrote:

 Melissia wrote:

Asmodios wrote:
You don't do it because its something called survivorship bias in statistics and is a truly terrible way to analyze data.

Say for example you are creating a new drug to cure a cold. There are 100 trials and in 57% of the time the cold is cured in the other 43% the patient dies. Saying "well looking at the top 57% of the trials this new drug is amazing.... I mean why even pay attention to the ones that didn't make it". Purposely skewing your data to remove failures is going to obviously give a skewed result to what you are looking at.

+1 to this. Statistics isn't magic, you have to pay attention to all the numbers to get a realistic view, and it's very easy and tempting to twist statistics to say something it doesn't say.

How do you remove data that is caused be a secondary factors.
If someone is 0-6 they can't play the game how they place isn't going to be a realistic representation of the power of any codex.
Their are players who do got to events for lolz with idiotic incoherent lists or knowingly introducing underpowered units.
Yiu need to be able to remove them from the data or even your raw data is flawed.

If someone goes 0-6 though, especially in a game like this, how do you know they don't know how to play? This game is highly based on luck, in both matchups and gameplay. It's like 1/3 luck, 1/3 skill, 1/3 list building. If he got some bad matchups early on, then some bad luck later, its possible a decent player could end up losing a lot of games. You cannot assume someone with a worse record is absolutely better than someone with a winning record, which is why it's important to include every game in the win/loss rates.

^
exactly this is why you can't just go pulling out numbers. Of course, you will get the statistical result you want pulling out the data you "feel" doesn't represent preconceived notions

If someone went into a tournament with anything except the intention to win, the data is already gamed and any conclusions you derive from it are likely flawed.

Unless you can find a way to separate the 'bad' data from the 'good' data without introducing human bias in how its selected, you're basically stuck with a data set that is useless.

You can't tell from win/loss rate whether or not someone was "trying to win."

But you can from the type of army they take, at least if you also know the tournament rules.

Karol wrote:
But you can from the type of army they take, at least if you also know the tournament rules.

No you can't. What you can do is make assumptions, but you can't actually say. And anyone can at the same time say your assumptions are wrong and baseless.

And they'd be right. You're making assumptions specifically that make your point look better, and for no other reason than that. Not assumptions you have any proof of being true. You're trying to abuse statistics to prove your point, and nothing more, and nobody is obligated to respect your asinine assumptions.

Everyone at a tournament wants to win. They build an army that they think gives them a good chance to win, within whatever pre-established "fluff" they want for their army. It may lead to some sub-optimal choices, but it's gotta be equal between all players using all races. I'm sure there are Eldar players with mono-codex lists, using less competitive craftworlds. Just like there are mono-codex Guard players, who take mechanized lists with lots of Chimeras. Not optimal, but they think it will give them a chance.

Nobody is going in not hoping to win.

 Horst wrote:
Everyone at a tournament wants to win. They build an army that they think gives them a good chance to win, within whatever pre-established "fluff" they want for their army. It may lead to some sub-optimal choices, but it's gotta be equal between all players using all races. I'm sure there are Eldar players with mono-codex lists, using less competitive craftworlds. Just like there are mono-codex Guard players, who take mechanized lists with lots of Chimeras. Not optimal, but they think it will give them a chance.

Nobody is going in not hoping to win.

You don't know that as a certainty. Everyone likes to swing around claims as fact on the internet, but that's simply not how it works.

 LexOdin9 wrote:

 Horst wrote:
Everyone at a tournament wants to win. They build an army that they think gives them a good chance to win, within whatever pre-established "fluff" they want for their army. It may lead to some sub-optimal choices, but it's gotta be equal between all players using all races. I'm sure there are Eldar players with mono-codex lists, using less competitive craftworlds. Just like there are mono-codex Guard players, who take mechanized lists with lots of Chimeras. Not optimal, but they think it will give them a chance.

Nobody is going in not hoping to win.

You don't know that as a certainty. Everyone likes to swing around claims as fact on the internet, but that's simply not how it works.

Fine. Most people try to win. Very few people will go into a tournament thinking, "I'm not even going to try!". You have to pay to get into these things after all, and they're not always cheap. I've never personally met a person who went to tournaments who didn't have a goal of winning, or at least doing well with their lore-based army.

Yeah, especially at larger tournaments you don't blow that much money and time to show up and not even -try- to win. Plenty of people go in knowing they aren't going to do well, or even win the majority of their matches. But they still try to win those matches. Yes, there is probably someone somewhere who just showed up to troll but that is so insignificant using it to discard tournament data is irrational.

bananathug wrote:
Orks are getting the "One of the best players in the world is bringing them to the biggest tournament in north america so they must be good" bump.

I thought time was going to be an issue but the people I talk to say the good players aren't having a time even with chess clocks. LVO will be a good test.

He's sponsored to bring them. The sponsor has literally given him the army to use.

I've watched these so called top players play Orks with a chess clock and they have all struggled. So much so they don't actually play the rules written as they ignore rolling certain attacks or taking certain actions they deem a time sink.

I can't wait to see the results of the LVO, you're right in that it'll be a good test.

 Horst wrote:
Everyone at a tournament wants to win. They build an army that they think gives them a good chance to win, within whatever pre-established "fluff" they want for their army. It may lead to some sub-optimal choices, but it's gotta be equal between all players using all races. I'm sure there are Eldar players with mono-codex lists, using less competitive craftworlds. Just like there are mono-codex Guard players, who take mechanized lists with lots of Chimeras. Not optimal, but they think it will give them a chance.

Nobody is going in not hoping to win.

I think the difference is that I try to win matches once the table top game begins but even in a tournament, I let fluff, my own head cannon and personal preferences in models to make my list. I personally don't like Knights or other super heavies in a normal 40k game and therefor would not take one even if I recognize that objectively it would boost my chances of winning.

The guy from Variance Hammer.

He builds the army he wants to build but he knows he's not going to win. It sounds like he goes to a lot of tournaments. He is also some kind of professional statistician. His job is to seperate signal from noise.

He does some analysis on exactly this question its not up to date, I think he jas the padt 2 years but not this year.

He only analyses players that have been to 3 events or more, and I think he has a criteria for wins lose record.

 SHUPPET wrote:
Blood Angels is one of the lowest tier armys in the game outside of soup. But because this isn't a soup game, they can contribute one of the core units to one of the strongest lists in the game. They are a top tier codex soup wise, trash tier solo.

Oddly enough, I agree with you. But you don't see a problem with this?

Ice_can wrote:
How do you remove data that is caused be a secondary factors.
If someone is 0-6 they can't play the game how they place isn't going to be a realistic representation of the power of any codex.
Their are players who do got to events for lolz with idiotic incoherent lists or knowingly introducing underpowered units.
Yiu need to be able to remove them from the data or even your raw data is flawed.

You do it by increasing your sample size. MOST tournament lists are going to be competitive to a certain degree. By increasing your sample size you make outliers like say, a guy who brings a non-competitive fluffy army to a tournament statistically less important.

By only looking at a small sample of the overall quantity, you introduce a screening criteria that is probably not relevant to the results that you want to test. Say for example, you look at only the WINNING list for the past ten tournaments. What if in one tournament, by sheer happenstance there were no DE players? DE is then erroneously underrepresented in the resultant data-set through sheer chance. And, since the sample size is so small, this effects the accuracy of the data immensely.

Almost all data sets have "noise" (garbage data). The trick is to make the sample size so large that a pattern can be discerned through the noise.

w1zard wrote:

 SHUPPET wrote:
Blood Angels is one of the lowest tier armys in the game outside of soup. But because this isn't a soup game, they can contribute one of the core units to one of the strongest lists in the game. They are a top tier codex soup wise, trash tier solo.

Oddly enough, I agree with you. But you don't see a problem with this?

OH I see a problem with it. I think the allies rules are terrible for competitive play, and REWARD you with big amounts of CP for cherrypicking the best things between dexes.

I'm not a fan of it at all.


But I'm just saying, whether I like it or not, this is how the game plays now. When we tiered Tau in 7th we tiered Riptide wing, not kroot spam. We take the strongest possible build available to a dex in competitive play and thats what tier they are, not the weakest. In competitive play, playing a keyword army as a mono-faction list is a personal restriction, and not reflective of the actual tier of that army. Otherwise Knights are garbage as they can't score or hold ground for gak and struggle to make the CP to do anything. A monofaction tier list would be interesting. But also not what the original post here says.

I know better than to get sucked into this conversation, but I'm going to do it anyway. I feel like people have done a poor job of defining the question, or at least have been defining the question in ways that fit their preferred answer. So I'm going to ask questions, then give my answers, and people can get as angry as they want with them.

1. If played by a strong, competitive player, what is the relative strength of the various factions and combinations of factions, from top to bottom?

Several lists have been given upthread, starting with Knights + IG and working down to poor Grey Knights. I by and large agree with the various lists. I think the argument about soup tiers and mono tiers is kind of missing the point. Both soup and mono lists are out there. If little Timmy at the game shop wants to know how his Iron Fists stack up, well, they lose to Eldar soup, they beat Grey Knights, and they are something like even vs Sisters of Battle.

2. When played by a casual player, does the tier list change in a meaningful way?

Basically asking if any of the factions or soups require a player to be an expert at the game to succeed with, or alternatively, asking if a tier list based on major tournament results is relevant to little Timmy going to his local shops Saturday afternoon event. Some games, particularly fighting games, have some fairly solid skill barriers, where until you learn a certain thing or how to deal with a certain strategy you will lose to the players who know it. I don't think that any faction in 40k so difficult to play that it takes being an expert at the game to get value out of them.

3. Does the event format change the tier list in a meaningful way?

I don't think the common major formats vary enough to impact the tiers, but I'm not up to date on the fine details of every major tournament. I would expect that time limits are the big question here. Orks (and to a lesser extent 'nids) might be able to take some final tables if someone can figure out how to move 300 models 6 times in one game.

3. Is the data we have strong enough to show that the tier lists are accurate?

No. In order to this right, you would shovel all of the data into a regression model in something like R or SPSS and see if there is a statistically significant difference in win rates between the factions. Given the number of confounding factors (sample size, player, event format, faction popularity, strength of schedule), your model is going to stare blankly at you then tell you to frack off. Getting a meaningful result out of the existing data set is going to be very tricky and the ability to draw conclusions limited.

Any attempt to model factions strength needs to account for faction popularity and player skill. If 50 people bring Ultramarines to a 100 person event, what does it mean that 4 of them placed in the top ten? What about if the only Sisters of Battle player got second? The extent to which player skill, faction strength, and luck impact success in the game is an open question. Any attempt to use the existing event results to rank faction strength needs to account for these complicated factors.

My opinion is that we can say certain factions are very strong, others are strong, others are mid tier, and so forth. I think that given the nature of the game and the data we have about the game, these discussions are ultimately going to come down to judgment calls based on individual experience.

4. Are any of the factions dominant to the extent that changes should be made to improve the health of the game?

I think the upper tier soups need toned down, but that overall the game is in a good place. I think that there will always be a best army, and a worst army. I think the extent to which the best builds dominate the game is less than it was in previous editions. I don't like losing a third of my army to a Knight, but I remember Leafblower, unkillable GK rhinos, and invisible dogstars.

unkillable GK Rhinos? Did I miss an era or something?

 Quickjager wrote:
unkillable GK Rhinos? Did I miss an era or something?

I think he's talking about like 5th ed GK dex man, i think he was reminiscing on things that were really strong in by-gone editions

Yea 5th was kinda the high point for durability in general but vehicles especially required either a close range melta shot or really lucky rolls with anti tank weapons. I remember my friends orks putting over half his armies fire power into a razorback and coming away with it just losing a stormbolter.

HoundsofDemos wrote:
Yea 5th was kinda the high point for durability in general but vehicles especially required either a close range melta shot or really lucky rolls with anti tank weapons. I remember my friends orks putting over half his armies fire power into a razorback and coming away with it just losing a stormbolter.

To be fair, you'd have just as much chance of blowing up every vehicle they had in one round too, that worked both ways. The only real issue was transports, any vehicle that relied on shooting more than simply staying alive was guaranteed to at the very least be doing nothing for a turn if you hurt it at all

Not if it got immobilised!

'Oh no, my basilisk can't advance!'

 An Actual Englishman wrote:
Lol why do people keep placing Orks so high when none of the stats reflect these assumptions? Orks have a 45% win rate. We have yet to win one of the big big tournaments. Our codex is the most recent and our faction does not perform well under time constraints that are becoming the norm.

E - mono DE performs well in large part thanks to AoV. GSC and their allies are about to get a slice of that hot pie. It’s ridiculous to me that only a few factions have such a ridiculous, mechanics breaking ability. I hope GW roll it out to all factions or remove it from all. Stopping a key stratagem is game winning. You can’t put a price on such a powerful ability and in my opinion it’s too much for so few factions to have access to.

i had an ork timeout at LVO. Surprising.

 An Actual Englishman wrote:
Lol why do people keep placing Orks so high when none of the stats reflect these assumptions? Orks have a 45% win rate. We have yet to win one of the big big tournaments. Our codex is the most recent and our faction does not perform well under time constraints that are becoming the norm.

E - mono DE performs well in large part thanks to AoV. GSC and their allies are about to get a slice of that hot pie. It’s ridiculous to me that only a few factions have such a ridiculous, mechanics breaking ability. I hope GW roll it out to all factions or remove it from all. Stopping a key stratagem is game winning. You can’t put a price on such a powerful ability and in my opinion it’s too much for so few factions to have access to.

My personal belief about that is because Orkz have had such a god awful codex for so long (No codex in 5th or 6th edition and arguably the worst codex in 7th) that players became used to seeing orkz as the "NPC" Race and/or the "Easy Win" race so much so that now that we have a mid tier codex it feels like we are top tier simply because opponents now have to actually try as opposed to being handed wins. Not to mention that the competitive players who stuck it out with orkz are now winning even more so simply because they were used to having to scrape out wins by just out smarting their opponents on almost every turn and now the Ork army is a bit easier to play.

SemperMortis wrote:

 An Actual Englishman wrote:
Lol why do people keep placing Orks so high when none of the stats reflect these assumptions? Orks have a 45% win rate. We have yet to win one of the big big tournaments. Our codex is the most recent and our faction does not perform well under time constraints that are becoming the norm.

E - mono DE performs well in large part thanks to AoV. GSC and their allies are about to get a slice of that hot pie. It’s ridiculous to me that only a few factions have such a ridiculous, mechanics breaking ability. I hope GW roll it out to all factions or remove it from all. Stopping a key stratagem is game winning. You can’t put a price on such a powerful ability and in my opinion it’s too much for so few factions to have access to.

My personal belief about that is because Orkz have had such a god awful codex for so long (No codex in 5th or 6th edition and arguably the worst codex in 7th) that players became used to seeing orkz as the "NPC" Race and/or the "Easy Win" race so much so that now that we have a mid tier codex it feels like we are top tier simply because opponents now have to actually try as opposed to being handed wins. Not to mention that the competitive players who stuck it out with orkz are now winning even more so simply because they were used to having to scrape out wins by just out smarting their opponents on almost every turn and now the Ork army is a bit easier to play.

I don't think so. People who are very good at the game (as measured by tournament placings) thought Orks would be near to the top. Not forum warriors - not random people who never play.

Now its fair to ask why did they seem to underperform at the LVO. To be fair, part of this might be bad matchups. Both of the theoretical stars ran into that 2nd placing Ynnari list and lost.
That list was partly designed to have more anti-horde (which is really anti-ork) firepower. - again, because people who are good at the game thought Orks would be a problem.

This suggests to me that Orks may have a meta problem. Its relatively easy to tailor a list which has a little more anti-Ork stuff, and this significantly skews your odds of winning that game, without disproportionately harming you against other lists.
Whereas there doesn't appear to be an anti traditional Imperial Soup list* which is why that list tends to win out.

*I don't think one has been identified even though it might have horrible matchups, which is usually a sign of being the meta defining list.

Tyel wrote:

SemperMortis wrote:

 An Actual Englishman wrote:
Lol why do people keep placing Orks so high when none of the stats reflect these assumptions? Orks have a 45% win rate. We have yet to win one of the big big tournaments. Our codex is the most recent and our faction does not perform well under time constraints that are becoming the norm.

E - mono DE performs well in large part thanks to AoV. GSC and their allies are about to get a slice of that hot pie. It’s ridiculous to me that only a few factions have such a ridiculous, mechanics breaking ability. I hope GW roll it out to all factions or remove it from all. Stopping a key stratagem is game winning. You can’t put a price on such a powerful ability and in my opinion it’s too much for so few factions to have access to.

My personal belief about that is because Orkz have had such a god awful codex for so long (No codex in 5th or 6th edition and arguably the worst codex in 7th) that players became used to seeing orkz as the "NPC" Race and/or the "Easy Win" race so much so that now that we have a mid tier codex it feels like we are top tier simply because opponents now have to actually try as opposed to being handed wins. Not to mention that the competitive players who stuck it out with orkz are now winning even more so simply because they were used to having to scrape out wins by just out smarting their opponents on almost every turn and now the Ork army is a bit easier to play.

I don't think so. People who are very good at the game (as measured by tournament placings) thought Orks would be near to the top. Not forum warriors - not random people who never play.

Now its fair to ask why did they seem to underperform at the LVO. To be fair, part of this might be bad matchups. Both of the theoretical stars ran into that 2nd placing Ynnari list and lost.
That list was partly designed to have more anti-horde (which is really anti-ork) firepower. - again, because people who are good at the game thought Orks would be a problem.

This suggests to me that Orks may have a meta problem. Its relatively easy to tailor a list which has a little more anti-Ork stuff, and this significantly skews your odds of winning that game, without disproportionately harming you against other lists.
Whereas there doesn't appear to be an anti traditional Imperial Soup list* which is why that list tends to win out.

*I don't think one has been identified even though it might have horrible matchups, which is usually a sign of being the meta defining list.

Yes, top players thought orkz would be good, just like Reece thought Stompa's and Killa Kanz would be top tier in 8th edition and dominate the meta. Take their "thoughts and ideas" with a grain of salt, just because they have success with other armies does not immediately make them an expert on all armies and the competitive balance of those armies. The fact remains that the guy who WON LVO last year couldn't even crack the top 8 this year because he took Orkz. You can argue that it was bad matchups, but again, that is the point of LVO, you don't know what armies you are going to face and have to prepare for everything....Also, the Loota bomb should be relatively good against a Flyer list in general because with 1 strat they don't suffer from -1 to hit (and orkz don't go beyond -1 to hit) and can shoot twice. So blasting away with 25-75 S7 -1AP 2Dmg shots a turn and than doing it twice with a strat, all hitting on 5s and 6s with each hit causing another shot as well as any rolls of 1 being rerollable, I would have thought that an Aeldari Flyer list would get decimated turn 1.

Regardless, the key take away on this for me is the fact that A lot of power gamers thought orkz would be competitive and beat out Imperial soup/aeldari soup and other top factions to win LVO, none of them got into the top 8, including players who won serious tournaments last year with other factions. (Nick was Aeldari last year)