Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

Isn't one of the hallmarks of modern power lists the ability to mitigate luck via special rules that remove or mitigate chance? Stuff like re-rolls, "can't roll lower than <x>", changes from dX to flat X, that sort of thing.

 waefre_1 wrote:
Isn't one of the hallmarks of modern power lists the ability to mitigate luck via special rules that remove or mitigate chance? Stuff like re-rolls, "can't roll lower than <x>", changes from dX to flat X, that sort of thing.

Yes.

But at the same time, there will always be some luck involved. The less your plan depends on rolling hot, the better you’re likely to do, but if you roll literally nothing but 1s (except on morale, where you roll 6s) and your opponent the opposite? You’ll lose.

Now, that’s obviously a hyperbolic example, but there’s no way to totally remove luck right now in 40k.

Tygre wrote:
As player skill can affect an armies performance, I would take out outliers and average the results. I would take the middle 50% of games so that the best players and the worst players don't distort the results. Or maybe a bigger chunk of the middle if there are enough results.

In game balance, you're interested in the bounds of the set rather than the middle region. So you actually want to know what the best of the players are playing, because that tells you where the upper limit of the power curve is and which armies and units are too strong.

You might also be interested in the low end, if you want to limit how badly someone can do in spite of their [lack of] skill, but in general, the middle range isn't the most relevant for tamping down on overpowed stuff or bringing up underpowered stuff.

Balance at low levels is the most important one, because it is directly tied to the health of the game and influx of new players.

Balance at top levels is mostly irrelevant for the game. At worst, it can make for bad press, but 40K isn't really that big on press in the first place.

Balance at medium level is tied to player retention. It is very important in order for the game to be fun. You are not going to keep playing or getting more factions if you can't have a good match.

Warmachine was the example of this. That game was very well balanced at top level, but anything under that was a complete disaster.
We lost many new players because they spent the first 10 games losing in the first turn.
Then many more left (including myself) because since there was no casual scene, but only cutthroath level, in the end the game stops being fun.

Low and medium level balancing hinges on the faction's internal balance, while the top level is exclusively an external balance affair.
Warmachine had good external balance, but the factions were full of auto includes and trap choices.

I would say armies have
- a floor
- a ceiling
- difficulty to access the ceiling

I would forget the floors (those are for the weak) and rather discuss the difficulty to access the ceiling


Automatically Appended Next Post:

Spoletta wrote:
Balance at low levels is the most important one, because it is directly tied to the health of the game and influx of new players.

Solution: noob friendly armies (AA, Necrons ?)

 Canadian 5th wrote:

Bharring wrote:
For "at-it's-best", use metrics related to "at-it's-best".

Top placements in top tournaments removes, or at least significantly mitigates, a number of biases.

One bias the W/L ratio has is the presence of non-optimal lists. People who show up at tournies to show off their decade-old army they've only changed to fit newer points/rules still count for W/L ratios. But they don't make the top 8 in large fields, so don't bias those numbers.

Another bias is player skill. While some players can win with anything, so can other players. And so those "some players" will spike the W/L of their chosen faction, biasing it away from what we're looking for. But, in the top 8, they'll be facing other high-skill player, driving down the bias.

Discoverability of the codex's strength is another bias. If the codex can be stupidly OP, but most players don't/haven't realized it, the W/L record will reflect something closer to what the average player has realized. But if you're looking for OP/UP codexes, you care more about the strongest ways to play the codex. If only 5 of 50 players at a tourny know how to build a good list with $faction, the top 8 metric will capture that strength much more clearly than W/L.

W/L and Top 8 are better measures for different things, really. How powerful a book can be tacks a lot closer to Top 8 representation than W/L.

Top 8s can show spikes but can also fail to show which armies are consistent performers and which just got lucky on a given weekend. Is a faction that took 3 of the top 4 positions at a single tournament and then failed to make top 16 the next four better than the faction that finished top 5 at all 5 events and has yet to claim a top 4 spot?

Only in isolation.

Instead you have to consider multiple to 8s.


However the ultimate reality is that the best armies will feature most in tope 8s and have the best W/L ratio over a large dataset. The key is the size of the dataset.

stratigo wrote:
Only in isolation.

Instead you have to consider multiple to 8s.


However the ultimate reality is that the best armies will feature most in tope 8s and have the best W/L ratio over a large dataset. The key is the size of the dataset.

Of course, that's why my first post in this thread said:

"Wins compared to losses with both weighted by the faction being played, the round in which the game occurred, and the record of the player piloting the army will always give better results than simply look at the top x places of an event. Something like Goonhammer's Glicko system will also give better results as going 4-1 and making top 8 and going 3-2 and missing out can come down to luck more than skill and a rating system will be able to better account for this than the raw data will."

Obviously, the best armies will also get the most top 8s but that doesn't do much to rank the middle-tier factions and does literally nothing to rank low-tier factions which is where any sample size of looking at just top 8s will always fail.

 JNAProductions wrote:
Yes.

But at the same time, there will always be some luck involved. The less your plan depends on rolling hot, the better you’re likely to do, but if you roll literally nothing but 1s (except on morale, where you roll 6s) and your opponent the opposite? You’ll lose.

Now, that’s obviously a hyperbolic example, but there’s no way to totally remove luck right now in 40k.

Ok but with the data we get, we can't say that a substential difference in top 8 finishs for lets say tau happned, because the tau players around the world roll a lot of 1s, while on the other hand custodes players are gods as far as rolling +3 goes.

Luck maybe a determinant in specific game cases, but not as far as results of whole factions goes. Unless the faction in question has like one dude playing it, then they maybe ove rlap

Statistics don't replace critical thinking
Whenever you condense a complex issue into a single number, that number is going to be stupid in at least some ways.

Just because an army keeps popping into the top 8, doesn't mean it's powerful or even in a good place. Single exceptional players have repeatedly placed well with armies that were objectively bad at that time. Single skew lists have done well in competitive metas despite the army being hot garbage as soon as you strayed from that list, possibly leading to the army disappearing from the top tables when the meta shifts. There also is the issue that armies which require counter-play usually place high a lot when they are new, but drop off once they are "solved".

W-L is a slightly better milestone, but is heavily influenced by how popular an army is and how loyal their players are. When many top players switch to a specific army, it's W-L ratio will increase. A loyal, but inept player will decrease the W-L ratio, especially when only a few players remain with an unpopular faction. It also hides that some armies are unable to win against certain other armies (aka hard counters). An army might have a high W-L ratio despite never placing in the top brackets because they are unable to go undefeated.

Neither really shows the power level of a faction as a whole as lists might or might not vary wildly, so you have to consider what is actually in the lists. An army with wildly varying lists and options for many different tasks is definitely more powerful than one that has 1500 points locked into the very same five datasheets or one hinging on a clutch to stay afloat.

There is a reason why goonhammer writes long articles on this topic with multiple tables, charts and numbers and doesn't just look at a single number

 Jidmah wrote:


Just because an army keeps popping into the top 8, doesn't mean it's powerful or even in a good place. Single exceptional players have repeatedly placed well with armies that were objectively bad at that time. Single skew lists have done well in competitive metas despite the army being hot garbage as soon as you strayed from that list, possibly leading to the army disappearing from the top tables when the meta shifts. There also is the issue that armies which require counter-play usually place high a lot when they are new, but drop off once they are "solved".

I think this is mostly a result of different competitivenesses.
Top bracket armies in tournaments are mostly concerned with top meta lists that a faction is capable of using. This is obviously quite flexible as metas can change on a whim.

There's also a more casual degree of balance, where a player might just be taking a selection of units without necessarily trying to go the absolute best cheese. This is more reflected in overall W/L, but not well. Factions that can bring top-meta lists will see an uptick in W/L as hardcore tournament players will be taking them to the top.
Conversely factions without top-meta lists will see a swing downwards as the only players taking them will be the casual players who don't care so much.
Popular factions will tend towards the middle, as both super casual players and hardcore players will be more likely to play them regardless of actual meta-ness.