Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

I see quite a few opinions about GW not play testing enough before releasing a product so that led me to thinking. Is it actually possible to play test this game? So perhaps a little math hammering can help with this conundrum. And so I proceeded to try to do Maths again.

We have now 17 armies being sold on the Gw site. If we include data slates, chapters, FW lists (dark harvest etc) that's probably 25? It's just an approximate number.

So to test a new edition, we need every army to play against every army. That'll be a total of 300. The formula being games = (25 x (24+11))/2

So each army playing another army once is not enough for an average. Perhaps 5 games? And perhaps 5 that army 1 starts first and 5 that army 2 starts first? That'll be 10.

So right now, 3000 games would need to be played to play test a new edition. If each games takes 3 hours (we don't want to rush the play testers who are looking for mistakes), that'll be 9000 hours of play.

If Gw hires ten playtesters, who play for 9 hours a day due to labour laws, that's 45 hours a week (flouting it a bit), or 225 hours for them. So in 40 weeks, play testing completes.

This is without adding in different permutations of psychic disciplines. So perhaps maybe 120 weeks in all? And different types of list? DA has regular, Deathwing and Ravenwing. Eldar has Biel Tan, Sain Han and I can't remember the rest.

So perhaps in a game this broad, play testing under controlled situations is next to impossible. It's a pity that is so, if not everyone would be happy with a well playtesters product. And it is also a pity that perhaps this is requested by many but the thought of this task being daunting never crossed anyone's mind.

But this ones I really pity are the 10 guys who spent 120 weeks playing warhammer 40k everyday. I hope they found a good place to recuperate for the rest of their lives.

...or, you could just ask the people who know the game and how to play it (read: competitive players) and hire some of them for their advice.

But alas, GW has repeatedly stated / shown that they are not interested in putting effort in releasing a balanced and well-rounded ruleset.

Is it possible? Yes. But GW doesn't care, either because they're really that stupid or more likely because they aren't taking into account that you can do something, because they'd never think of it.

It reminds me very much of 3.5 D&D, which while it was (supposedly...) playtested, it was common knowledge and verified by third-party playtesters that WotC had specific ideas of how the game should be played, so willfully ignored things that were broken because they'd never in a million years consider doing it and therefore incapable of realizing that it was broken and needed to be fixed. For example, in base D&D it was possible for a high-level Druid to completely break the game by using Wildshape to turn into something ridiculous like a Dire Bear or a T-Rex or whatnot (I forget the specifics). But no designer would ever consider that, so it was never addressed (of course D&D being D&D, there was at least the DM who could reel in abuse). Or, Wizards/Sorcerers commonly used utility spells to render creatures out of a fight (what was commonly known as "save or suck" spells), but WotC played Sorcerers as blasters, taking offensive spells that generally were lackluster compared to utility. Then you had the Character Optimization group (aka the powergamers) who would purposely find those broken combos, sometimes just for the lulz but often to point out "This is broken and needs to be fixed, here's why" and to showcase the lack of general thought given by the designers. All in all though that's not a huge deal in a tabletop RPG because you have a GM to arbitrate things and nip abusive combos in the bud. Not so in a wargame in the vast majority of circumstances.

I think GW is in the same boat - you'll never see half the abusive combos come up in their games because that's not how they play, so they can't fathom that anyone would play like that at all. The fact that 40k (as of 6th edition anyways) is clearly meant to be played casually with a gaming club or known group lends itself more to that fact, as what kind of spanker would play something deliberately broken against their mates, knowing it would ruin their fun, just to win?

 Sigvatr wrote:
...or, you could just ask the people who know the game and how to play it (read: competitive players) and hire some of them for their advice.

Umm

Competitive players clearly refuse to even acknowledge how the game is to be played, much less know how to do it.

With the current way GW develops the game, no, real playtesting is impossible. Or, perhaps, a better term would be unfeasible. New editions of core rules, with older editions of codices that have been released over a long spread of years, played at varying points levels, makes it incredibly difficult for truly effective playtesting.

GW really needs to reboot the whole game a la 3rd Edition with a complete rewrite and new editions of all the codices (preferably all at the same time to prevent balance issues between older and newer codices). I am, in fact, quite surprised GW has not done this, as it means they get to sell lots of rulebooks and codices all at once to make their yearly financials look good.

 Zweischneid wrote:

 Sigvatr wrote:
...or, you could just ask the people who know the game and how to play it (read: competitive players) and hire some of them for their advice.

Umm

Competitive players clearly refuse to even acknowledge how the game is to be played, much less know how to do it.

I completely disagree. Competitive players, particularly those who frequent tournaments, generally have a much greater understanding of the rules and various rule interactions and overpowered combos than casual players due to the lens through which they view the game, they would make far better playtesters as they have trained themselves to look at how they can "game the system" (for want of a better term) in order to win.

 Sigvatr wrote:
...or, you could just ask the people who know the game and how to play it (read: competitive players) and hire some of them for their advice.

But alas, GW has repeatedly stated / shown that they are not interested in putting effort in releasing a balanced and well-rounded ruleset.

They have. The rules team from what have been able to gather is made up of actual players. They key word being players, not player, and you can see the contradiction in some of the wording used in the BRB

I think a lot of the problems just come from not using a wide enough group of people for play testing and not accepting feedback from the community that finds flaws in order to fix those flaws.

A small group of people may play for a very long time and never stumble across some of the broken combos across various books and may never notice some of the incorrectly or vaguely written rules.

Widen that group up a bit and one person will notice one thing and share it then another will notice another thing and share it and before you know it, your sloppy pathetic excuse for a set of rules actually starts to come together in a somewhat balanced and cohesive way.

Of course you still need a small group at the head to actually implement things, too many cooks spoil the broth and all that jazz, the actual writing team should be small, but the playtesting group should be as large as is practical.

 Zweischneid wrote:

 Sigvatr wrote:
...or, you could just ask the people who know the game and how to play it (read: competitive players) and hire some of them for their advice.

Umm

Competitive players clearly refuse to even acknowledge how the game is to be played, much less know how to do it.

You have your threads, not interesting in further baits - keep them to yourself. Thanks.


Automatically Appended Next Post:

Zodiark wrote:

 Sigvatr wrote:
...or, you could just ask the people who know the game and how to play it (read: competitive players) and hire some of them for their advice.

But alas, GW has repeatedly stated / shown that they are not interested in putting effort in releasing a balanced and well-rounded ruleset.

They have. The rules team from what have been able to gather is made up of actual players. They key word being players, not player, and you can see the contradiction in some of the wording used in the BRB

There's a glaring difference between "player" and "competitive player", though, with the latter a much deeper and more analytic understanding of the rules.

AllSeeingSkink wrote:
I think a lot of the problems just come from not using a wide enough group of people for play testing and not accepting feedback from the community that finds flaws in order to fix those flaws.

A small group of people may play for a very long time and never stumble across some of the broken combos across various books and may never notice some of the incorrectly or vaguely written rules.

Widen that group up a bit and one person will notice one thing and share it then another will notice another thing and share it and before you know it, your sloppy pathetic excuse for a set of rules actually starts to come together in a somewhat balanced and cohesive way.

Of course you still need a small group at the head to actually implement things, too many cooks spoil the broth and all that jazz, the actual writing team should be small, but the playtesting group should be as large as is practical.

This. Plus hiring all those play testers would cost a lot of money, (which as a share holder I'm suppose to hate).

In a perfect world GW should take a page from Privateer Press, and hold a world wide open beta.... You would get 9000 hours worth of FREE play testing in 1 week. It's why the warmachine rule set is really tight & balanced, only using in-house play testing for new releases.

AllSeeingSkink wrote:
I think a lot of the problems just come from not using a wide enough group of people for play testing and not accepting feedback from the community that finds flaws in order to fix those flaws.

A small group of people may play for a very long time and never stumble across some of the broken combos across various books and may never notice some of the incorrectly or vaguely written rules.

Widen that group up a bit and one person will notice one thing and share it then another will notice another thing and share it and before you know it, your sloppy pathetic excuse for a set of rules actually starts to come together in a somewhat balanced and cohesive way.

Of course you still need a small group at the head to actually implement things, too many cooks spoil the broth and all that jazz, the actual writing team should be small, but the playtesting group should be as large as is practical.

Word, this is a common flaw in all sorts of "test groups" if they are self selecting (volunters) or hired, you run the risk of "group think".

 Zweischneid wrote:

 Sigvatr wrote:
...or, you could just ask the people who know the game and how to play it (read: competitive players) and hire some of them for their advice.

Umm

Competitive players clearly refuse to even acknowledge how the game is to be played, much less know how to do it.

Given that until 6th edition GW acknowledged competitive gaming, this part about "how the game is to be played" is a bunch of bollocks. A sane company would let the competitive crowd playtest to make sure the game is balanced.

Actually there are a lot of interesting points brought up. Like the point about it being not feasible as too many codices are still out of date. And tge other point about not play testing, but hearing from the community.

But just to stir up thoughts,, I will attempt to play Devils Advocate for a while, hopefully avoiding damage in the process, for the stones thrown at me.

It is hard to define a broken combo. When the new edition first hit, intuitively a lot of us were thinking OMG what if I face an opponent with 56+1D6 psychic dice?? That must be a broken combo. Yet, we have not really tested it in many games to really confirm it. Yes, I know it is pretty obvious but so was a flat world obvious to many people in the past, before a dude, through scientific study, said it was a sphere.

To expect GW to take anyone's word for it and FAQ a broken combo, is akin to a scientific study taking another scientist's anecdotal evidence, and changing the conclusion. (Boss, this guy in xxx country did our experiment once and got a different result. We must have been wrong the last 2000 times.)

Also, how do we define a good/competitive player? One who played many games? One who won many games? One who has the most armies? One who is most vocal? There is never a good criteria that GW can possibly find that will satisfy all.

So with all that, there seems no real viable solution besides spending 120 weeks with another 10 guys, everytime a broken combo is submitted to them through gwplc.

I think the problem is that your hypothetical is taking the opposite extreme and trying to hammer in all of the possible problems to try and achieve a 100% balanced game. That is as unreasonable as 0% play testing which I feel we are closer to.

The issue at large is that a unit like the Pyrovore/Rough Riders exist in the same game as the Riptide/Heldrake. Just a preliminary glance at these units tells ANYONE that something is not quiet right.

That isn't even counting for the extremely and obviously broken list you can make when abusing psychic powers.

Arbiter_Shade wrote:
I think the problem is that your hypothetical is taking the opposite extreme and trying to hammer in all of the possible problems to try and achieve a 100% balanced game. That is as unreasonable as 0% play testing which I feel we are closer to.

The issue at large is that a unit like the Pyrovore/Rough Riders exist in the same game as the Riptide/Heldrake. Just a preliminary glance at these units tells ANYONE that something is not quiet right.

That isn't even counting for the extremely and obviously broken list you can make when abusing psychic powers.

Nobody is talking about 100% balance, which is just as impossible as 0%. But competitive gamers playtesting things, or even if GW looked beyond their own little bubble, it'd be a lot better than what we have now.

 milkboy wrote:
Actually there are a lot of interesting points brought up. Like the point about it being not feasible as too many codices are still out of date. And tge other point about not play testing, but hearing from the community.

But just to stir up thoughts,, I will attempt to play Devils Advocate for a while, hopefully avoiding damage in the process, for the stones thrown at me.

It is hard to define a broken combo. When the new edition first hit, intuitively a lot of us were thinking OMG what if I face an opponent with 56+1D6 psychic dice?? That must be a broken combo. Yet, we have not really tested it in many games to really confirm it. Yes, I know it is pretty obvious but so was a flat world obvious to many people in the past, before a dude, through scientific study, said it was a sphere.

I'd just like to point out that people knew the Earth was round over 7000 years ago.

 milkboy wrote:

It is hard to define a broken combo. When the new edition first hit, intuitively a lot of us were thinking OMG what if I face an opponent with 56+1D6 psychic dice?? That must be a broken combo. Yet, we have not really tested it in many games to really confirm it. Yes, I know it is pretty obvious but so was a flat world obvious to many people in the past, before a dude, through scientific study, said it was a sphere.

To expect GW to take anyone's word for it and FAQ a broken combo, is akin to a scientific study taking another scientist's anecdotal evidence, and changing the conclusion. (Boss, this guy in xxx country did our experiment once and got a different result. We must have been wrong the last 2000 times.)

If a scientist spots an error in a theory, and proves that error through rigorous and peer-reviewed research, then it would indeed overturn the existing theory.

 milkboy wrote:
Also, how do we define a good/competitive player? One who played many games? One who won many games? One who has the most armies? One who is most vocal? There is never a good criteria that GW can possibly find that will satisfy all.

So with all that, there seems no real viable solution besides spending 120 weeks with another 10 guys, everytime a broken combo is submitted to them through gwplc.

Or, as has been mentioned, an open beta. As well as Warmachine, Malifaux 2E did the same. Rules PDF's would be issued, and errors or abusive combos would be reported on the forums, and adjusted in time for the next release. The end result was a tight and well-balanced game. Of course, GW would probably never do that, because it might lessen the amount they can charge for rulebooks.

At the very least, if 40K players are expected to shell out the amount they do for rules, then yes, there should be a rigorous and scientific approach to making sure those rules work, there are no game-breakers sneaking through and that everything in the codexes are worth their points.

WayneTheGame wrote:


Nobody is talking about 100% balance, which is just as impossible as 0%.

No, but you are talking about balance as if it were a universally good thing, and more balance better than less balance. That is not the case.

 Zweischneid wrote:

WayneTheGame wrote:


Nobody is talking about 100% balance, which is just as impossible as 0%.

No, but you are talking about balance as if it were a universally good thing, and more balance better than less balance. That is not the case.

Oh are we back to this again? Balance *is* a universally good thing, infinitely better than lack of balance.

If you really want to have this discussion again, let's take it elsewhere.

I can't even imagine 40k ever getting to the point where becoming more balanced would detract from the game, it took Starcraft ages to reach that point and they had weekly patches instead of having to wait for a new edition and a once-in-a-blue-moon FAQ.

Oh Zwei, you're so funny.

On topic, a beta test team would find much of the glaring issues. There'd still be powerful builds and weak units, but given a decent sized team of people playing from a variety of backgrounds would find the worst of the issues in quick order. Its not hard to bring 40k up to a sensible level of internal balance, and the external balance between codices/top builds would be a little harder but still reasonable.

Its like anything in life really; effort in equals results out. Put more effort in, you'll get a better product. Unfortunately, 40k is the opposite of that.

So yes, it is absolutely a reasonable thing to expect and be able to accomplish. It wouldn't be perfect, but no one wants or even expects that. Just better.

Like making Rough Riders halfway decent.


Automatically Appended Next Post:

 PrinceRaven wrote:
I can't even imagine 40k ever getting to the point where becoming more balanced would detract from the game, it took Starcraft ages to reach that point and they had weekly patches instead of having to wait for a new edition and a once-in-a-blue-moon FAQ.

You dont' want to go down that rabbit hole with Zwei.

 PrinceRaven wrote:
I can't even imagine 40k ever getting to the point where becoming more balanced would detract from the game, it took Starcraft ages to reach that point and they had weekly patches instead of having to wait for a new edition and a once-in-a-blue-moon FAQ.

And the game is still by no means balanced

Which game?

 PrinceRaven wrote:
Which game?

I think Starcraft, based on your reference that was quoted.

 Blacksails wrote:

 PrinceRaven wrote:
Which game?

I think Starcraft, based on your reference that was quoted.

Correct. Starcraft is by far nowhere near a balanced game.

Well of course Starcraft isn't really balanced any more, not only have they released a new game but after it got to the point where the game was so balanced the matches were decided more on micro and button clicking than tactics and strategy they shook it up and introduced a bit more imbalance into the game.

 PrinceRaven wrote:
Well of course Starcraft isn't really balanced any more, not only have they released a new game but after it got to the point where the game was so balanced the matches were decided more on micro and button clicking than tactics and strategy they shook it up and introduced a bit more imbalance into the game.

It was never balanced from its original inception tbh. But adding new things definitely did not help.

Sweet. Looks like we have another Zwei thread in the making. I'll go ahead and get things started in an on-topic way and see if we can skip a couple of preliminary pages:

Some people think the game is better because it's ludicrously unbalanced, so why should GW fix it? For example, the Penitent Engine is virtually unusable for its points cost. Playtesting it (or maybe even critically thinking about it for 30 seconds) would point out that its point cost is insanely high for what it does.

I contend that it would be a good thing if they lowered the points cost. However, other posters have been known to contend that lowering the cost, effectively making it more balanced, would somehow be a bad thing for the game.

Therefore, GW should not play test.

 slowthar wrote:
Sweet. Looks like we have another Zwei thread in the making. I'll go ahead and get things started in an on-topic way and see if we can skip a couple of preliminary pages:

Some people think the game is better because it's ludicrously unbalanced, so why should GW fix it? For example, the Penitent Engine is virtually unusable for its points cost. Playtesting it (or maybe even critically thinking about it for 30 seconds) would point out that its point cost is insanely high for what it does.

I contend that it would be a good thing if they lowered the points cost. However, other posters have been known to contend that lowering the cost, effectively making it more balanced, would somehow be a bad thing for the game.

Therefore, GW should not play test.

I see nothing wrong with lowering the cost. Then again I see any model that a player owns that never gets played as a waste of money.

WayneTheGame wrote:
Is it possible? Yes. But GW doesn't care, either because they're really that stupid or more likely because they aren't taking into account that you can do something, because they'd never think of it.

It reminds me very much of 3.5 D&D, which while it was (supposedly...) playtested, it was common knowledge and verified by third-party playtesters that WotC had specific ideas of how the game should be played, so willfully ignored things that were broken because they'd never in a million years consider doing it and therefore incapable of realizing that it was broken and needed to be fixed. For example, in base D&D it was possible for a high-level Druid to completely break the game by using Wildshape to turn into something ridiculous like a Dire Bear or a T-Rex or whatnot (I forget the specifics). But no designer would ever consider that, so it was never addressed (of course D&D being D&D, there was at least the DM who could reel in abuse). Or, Wizards/Sorcerers commonly used utility spells to render creatures out of a fight (what was commonly known as "save or suck" spells), but WotC played Sorcerers as blasters, taking offensive spells that generally were lackluster compared to utility. Then you had the Character Optimization group (aka the powergamers) who would purposely find those broken combos, sometimes just for the lulz but often to point out "This is broken and needs to be fixed, here's why" and to showcase the lack of general thought given by the designers. All in all though that's not a huge deal in a tabletop RPG because you have a GM to arbitrate things and nip abusive combos in the bud. Not so in a wargame in the vast majority of circumstances.

I think GW is in the same boat - you'll never see half the abusive combos come up in their games because that's not how they play, so they can't fathom that anyone would play like that at all. The fact that 40k (as of 6th edition anyways) is clearly meant to be played casually with a gaming club or known group lends itself more to that fact, as what kind of spanker would play something deliberately broken against their mates, knowing it would ruin their fun, just to win?

true but following D&D 3.5 WOTC relased a new edition that was balanced, and everyone agreed 4th edition sucked.