Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

Slayer-Fan123 wrote:

 Melissia wrote:
Not all data points are of equal value. While still perfectly valid ways to play 40k, games that rely on house rules aren't as useful for balance decisions as games that don't rely on house rules.

As discussed, ITC doesn't affect balance that much though.

Wrong, considering many lists are invalid in ITC where they are valid in the official rules.

I mean I'm not even saying ITC is bad. It's just another way to play 40k. But it's definitely a change in balance. That's the entire PURPOSE of ITC. Claiming ITC doesn't actually change anything means ITC is pointless and worthless and no one should follow it. It's a disingenuous argument that devalues every other argument you've made in this thread.

 Melissia wrote:

Slayer-Fan123 wrote:

 Melissia wrote:
Not all data points are of equal value. While still perfectly valid ways to play 40k, games that rely on house rules aren't as useful for balance decisions as games that don't rely on house rules.

As discussed, ITC doesn't affect balance that much though.

Wrong, considering many lists are invalid in ITC where they are valid in the official rules.

I mean I'm not even saying ITC is bad. It's just another way to play 40k. But it's definitely a change in balance. That's the entire PURPOSE of ITC. Claiming ITC doesn't actually change anything means ITC is pointless and worthless and no one should follow it. It's a disingenuous argument that devalues every other argument you've made in this thread.

In terms of unit strength it doesn't change much. Maybe per codex, but stats showed codices were closer in win rate using ITC in the previous ITC thread. That doesn't stop the top from being the top though, ergo nerfing based on that data is perfectly reasonable.

Then as I said GW doesn't know how to do that. They'll look at a codex and throw darts at it.

Slayer-Fan123 wrote:

 Melissia wrote:

Slayer-Fan123 wrote:

 Melissia wrote:
Not all data points are of equal value. While still perfectly valid ways to play 40k, games that rely on house rules aren't as useful for balance decisions as games that don't rely on house rules.

As discussed, ITC doesn't affect balance that much though.

Wrong, considering many lists are invalid in ITC where they are valid in the official rules.

I mean I'm not even saying ITC is bad. It's just another way to play 40k. But it's definitely a change in balance. That's the entire PURPOSE of ITC. Claiming ITC doesn't actually change anything means ITC is pointless and worthless and no one should follow it. It's a disingenuous argument that devalues every other argument you've made in this thread.

In terms of unit strength it doesn't change much. Maybe per codex, but stats showed codices were closer in win rate using ITC in the previous ITC thread. That doesn't stop the top from being the top though, ergo nerfing based on that data is perfectly reasonable.

Then as I said GW doesn't know how to do that. They'll look at a codex and throw darts at it.

Nerfing isn't even the correct answer most of the time. If you use ITC data or even tourny data in general, then Nids would just get price hikes or other nerfs to the hive tyrant all the time. Again, what does the tourny scene tell anyone about Pyrovores and Tyrannocytes? It's bad data that just messes things up worse because the test it self is bad and only includes a small subsection of units under crap conditions.

 Lance845 wrote:

Slayer-Fan123 wrote:

 Melissia wrote:

Slayer-Fan123 wrote:

 Melissia wrote:
Not all data points are of equal value. While still perfectly valid ways to play 40k, games that rely on house rules aren't as useful for balance decisions as games that don't rely on house rules.

As discussed, ITC doesn't affect balance that much though.

Wrong, considering many lists are invalid in ITC where they are valid in the official rules.

I mean I'm not even saying ITC is bad. It's just another way to play 40k. But it's definitely a change in balance. That's the entire PURPOSE of ITC. Claiming ITC doesn't actually change anything means ITC is pointless and worthless and no one should follow it. It's a disingenuous argument that devalues every other argument you've made in this thread.

In terms of unit strength it doesn't change much. Maybe per codex, but stats showed codices were closer in win rate using ITC in the previous ITC thread. That doesn't stop the top from being the top though, ergo nerfing based on that data is perfectly reasonable.

Then as I said GW doesn't know how to do that. They'll look at a codex and throw darts at it.

Nerfing isn't even the correct answer most of the time. If you use ITC data or even tourny data in general, then Nids would just get price hikes or other nerfs to the hive tyrant all the time. Again, what does the tourny scene tell anyone about Pyrovores and Tyrannocytes? It's bad data that just messes things up worse because the test it self is bad and only includes a small subsection of units under crap conditions.

This isn't true at all. Clearly if people aren't taking pyrovores and tyrannocites GW should just reduce them by 10pts every year until people DO start taking them at which point they're broken and need their rules nerfed to compensate for their now lower points values.

There is no error in this system.

 H.B.M.C. wrote:

The Salt Mine wrote:
40k is not some video game...

*record scratch*

You think video game QA is quick?

Did you read the rest of the sentence? That is not what I said at all I never even implied that video game testing was a quick process. I was implying that it is a quicker process than play testing 40k by virtue of them just being easier to set up and start playing. I can start up a game of League in 5-10 minutes. Setting up a game of 40k can take 30-40 minutes with list creation, setting up a table, etc.

Also you stated how that is not how play testing works. I am curious what your idea of play testing is. It sounds like you have some experience with it so enlighten us how you would perform 40k play testing.

Define "Real 40k".


For me, it's when two or more individuals play in-universe factions on the tabletop, using models to represent their armies.


I think what most may be thinking of when they say "Real 40k" is "Official 40k", which is rules, lore, and scenarios published by Games Workshop, in various in-house sources.

ITC is merely a useful tool; a series of house rules that helps make running tournaments smoother and with less hassle. Some people like using ITC outside of tournaments. Others do not. Neither group is wrong, since it's a legitimate way to run games and campaigns, as long as players remember that ITC isn't official nor is it the end-all, be-all answer to the various issues 40k has with the official rules set/scenarios.

Any debate over whether or not ITC is "Real 40k" or not is one that muddies the waters, and nonsensical to boot, in my honest opinion

The Salt Mine wrote:

 H.B.M.C. wrote:

The Salt Mine wrote:
40k is not some video game...

*record scratch*

You think video game QA is quick?

Did you read the rest of the sentence? That is not what I said at all I never even implied that video game testing was a quick process. I was implying that it is a quicker process than play testing 40k by virtue of them just being easier to set up and start playing. I can start up a game of League in 5-10 minutes. Setting up a game of 40k can take 30-40 minutes with list creation, setting up a table, etc.

Also you stated how that is not how play testing works. I am curious what your idea of play testing is. It sounds like you have some experience with it so enlighten us how you would perform 40k play testing.

It doesnt take 30-40 minutes to set up a play test. The lists will be determined by the test case and the tables organized for use to gather data. Missions and set up will be determined in a controlled environment.

 kodos wrote:

Lammia wrote:
strongest change depending on pilot, mission and board setup...

Which is why people here saying that ITC is not the real 40k and by itself is a problem for balancing as if the factions are tested with the original missions, terrain rules and victory conditions changing any of those throws the balance out of the window.

on the other hand, if something changes from medicore to be completely over the top by just having different missions, the very core of the game already has a problem.

Laughable. Just saying. Competition requires just one thing: the rules of engagement and victory condition be known before engaging. Thats it.

Ok, let's put some data in here so that we understand effectively how much changing the missions will also change the balance of the game.

The results of the official February GT are in and we can see that:

- There are 2 undefeated lists, orks and CWE. Before anyone says that the ork just flooded the objectives with bodies, i would like to point out that he is also first in destruction points.

- We have one list with 4 wins and one draw. Necrons.

- We have 11 lists that went 4-1, and here we finally see the marines. 4 Iron hands, 1 Crimson fist, 1 Blood angel, 1 marine soup. We also have in there one astra militarum, 2 CWE and a custodes list.

A lot of iron hands and CWE sure, but we have 9 Factions in the first 14 lists. I have yet to see an ITC event achieve the same.

 Lance845 wrote:

The Salt Mine wrote:

 Lance845 wrote:
@Vaktathi Which is why I said GW needs to hire real game developers and real testers. They need people who will systematically build lists and test units under controlled conditions that will gather data on every unit and a variety of strategies. Not just which units the tournies are deciding to spam because they found an exploit

No event data is telling GW anything about pyrovores or tyrannocytes for example. So again, value the data correctly.

Event data gives you a very specific set of data, based on a small % of the units, as utilized by a small minority of players, using a non standard format and missions, with the express goal of victory, not critical testing.

How useful is that really?

The kind of play testing you are suggesting would take a literal army of people a very long time to do. 40k is not some video game that you can instantly set up a game and test stuff out over and over in a timely fashion. The average 2k point game takes 2-3 hours. Thats a decent chunck of a work day. It would take a full work day to get just 4 games in and thats 2 peoples time. Thats 20 games a week if they do nothing but play games. They still have make reports and comunicate with designers. There are literally billions if not trillions of possible unit interactions in this game. If you wanted 100 play testers even being payed minimum wage thats 2 million EUR a year in wages. I don't even think 100 people would be nearly enough to get all this testing done in a timely fashion you are likely looking at thousands of people required.

There is a reason GW uses outside data to make balancing changes. The kind of resources it takes to properly play test something like 40k is pretty stagering. Most online games continually take data from players and make balance changes as well.

All this isn't to say GW couldn't do a better job. I think there is always room for improvement.

GW doesn't need a literal army. It needs 1-2 testers per army working 8 hour days 40 hours a week. with 1 lead tester/supervisor. 8th was up for pre order june 3rd 2017. Which means it went to the printers about now that year. Roughly that staff would have had 3 years or 6240 hours PER EMPLOYEE to perform testing on the game since 8th came out. When they were going to make a change like the new SM codex every tester would be directed to play as or against SM to see how those rules function and the sheer power creep would have been obvious after the first week. They don't need to play full 2k games for every test case. And they should probably be spending 6-7 hours a day running that days assigned test cases before spending the last hour filling out their paperwork (which should be a digital log) and submitting it to the supervisor. Supervisor is the one in the meetings with the developers (most likely weekly) and compiling the data gathered, and designing and assigning the test cases.

Hiring real testers doesn't take what it does for video games. And the biggest issues become obvious and get addressed first.

I think you are going a little overboard with what is needed here. They literally only need like 6 play testers. The group should be the ones writing the rules for the game and they play 2-3 games a day in different kinds of mission types with a specific mandate to try to deliberately find the combos and units that are too good and not good enough.

Slayer-Fan123 wrote:


As discussed, ITC doesn't affect balance that much though. The top codices will still continue to be at the top. Therefore the data collected (army traits, maxed out units, strats) can be used for balancing decisions.

However the issue is that GW just kinda throws darts.

So how come marines keep winning (and dominating) ITC events and keep not winning the events that GW runs with the Chapter Approved missions.

This weekend the 40K GT had over 1/3 of the entries codex marines - yet no marines made the top 4. When was the last ITC mission major event that had zero codex marines in the top 4? It's like the facts consistently don't fit this claim that it's always the same top factions regardless of tournament format - so people just keep repeating the same claim but louder. It's so Dakka.

If you really love ITC then good for you. Go and enjoy it and we expect to see no complaints about the format or its balance issues because you are enjoying it so much.

happy_inquisitor wrote:

Slayer-Fan123 wrote:


As discussed, ITC doesn't affect balance that much though. The top codices will still continue to be at the top. Therefore the data collected (army traits, maxed out units, strats) can be used for balancing decisions.

However the issue is that GW just kinda throws darts.

So how come marines keep winning (and dominating) ITC events and keep not winning the events that GW runs with the Chapter Approved missions.

This weekend the 40K GT had over 1/3 of the entries codex marines - yet no marines made the top 4. When was the last ITC mission major event that had zero codex marines in the top 4? It's like the facts consistently don't fit this claim that it's always the same top factions regardless of tournament format - so people just keep repeating the same claim but louder. It's so Dakka.

If you really love ITC then good for you. Go and enjoy it and we expect to see no complaints about the format or its balance issues because you are enjoying it so much.

ITC is about hiding from your opponent most of the game with most of your units. What is a GW GT like?

Spoletta wrote:
Ok, let's put some data in here so that we understand effectively how much changing the missions will also change the balance of the game.

The results of the official February GT are in and we can see that:

- There are 2 undefeated lists, orks and CWE. Before anyone says that the ork just flooded the objectives with bodies, i would like to point out that he is also first in destruction points.

- We have one list with 4 wins and one draw. Necrons.

- We have 11 lists that went 4-1, and here we finally see the marines. 4 Iron hands, 1 Crimson fist, 1 Blood angel, 1 marine soup. We also have in there one astra militarum, 2 CWE and a custodes list.

A lot of iron hands and CWE sure, but we have 9 Factions in the first 14 lists. I have yet to see an ITC event achieve the same.

You stated a topping that's no different than ITC outside of Necrons. Marines, Marines, Marines, Eldar, Imperial Guard (probably with a Knight at that), etc. If you're trying to kid yourself by saying one tournament proves your point, that's hilarious. Consistency is key, so if you're going to throw numbers around, do it enmasse. Otherwise that's the same as saying 6th Edition Rubric Marines were good because someone topped a tournament with them and Ahriman once. It happened too, yet people forget.


Automatically Appended Next Post:

happy_inquisitor wrote:

Slayer-Fan123 wrote:


As discussed, ITC doesn't affect balance that much though. The top codices will still continue to be at the top. Therefore the data collected (army traits, maxed out units, strats) can be used for balancing decisions.

However the issue is that GW just kinda throws darts.

So how come marines keep winning (and dominating) ITC events and keep not winning the events that GW runs with the Chapter Approved missions.

This weekend the 40K GT had over 1/3 of the entries codex marines - yet no marines made the top 4. When was the last ITC mission major event that had zero codex marines in the top 4? It's like the facts consistently don't fit this claim that it's always the same top factions regardless of tournament format - so people just keep repeating the same claim but louder. It's so Dakka.

If you really love ITC then good for you. Go and enjoy it and we expect to see no complaints about the format or its balance issues because you are enjoying it so much.

Once again, you're not giving numbers or anything nor are you giving lists (because any narrative/casual event is not to be taken seriously for discussing balance issues that do clearly exist in the game).

Slayer-Fan123 wrote:

Once again, you're not giving numbers or anything nor are you giving lists (because any narrative/casual event is not to be taken seriously for discussing balance issues that do clearly exist in the game).

It is a major GT event, not a narrative/casual event. You are just spewing trash here - you know absolutely nothing and are just full of ignorance.

Slayer-Fan123 wrote:

Spoletta wrote:
Ok, let's put some data in here so that we understand effectively how much changing the missions will also change the balance of the game.

The results of the official February GT are in and we can see that:

- There are 2 undefeated lists, orks and CWE. Before anyone says that the ork just flooded the objectives with bodies, i would like to point out that he is also first in destruction points.

- We have one list with 4 wins and one draw. Necrons.

- We have 11 lists that went 4-1, and here we finally see the marines. 4 Iron hands, 1 Crimson fist, 1 Blood angel, 1 marine soup. We also have in there one astra militarum, 2 CWE and a custodes list.

A lot of iron hands and CWE sure, but we have 9 Factions in the first 14 lists. I have yet to see an ITC event achieve the same.

You stated a topping that's no different than ITC outside of Necrons. Marines, Marines, Marines, Eldar, Imperial Guard (probably with a Knight at that), etc. If you're trying to kid yourself by saying one tournament proves your point, that's hilarious. Consistency is key, so if you're going to throw numbers around, do it enmasse. Otherwise that's the same as saying 6th Edition Rubric Marines were good because someone topped a tournament with them and Ahriman once. It happened too, yet people forget.

If I'm not mistaken, this was the first GW GT since the release of CA2019, which marked an adjustment to the format with new scenarios - how do you expect Spoletta to report on multiple events as points of data if there's only been one GW GT in that period?

Keep up, old chap...

 Dysartes wrote:

Slayer-Fan123 wrote:

Spoletta wrote:
Ok, let's put some data in here so that we understand effectively how much changing the missions will also change the balance of the game.

The results of the official February GT are in and we can see that:

- There are 2 undefeated lists, orks and CWE. Before anyone says that the ork just flooded the objectives with bodies, i would like to point out that he is also first in destruction points.

- We have one list with 4 wins and one draw. Necrons.

- We have 11 lists that went 4-1, and here we finally see the marines. 4 Iron hands, 1 Crimson fist, 1 Blood angel, 1 marine soup. We also have in there one astra militarum, 2 CWE and a custodes list.

A lot of iron hands and CWE sure, but we have 9 Factions in the first 14 lists. I have yet to see an ITC event achieve the same.

You stated a topping that's no different than ITC outside of Necrons. Marines, Marines, Marines, Eldar, Imperial Guard (probably with a Knight at that), etc. If you're trying to kid yourself by saying one tournament proves your point, that's hilarious. Consistency is key, so if you're going to throw numbers around, do it enmasse. Otherwise that's the same as saying 6th Edition Rubric Marines were good because someone topped a tournament with them and Ahriman once. It happened too, yet people forget.

If I'm not mistaken, this was the first GW GT since the release of CA2019, which marked an adjustment to the format with new scenarios - how do you expect Spoletta to report on multiple events as points of data if there's only been one GW GT in that period?

Keep up, old chap...

He's using it as THE point of data is the point I'm making.

 Xenomancers wrote:

 Lance845 wrote:

The Salt Mine wrote:

 Lance845 wrote:
@Vaktathi Which is why I said GW needs to hire real game developers and real testers. They need people who will systematically build lists and test units under controlled conditions that will gather data on every unit and a variety of strategies. Not just which units the tournies are deciding to spam because they found an exploit

No event data is telling GW anything about pyrovores or tyrannocytes for example. So again, value the data correctly.

Event data gives you a very specific set of data, based on a small % of the units, as utilized by a small minority of players, using a non standard format and missions, with the express goal of victory, not critical testing.

How useful is that really?

The kind of play testing you are suggesting would take a literal army of people a very long time to do. 40k is not some video game that you can instantly set up a game and test stuff out over and over in a timely fashion. The average 2k point game takes 2-3 hours. Thats a decent chunck of a work day. It would take a full work day to get just 4 games in and thats 2 peoples time. Thats 20 games a week if they do nothing but play games. They still have make reports and comunicate with designers. There are literally billions if not trillions of possible unit interactions in this game. If you wanted 100 play testers even being payed minimum wage thats 2 million EUR a year in wages. I don't even think 100 people would be nearly enough to get all this testing done in a timely fashion you are likely looking at thousands of people required.

There is a reason GW uses outside data to make balancing changes. The kind of resources it takes to properly play test something like 40k is pretty stagering. Most online games continually take data from players and make balance changes as well.

All this isn't to say GW couldn't do a better job. I think there is always room for improvement.

GW doesn't need a literal army. It needs 1-2 testers per army working 8 hour days 40 hours a week. with 1 lead tester/supervisor. 8th was up for pre order june 3rd 2017. Which means it went to the printers about now that year. Roughly that staff would have had 3 years or 6240 hours PER EMPLOYEE to perform testing on the game since 8th came out. When they were going to make a change like the new SM codex every tester would be directed to play as or against SM to see how those rules function and the sheer power creep would have been obvious after the first week. They don't need to play full 2k games for every test case. And they should probably be spending 6-7 hours a day running that days assigned test cases before spending the last hour filling out their paperwork (which should be a digital log) and submitting it to the supervisor. Supervisor is the one in the meetings with the developers (most likely weekly) and compiling the data gathered, and designing and assigning the test cases.

Hiring real testers doesn't take what it does for video games. And the biggest issues become obvious and get addressed first.

I think you are going a little overboard with what is needed here. They literally only need like 6 play testers. The group should be the ones writing the rules for the game and they play 2-3 games a day in different kinds of mission types with a specific mandate to try to deliberately find the combos and units that are too good and not good enough.

No. When game developers are their own testers they are too close to the project to effectively test it. They know how things are "supposed" to work so they inherently run things that way even if that not how it actually works. The devs can run some tests, but they need dedicated testers to run it through the meat grinder.

 Dysartes wrote:

If I'm not mistaken, this was the first GW GT since the release of CA2019, which marked an adjustment to the format with new scenarios - how do you expect Spoletta to report on multiple events as points of data if there's only been one GW GT in that period?

Keep up, old chap...

You are mistaken. The GW GT Final was in January and has previously been discussed on a couple of these threads on tournament format & balance.

Caledonian Uprising also used the CA19 missions in January - it used a combo of Eternal War and the new Maelstrom. Also recently discussed here on Dakka.

Slayer-Fan123 wrote:Once again, you're not giving numbers or anything nor are you giving lists (because any narrative/casual event is not to be taken seriously for discussing balance issues that do clearly exist in the game).

Um... the GT is no more casual or narrative based than the LVO/ITC.

Lovely case of ITCentrism though!

I think you are going a little overboard with what is needed here. They literally only need like 6 play testers. The group should be the ones writing the rules for the game and they play 2-3 games a day in different kinds of mission types with a specific mandate to try to deliberately find the combos and units that are too good and not good enough.

1ksons have 5 unique HQ choices in their codex. Now lets take a supreme command detachment which can take up to 5 HQ units. There are 3,125 possible combinations to construct that detachment not counting wargear options, psychic powers, warlord traits, relics, and now cults. That is one detachment from one army using just HQ units.

Now lets look at how many ways units can be over the top. Are they over the top do to stratagem interactions such as IH levi dreads? Are they over the top when you can spam them such as Plague Burst Crawlers where before the rule of 3? Are they just they too good just because of their point costs? Are they over powered in specific mission situations? Are they over powered because they benefit from certain auras too much? Are they over powered because of abilities in a different codex that they can benefit from? These are just a few I could come up with off the top of my head I am sure there are more reasons for units to be over powered than this.

Now lets look at how many ways a unit can be under performing. Do they just cost to much? Do they require an inordinate amount of resources to make them remotely viable like the CSM daemon engines right now? Are they a good unit but the current meta just doesn't have a place for them? Do they just not have a place in certain mission sets but do in others? Again just some reasons I came up with off of the top of my head there are definitely much more than these listed here.

Now how many games would you want to play to test a particular small change to a model/interaction to get a solid grasp of how good or bad it is 1, 5, 10, or 100? Lets just assume you want to get 1 game in against every faction in the game there are 36 unique faction entries on the GW website (Granted a lot of those are space marines but with the supplements out now I think they play differently enough to count). Assuming a 2 hour game which is pretty generous if you ask me that 72 hours of game play time to just get 1 game against all armies with one list. That one game isn't even remotely going to cover all the possible interactions that can happen both internally from different units in the codex and externally from other codices.

Now lets take a look at what you want the over all design philosophy. Do they want to play test and design your game for a more beer and pretzels casual night? Are they trying to develop the game into a more competitive setting? Do they want the game to have numerous different rules sets to appeal to the largest possible amount of consumers? Do they want every army to be self sufficient? Do they want some armies to rely on allies to make up for inherent weaknesses of their codex? Do they design codices to intentionally have weaknesses?

Now who do you want even doing the play testing? Average John Doe who has a basic understanding of the rules or the top 40k players around that consistently win tournaments? That is going to have a huge impact on your findings as well.

Not that this conversation even matters because GW does do internal and external play testing. Stuff still slips through the cracks either because of the massive amount of rules interactions that are possible and the testers just don't catch them. Or through deliberately just ignoring the play testers findings like in the case of Iron Hands.

Slayer-Fan123 wrote:
Once again, you're not giving numbers or anything nor are you giving lists (because any narrative/casual event is not to be taken seriously for discussing balance issues that do clearly exist in the game).

I am not surprised to see that your ignorance matches your arrogance.

 Lance845 wrote:
No. When game developers are their own testers they are too close to the project to effectively test it. They know how things are "supposed" to work so they inherently run things that way even if that not how it actually works. The devs can run some tests, but they need dedicated testers to run it through the meat grinder.

Very true. A good current example of this is Blizzard's latest releases, with many of World of Warcraft systems not being altered to match feedback (or being altered after feedback, but that was not properly tested), and then there is Warcraft 3: Reforged's release...

There are a lot of things that random player #463 will find that the devs would not even have thought of. Sometimes that turns in to announced features, sometimes it turns in to "exploits".

The Salt Mine wrote:
1ksons have 5 unique HQ choices in their codex. Now lets take a supreme command detachment which can take up to 5 HQ units. There are 3,125 possible combinations to construct that detachment not counting wargear options, psychic powers, warlord traits, relics, and now cults. That is one detachment from one army using just HQ units.

And you think that means they need to play 3125 games just to test those combinations?

Come on. That's not how play-testing works. They can test the 5 units across a series of games and see how they perform on average. They don't need to do hundreds (or thousands) of games.

And there's no such thing as the "Rule of 3".

 H.B.M.C. wrote:

The Salt Mine wrote:
1ksons have 5 unique HQ choices in their codex. Now lets take a supreme command detachment which can take up to 5 HQ units. There are 3,125 possible combinations to construct that detachment not counting wargear options, psychic powers, warlord traits, relics, and now cults. That is one detachment from one army using just HQ units.

And you think that means they need to play 3125 games just to test those combinations?

Come on. That's not how play-testing works. They can test the 5 units across a series of games and see how they perform on average. They don't need to do hundreds (or thousands) of games.

And there's no such thing as the "Rule of 3".

As much as most sane people apply the rule of 3 it is an optional rule for one of 3 ways to play in GWs eyes so they would still need to consider (lightly) 5x unit combos for open/narrative.

 H.B.M.C. wrote:

The Salt Mine wrote:
1ksons have 5 unique HQ choices in their codex. Now lets take a supreme command detachment which can take up to 5 HQ units. There are 3,125 possible combinations to construct that detachment not counting wargear options, psychic powers, warlord traits, relics, and now cults. That is one detachment from one army using just HQ units.

And you think that means they need to play 3125 games just to test those combinations?

Come on. That's not how play-testing works. They can test the 5 units across a series of games and see how they perform on average. They don't need to do hundreds (or thousands) of games.

And there's no such thing as the "Rule of 3".

That is not what I said I just gave an example of the large number of combinations you get in this game with just one set of units in a single detachment. And yes they do need hundreds of games to get any meaningful data out of anything. Definitely no hundreds of games for every interaction for sure there isn't enough time or manpower on the planet for that kind of undertaking. But they should be doing hundreds of test games for every major book update. However like most companies now a days its just not practical or cost effective to do anything like that when you have millions of consumers to do the beta testing for you. Slap the rules together give it a once over to make sure it at least works then let the consumers discover the bugs!

Dudeface wrote:
As much as most sane people apply the rule of 3 it is an optional rule for one of 3 ways to play in GWs eyes so they would still need to consider (lightly) 5x unit combos for open/narrative.

Doesn't change the fact that there is no such thing as the "Rule of 3". There is a guidelines for matched play games at organised events that can limit unit selections based upon the points value. It is not "3", nor is it a "rule".

The Salt Mine wrote:

Now who do you want even doing the play testing? Average John Doe who has a basic understanding of the rules or the top 40k players around that consistently win tournaments? That is going to have a huge impact on your findings as well.

Neither. Both are bad. I want people with jobs being testers to do testing.

Not that this conversation even matters because GW does do internal and external play testing. Stuff still slips through the cracks either because of the massive amount of rules interactions that are possible and the testers just don't catch them. Or through deliberately just ignoring the play testers findings like in the case of Iron Hands.

If they do they do it bad enough that it might as well be random people doing it. There is no slipping through the cracks. GW sucks at all the stuff they are doing. Their testing criteria has the wrong goals. Or they set up bad tests. Or they just have people playing games and consider it tests. Either way, they need professional testers designing and performing actual tests.

 Lance845 wrote:

The Salt Mine wrote:

Now who do you want even doing the play testing? Average John Doe who has a basic understanding of the rules or the top 40k players around that consistently win tournaments? That is going to have a huge impact on your findings as well.

Neither. Both are bad. I want people with jobs being testers to do testing.

You're going to be disappointed then. No tabletop company professional testers, it's just not cost effective in any way shape or form.

Lammia wrote:

 Lance845 wrote:

The Salt Mine wrote:

Now who do you want even doing the play testing? Average John Doe who has a basic understanding of the rules or the top 40k players around that consistently win tournaments? That is going to have a huge impact on your findings as well.

Neither. Both are bad. I want people with jobs being testers to do testing.

You're going to be disappointed then. No tabletop company professional testers, it's just not cost effective in any way shape or form.

I am not disappointed because I know GW sucks.

That being said, it is cost effective. GW has massive profit margins and a break neck release schedule. That release schedule would be more profitable in the long run if the products were all more reliable in the long run.

 Lance845 wrote:

Lammia wrote:

 Lance845 wrote:

The Salt Mine wrote:

Now who do you want even doing the play testing? Average John Doe who has a basic understanding of the rules or the top 40k players around that consistently win tournaments? That is going to have a huge impact on your findings as well.

Neither. Both are bad. I want people with jobs being testers to do testing.

You're going to be disappointed then. No tabletop company professional testers, it's just not cost effective in any way shape or form.

I am not disappointed because I know GW sucks.

That being said, it is cost effective. GW has massive profit margins and a break neck release schedule. That release schedule would be more profitable in the long run if the products were all more reliable in the long run.

WotC, FFG, several other TT companies also have the release schedules to justify professional Beta testers, none of them do though. It's not financially sensible for them to do so, whatever their current profit margin is or isn't.