Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

It's about Age of Sigmar, but does offer some insights.
https://www.warhammer-community.com/2017/02/09/generals-handbook-ii-behind-the-scenes/

I can't decide exactly what to read from this. Are they saying that all balancing, and playtesting is done in 2 days for the entirety of AoS? Is the idea that they did this a bunch of times with a bunch of players? Were 1 in 10 units actually used in the test games? 1 in 20? How diverse was the actual group? Was each faction represented? Were 4 of them the same faction?

Still a pile of questions, but this sort of thing is exactly the information I wanted from then when they dropped the Iron Curtain and started to re-engage.

tag8833 wrote:
...Are they saying that all balancing, and playtesting is done in 2 days for the entirety of AoS?...

Nope. They're saying they wrote it in one day and playtested it in one day. If they had two days of playtesting it might work better.

Two of the guys brought in were Dan and Wayne from the Heelanhammer podcast, who also run the South Coast GT for AoS. They have a podcast from last year ("Only the Faithful", not numbered but between 149 and 150) talking about how they were invited to a similar weekend meeting for the first Handbook.
If it's anything like that, this was a weekend basically to get outside feedback and ideas on the new rules an balancing. These hobbyists have all worked on some sort of points system themselves for tournaments and matched play when AoS was still pointless, and so GW reached out to them to basically add some variety to their discussion. It helps make sure people outside the design team can understand the rules, and highlights any oversights that might have gone unnoticed. I doubt all the balancing and discussion that happened is a set-in-stone affair, rather, it'll just be some outsourced user testing to make sure the design team is on the right track (almost like a Beta test for a video game).

 Fentlegen wrote:
Two of the guys brought in were Dan and Wayne from the Heelanhammer podcast, who also run the South Coast GT for AoS. They have a podcast from last year ("Only the Faithful", not numbered but between 149 and 150) talking about how they were invited to a similar weekend meeting for the first Handbook.
If it's anything like that, this was a weekend basically to get outside feedback and ideas on the new rules an balancing. These hobbyists have all worked on some sort of points system themselves for tournaments and matched play when AoS was still pointless, and so GW reached out to them to basically add some variety to their discussion. It helps make sure people outside the design team can understand the rules, and highlights any oversights that might have gone unnoticed. I doubt all the balancing and discussion that happened is a set-in-stone affair, rather, it'll just be some outsourced user testing to make sure the design team is on the right track (almost like a Beta test for a video game).

I am an Ideal playtester. Check out my resume:


But if you gave me 2 days, and only a handful of games to playtest and balance all of 40K, I'm going to miss stuff. Lots, and Lots of stuff.

I'm not saying I wouldn't leave it better than I found it in 2 days, I'm saying that 5 players each getting 2 games over the course of 2 days is not enough time to test and balance a game system. It's not enough outside the tank insight. It's not enough community involvement.

Video games have 100's of beta testers. They work for weeks to test, and validate a product.

5 guys for 2 days isn't enough. If that is the process they've got setup, they many reps to get to an acceptable level.

From playing GW games for a long time, I didn't think any real play testing was ever done. Which is scary and unprofessional. You'd think they would play thousands of games before releasing new editions/codices.

I always figured they'd played a few games and just sorta winged it.

That's half arsed at the best.
It should be easy for GW to get at least a hundred people who would happily donate a weekend to help playtest games like AoS and 40k.
An invitation to be involved on the Facebook page and their biggest issue would be finding a way of fitting enough tables in one place for the weekend.

 Dakka Wolf wrote:
That's half arsed at the best.
It should be easy for GW to get at least a hundred people who would happily donate a weekend to help playtest games like AoS and 40k.
An invitation to be involved on the Facebook page and their biggest issue would be finding a way of fitting enough tables in one place for the weekend.

Which is more or less what they'll be doing in SCGT tournament this year, apparently. 200 guys in a weekend of tournament seems to offer a few games.

What you guys are missing is this: GHB 2.0. isn't a designing of the game, but a tweaking and tuning of the current system. The Warlords, Blood and Glory, every book between GHBs and now, have been used to gather information on wether the balance currently set up is working or not. The five guys here are to apport the overseering/advising of the current tweaks status.

Lord Kragan wrote:
Which is more or less what they'll be doing in SCGT tournament this year, apparently. 200 guys in a weekend of tournament seems to offer a few games.

What you guys are missing is this: GHB 2.0. isn't a designing of the game, but a tweaking and tuning of the current system. The Warlords, Blood and Glory, every between GHBs and now, have been used to gather information on wether the balance currently set up is working or not. The five guys here are to apport the overseering/advising of the current tweaks status.

Exalted. And quoted for emphasis.

The article in question talks about how the team began working on the update, playing around with ideas gleaned from community feedback back in December.

 Captain Joystick wrote:

Lord Kragan wrote:
Which is more or less what they'll be doing in SCGT tournament this year, apparently. 200 guys in a weekend of tournament seems to offer a few games.

What you guys are missing is this: GHB 2.0. isn't a designing of the game, but a tweaking and tuning of the current system. The Warlords, Blood and Glory, every between GHBs and now, have been used to gather information on wether the balance currently set up is working or not. The five guys here are to apport the overseering/advising of the current tweaks status.

Exalted. And quoted for emphasis.

The article in question talks about how the team began working on the update, playing around with ideas gleaned from community feedback back in December.

Not to mention they are bringing in people who have their pulse on the game and have been theory crafting ideas for quite a while. If anything this was a meeting of the minds that was going to go over previous ideas and proposals already drawn up and ready to go, not a bunch of people spit balling ideas> I imagine the game portion was merely a mix of good times and having a go at a rule or 2 that had been hashed out but needed their invited experts to give a try.

Not to mention they are actually listening to us? Seems like a pretty good deal to me and I hope this goes straight into 40k come 8th edition

 Captain Joystick wrote:

Lord Kragan wrote:
Which is more or less what they'll be doing in SCGT tournament this year, apparently. 200 guys in a weekend of tournament seems to offer a few games.

What you guys are missing is this: GHB 2.0. isn't a designing of the game, but a tweaking and tuning of the current system. The Warlords, Blood and Glory, every between GHBs and now, have been used to gather information on wether the balance currently set up is working or not. The five guys here are to apport the overseering/advising of the current tweaks status.

Exalted. And quoted for emphasis.

The article in question talks about how the team began working on the update, playing around with ideas gleaned from community feedback back in December.

A tournament is kind of after the fact. By that time the rules have been written, read and loopholed. If FAQs regularly came out after tournaments and in response to tournaments I'd be more inclined to believe that.

 Dakka Wolf wrote:

 Captain Joystick wrote:

Lord Kragan wrote:
Which is more or less what they'll be doing in SCGT tournament this year, apparently. 200 guys in a weekend of tournament seems to offer a few games.

What you guys are missing is this: GHB 2.0. isn't a designing of the game, but a tweaking and tuning of the current system. The Warlords, Blood and Glory, every between GHBs and now, have been used to gather information on wether the balance currently set up is working or not. The five guys here are to apport the overseering/advising of the current tweaks status.

Exalted. And quoted for emphasis.

The article in question talks about how the team began working on the update, playing around with ideas gleaned from community feedback back in December.

A tournament is kind of after the fact. By that time the rules have been written, read and loopholed. If FAQs regularly came out after tournaments and in response to tournaments I'd be more inclined to believe that.

They did something similar with the previous' year SCGT. A good chunk of the june FAQ, they said, was actually questions made during that tournament.

Hopefully Chaos Dwarfs get played a bit, currently they could use some lower points in some major area's.

If JJ was involved you can be sure the result is a complete gak. He proved to be one of the worse GD ever.

Lord Kragan wrote:

 Dakka Wolf wrote:

 Captain Joystick wrote:

Lord Kragan wrote:
Which is more or less what they'll be doing in SCGT tournament this year, apparently. 200 guys in a weekend of tournament seems to offer a few games.

What you guys are missing is this: GHB 2.0. isn't a designing of the game, but a tweaking and tuning of the current system. The Warlords, Blood and Glory, every between GHBs and now, have been used to gather information on wether the balance currently set up is working or not. The five guys here are to apport the overseering/advising of the current tweaks status.

Exalted. And quoted for emphasis.

The article in question talks about how the team began working on the update, playing around with ideas gleaned from community feedback back in December.

A tournament is kind of after the fact. By that time the rules have been written, read and loopholed. If FAQs regularly came out after tournaments and in response to tournaments I'd be more inclined to believe that.

They did something similar with the previous' year SCGT. A good chunk of the june FAQ, they said, was actually questions made during that tournament.

Sounds like progress to me then, consider me satisfied.

Lord Kragan wrote:
Which is more or less what they'll be doing in SCGT tournament this year, apparently. 200 guys in a weekend of tournament seems to offer a few games.

Collecting Feedback at a tourney like that is a great idea. However, in 40K Astra Militarum, and Tyranids have been essentially purged from tourneys because their rules suck. Orks are still there, but only Ork players that play in very specific ways. Eldar are more than abundant at every tourney because they have the most powerful rules. So you primarily wouldn't fix what is broken if you take that approach.

Lord Kragan wrote:
What you guys are missing is this: GHB 2.0. isn't a designing of the game, but a tweaking and tuning of the current system. The Warlords, Blood and Glory, every book between GHBs and now, have been used to gather information on wether the balance currently set up is working or not. The five guys here are to apport the overseering/advising of the current tweaks status.

I don't play AoS. But I play lots and lots of 40K. If they brought me in and said "We can't change rules, we can't change stats, all we can do is tweak points". I don't think I could do a very good job in just a couple of days. I've heard it said that a Chimpanzee with a missing testicle could do a better job than GW's rules team when assigning points, and in many cases that might be true. Given 1 day I'm sure I could make it better, but I'm equally sure that I couldn't make it acceptably balanced. I think for instance, you'd have trouble finding a person who regularly plays 40K that would miss as badly as GW did on the Wraithknight. But where is that magic level? 100 points more? definitely. Still extremely points efficient at 395. 200 points more? Well it depends on the matchup. 150 points more? I'd just be speculating, because until I see how it affects the actual army lists I can't be sure.

tag8833 wrote:

Lord Kragan wrote:
Which is more or less what they'll be doing in SCGT tournament this year, apparently. 200 guys in a weekend of tournament seems to offer a few games.

Collecting Feedback at a tourney like that is a great idea. However, in 40K Astra Militarum, and Tyranids have been essentially purged from tourneys because their rules suck. Orks are still there, but only Ork players that play in very specific ways. Eldar are more than abundant at every tourney because they have the most powerful rules. So you primarily wouldn't fix what is broken if you take that approach.

In a lot of ways I agree with you on that but wouln't their absence raise flags that something isn't right with their rules? Especially with the excess of Eldar.
There is one thing that I worry about with relying on perception though - rather than seeing weakness being the big factor, a company observer might see it as a lack of popularity and simply discontinue the model line.

That's another thing, they've already pointed out that certain armies have a certain dominance in the comptetitive scene and others don't. Tomb Kings are more abundant than ever (ironic isn't it) in the tournaments and the fyreslayers are fairly a rarity. What' we've gotten as a peak is that the worst offenders in those armies will get points increases (it also helps that one of these 5 guys-cannot remember which- said that Tomb Kings where too undercosted and that made them tournament winning. He went on and said he'd prove that claim by winning the UK masters... guess who made a point in case and who dominated that tournament?) Meanwhile the fyreslayers, which were considered overcosted, have received reductions.

The thing with AoS is that, outside very few cases (and they are very subjective), there's nothing that actually broken on a unit per unit basis. It's undercosted stuff that causes the current imbalance.

The Deer Hunter wrote:
If JJ was involved you can be sure the result is a complete gak. He proved to be one of the worse GD ever.

What games did he make? I can't remember now. I liked reading his articles in the old White Dwarfs in the early 2000s. I always liked what he had to say. Surprised that his games would be horrible. Since I can't recall what has he made or codices he wrote?

Jervis was heavily involved in Blood Bowl and the more recent versions of Epic. Those are 2 of the better games GW has produced, especially in terms of balance.

Having said that, a lot of JJ's work for WH and 40k hasn't been so great. I don't think it's the case that he's a bad designer, I think it's possibly more the case that his style lends itself better to certain types of projects and he often struggles to get create balanced rules that fit into a greater whole. Both BB and Epic were more or less stand-alone games in many ways, designed almost entirely in one go.

The biggest thing for testing a game updates for something like 40k isn´t to play games at all but theory craft and build lists for each army.

Look at their synergy and see what works together and how that is in theory likely to effect the culture of the game.

Allies matrix for example. Find a nasty formation. How many armies can use it eithout breaking come the apocalypse. (Riptide Wing). Hmmm maybe matrix is too slack.

Slipspace wrote:
Jervis was heavily involved in Blood Bowl and the more recent versions of Epic. Those are 2 of the better games GW has produced, especially in terms of balance.

Having said that, a lot of JJ's work for WH and 40k hasn't been so great. I don't think it's the case that he's a bad designer, I think it's possibly more the case that his style lends itself better to certain types of projects and he often struggles to get create balanced rules that fit into a greater whole. Both BB and Epic were more or less stand-alone games in many ways, designed almost entirely in one go.

The thing is Jervis is from the old school so has those type of ideas. The game is less a competition between two players to see who is better and can win, and more a collaborative thought exercise with models. Which is a fine idea to take but bad when it injected into everything else

Wayniac wrote:

Slipspace wrote:
Jervis was heavily involved in Blood Bowl and the more recent versions of Epic. Those are 2 of the better games GW has produced, especially in terms of balance.

Having said that, a lot of JJ's work for WH and 40k hasn't been so great. I don't think it's the case that he's a bad designer, I think it's possibly more the case that his style lends itself better to certain types of projects and he often struggles to get create balanced rules that fit into a greater whole. Both BB and Epic were more or less stand-alone games in many ways, designed almost entirely in one go.

The thing is Jervis is from the old school so has those type of ideas. The game is less a competition between two players to see who is better and can win, and more a collaborative thought exercise with models. Which is a fine idea to take but bad when it injected into everything else

Jervis thinks matched-point pickup games are the worst way to play game. He is more sympathetic to competitive play, but feels like the best way to play is telling a story together with a pre-defined outcome, and unbound / unequal armies.

He seems to be in charge of basically all of the "New" rules that come out of GW. He oversaw 7th ed 40K. He did Deathwatch: Overkill, Space Hulk, and Stormcloud attack. Age of Sigmar pre general's handbook was his auteur vision. He seems to have overseen the General's Handbook, and seems to be overseeing General's Handbook 2. I expect he's done the other "New" rulesets that come out as well, but as far as I know he hasn't taken credit for them.

So there is a mixed track record. The original AoS was bad. Deathwatch: Overkill was pretty good, though the rules could have easily been tighter. 7th ed 40K army comp was awful. Stormcloud attack was kinda neat, but the rules could have easily been tighter.

I think that stylistically he writes very loose rules that leave alot up to interpretation. I think he isn't terrible understanding mechanics, but lacks a basic understanding of combo's, and army vs army interactions. I think he is a strong visionary whose vision frequently overwhelms the practical concerns of rules design. I think he is intensely arrogant which leads him to be less collaborative than other people tasked with writing rules of this sort. Overall, I think he is probably an excellent rules designer with obvious faults that should be working on a team with people who can offset his weaknesses, but either his force of personality, or his work style tends to drive others away. The notable rules designers that have left GW, frequently leaving with bad things to say about Jervis Johnson.

I think he probably is intensely talented, but in a way akin to George Lucas. He needs people around him who can collaborate on a project and tell him "No".

Also a day of brainstorming and calculating numbers can do far more for balance than realistic playtesting ever could.

I say realistic because the logistics of gettting a 100 players playing 2-3 games over the weekend and then report it are massive. Also compared to the calculus work tha tis needed to get those gathered statistics to make sense is more complicated than using the same kind to run theorycrafting.

You could say just provide basic feedback but that only provides knowledge of a problem not how to solve it and balnce is a tricky thing, it's very easy to overshoot your target and make a bad unit OP or vice versa.

Also balance isn't the only factor to consider. Diversity provides increased replayability but makes balance inherently more difficult. SOme of the most fun I have had in games of all stripes is playing it the way Jervis likes: suboptimal but in a style I like.

tag8833 wrote:

Wayniac wrote:

Slipspace wrote:
Jervis was heavily involved in Blood Bowl and the more recent versions of Epic. Those are 2 of the better games GW has produced, especially in terms of balance.

Having said that, a lot of JJ's work for WH and 40k hasn't been so great. I don't think it's the case that he's a bad designer, I think it's possibly more the case that his style lends itself better to certain types of projects and he often struggles to get create balanced rules that fit into a greater whole. Both BB and Epic were more or less stand-alone games in many ways, designed almost entirely in one go.

The thing is Jervis is from the old school so has those type of ideas. The game is less a competition between two players to see who is better and can win, and more a collaborative thought exercise with models. Which is a fine idea to take but bad when it injected into everything else

Jervis thinks matched-point pickup games are the worst way to play game. He is more sympathetic to competitive play, but feels like the best way to play is telling a story together with a pre-defined outcome, and unbound / unequal armies.

He seems to be in charge of basically all of the "New" rules that come out of GW. He oversaw 7th ed 40K. He did Deathwatch: Overkill, Space Hulk, and Stormcloud attack. Age of Sigmar pre general's handbook was his auteur vision. He seems to have overseen the General's Handbook, and seems to be overseeing General's Handbook 2. I expect he's done the other "New" rulesets that come out as well, but as far as I know he hasn't taken credit for them.

So there is a mixed track record. The original AoS was bad. Deathwatch: Overkill was pretty good, though the rules could have easily been tighter. 7th ed 40K army comp was awful. Stormcloud attack was kinda neat, but the rules could have easily been tighter.

I think that stylistically he writes very loose rules that leave alot up to interpretation. I think he isn't terrible understanding mechanics, but lacks a basic understanding of combo's, and army vs army interactions. I think he is a strong visionary whose vision frequently overwhelms the practical concerns of rules design. I think he is intensely arrogant which leads him to be less collaborative than other people tasked with writing rules of this sort. Overall, I think he is probably an excellent rules designer with obvious faults that should be working on a team with people who can offset his weaknesses, but either his force of personality, or his work style tends to drive others away. The notable rules designers that have left GW, frequently leaving with bad things to say about Jervis Johnson.

I think he probably is intensely talented, but in a way akin to George Lucas. He needs people around him who can collaborate on a project and tell him "No".

Probably right, especially the fact that he is good at understanding mechanics but doesn't get combos (probably because he never looks at them). however I've never seen anyone leave who has said anything bad about Jervis.

Impressive 2 days for GW.

Are they going to start putting this much effort into 40k?

The funny thing is that with a freely available rules system, you'd think that GW would be able to be more open about making test rules available to a wider audience for testing.

Back in the old days I have done play testing myself and organised testing sessions for a lot of things
like tournament restrictions for Old-hammer, Zombie Skirmish games (before they were cool), additional armies for various games (like adding Space Marines to SST) etc.

To all those who think theory hammer adds more balance I can say that this is wrong.
It helps but because every plan is killed when the opponent roles the first dice it can never replace real play testing.

As an example, how we tested tournament restriction for a new Warhammer 7th army.

10 people meet up, 5 who never played the army before and are not familiar with the rules, 5 who know the faction and who have an idea of what they want to play.

the first 5 get the book and read the rules (not just overview it but read it) while the other 5 are theory crafting and list building

this takes about halve a day and you get all the broken combos, FAQ questions and mistakes in the rules.

than you go into testing, you take the one list that everyone thinks is most broken and play it against 5 different opponents with 5 different factions.
than everyone takes the list he would like to play and do the same.

this takes the second halve of the day and you get a conclusion on how things work out on the table.
Main conclusion is most of the time that the most broken combo on paper is only a theory because it is to situational while something else that is more reliable, but not being noticed in the first run make more problems.

Now switched the players and the ones who are not familiar with the army and the specific play style play it against those who played it in the first run.

than we make the adjustments and restrictions and test it again.
this takes the second day and it is still not perfect.

and I don't think GW has ever done something similar to any of their games, otherwise basic mistakes like special rules that did not work on some units would not happen

Earth127 wrote:
Also a day of brainstorming and calculating numbers can do far more for balance than realistic playtesting ever could.

I say realistic because the logistics of gettting a 100 players playing 2-3 games over the weekend and then report it are massive. Also compared to the calculus work tha tis needed to get those gathered statistics to make sense is more complicated than using the same kind to run theorycrafting.

You could say just provide basic feedback but that only provides knowledge of a problem not how to solve it and balnce is a tricky thing, it's very easy to overshoot your target and make a bad unit OP or vice versa.

Also balance isn't the only factor to consider. Diversity provides increased replayability but makes balance inherently more difficult. SOme of the most fun I have had in games of all stripes is playing it the way Jervis likes: suboptimal but in a style I like.

That's simply not true. Yes, you can use maths/statistics to get a rough idea as a starting point but there is absolutely no substitute for testing.

I work in web design and one of the key principles of anything I do is that you test everything with actual users. You'd like to test with around 6-10 but usually you have to make do with 3-5 people. Even testing with a single person is vastly better than no testing at all.

Sure, balance isn't the only factor to consider but a well balanced game improves diversity as well because more units are useable in any given situation. Also, if you don't care so much about balance and just want to "Jervis" it then balance is completely irrelevant but that's no excuse for the actual designers not to provide a balanced game.

Finally, yes, the logistics of playtesting a game like 40k or AoS are quite involved and there's certainly a lot of effort required to gather and analyse data. You know who should be good at that though? A company whose entire business is based on making games. That's the reason they exist. Other companies manage it. It's tricky but that's not an excuse when it's the core of your business.

Not sure what to say here.

On the one hand, it's more work than I thought they put into playtesting (which is to say nothing.)

On the other hand, a good friend of mine worked for Bethesda Games and EA games as a product manager. Responsible for QA. He had groups of up to 50 playtesters reporting back to him on each release on thousands of points of data.

While I realize GW is not a big software company, this is totally insufficient for something people are supposed to pay money for.

Simon Grant gave an interview today on Warhammer's Twitch feed. He was talking about the fluff of the new Gathering Storm book. He seemed generally much more comfortable talking about fluff and Age of Sigmar than 40K rules, despite being the rules author behind most of the recent rules that have come out for 40K. He was asked how the Ynnari faction interacted with transports. He scoffed, and turned to the page expecting it to be spelled out. When it wasn't, despite being the author of those rules, he said he couldn't tell us the rules for that, because "This isn't the forum for rules discussion", but that an FAQ would be forthcoming. Then they joked about "Nick" whose job it was to write the FAQ's, and implied that he was hired for that purpose. It's hard to tell based on the jokey manner, but they referred to "Nick" the way I would refer to a new hire or intern. Someone that is lower than me on the totem pole.

I would encourage anyone interested in the process to check out the interview:
https://www.twitch.tv/videos/122479057
(Simon Grant starts 1/2 through or so)


So piecing it together based off of the info they've been willing to disclose.
They have a 5-person rules team.
- Simon Grant writes the rules in most or all of the campaign supplements, and new codexes like Genestealer Cults, and Deathwatch.
- Jervis Johnson writes the rules for many of the box games, and main rulebook editions.
- "Nick" writes the FAQs.
- The department manager (whose name they've mentioned, but I forgot). is not one of those 3 people.
- Phil Kelly Writes Fluff, and is likely not a member of the rules team.

It really feels like Simon Grant is the one most responsible for most of the 40K rules. He likes to express a rules philosophy that is very child-like. "It looked cool." "wouldn't it be awesome", but surely there must be more depth to it like considerations of playability, comboing, and balance, but maybe not.