Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

yukishiro1 wrote:

20 playtesters playing a game a day is 300 games a month. That's a very significant sample size to draw conclusions from, especially if you had a professional playtesting approach where all the data from every match is getting fed into a database, instead of just asking random people to play a few games and say what they thought.

That's a silly amount of money. Well over 1/3 of £1M in wages alone. Then there is payroll taxes and benefits. You're looking at £600K annually for a job with minimum wage employees with little advancement opportunity - so, burn out. Not to mention needing a manager or two to oversee that many people. For them to be worthwhile they'd need to increase sales by more than that. For a game that encompasses several facets not at all interested in precise balance it'd be a terrible investment - especially when perfect balance isn't obtainable anyway.

A couple of people organizing the communications would benefit them more. And behold -- in the past we'd never see them answer questions like this - it would have remained an open item until the next edition, if even that :

Now you're moving the goalposts. If you want to say an investment of much less than 1% of yearly gross revenues wouldn't be worth it to substantially improve game balance fine, make that argument, it's a subjective opinion you're welcome to have. That's not the same argument as saying they "could never play enough games to ever make a significant difference." That's just empirically false.

You also ignored the bit where I specifically said that a full-time playtesting staff wasn't my suggestion. Personally, I think it'd be better to just pay the people they have playtesting now, in return for upping the expectations and making it into an actual professional program where they track data professionally and analyze it meaningfully instead of just giving out books to people with youtube channels in tacit return for hyping the game. I know for a fact that several of the most famous and successful 40k players around have turned down the "opportunity" to playtest precisely because they disagree with giving their labor unpaid in order to pad GW's profits.

If you don't pay people, you can't expect professional results. The GW playtesting program as currently constituted isn't a serious thing, it's mainly just a PR and marketing execise. Don't get me wrong, individual playtesters are mostly giving a good faith effort to do what they're asked, it's just that they aren't being used in a useful way and it is not reasonable to expect that to change without acknowledging that they are providing a service and compensating them for it.

If you don't pay people, you can't expect professional results.

For that matter, paying people to test 40k doesn't even involve putting them on the full-time payroll, it just involves compensating them for their time instead of paying them in "perks" like inside access instead. The perk model encourages playtesters not to rock the boat and not to say anything GW doesn't like because GW is doing them a favor by letting them playtest their product. When you start paying people for the service they provide, you send the message that you actually value what they are doing and expect them to provide you value in return. You don't get professional results from unpaid volunteers, and it's unfair to expect it.

You keep using the word 'professional'. What are you actually trying to get at? (and no that's not an attack etc, I am actually struggling with your choice of wording)

A professional is by definition someone paid, but it has no real bearing on quality. Amateurs are by definition unpaid, but can be and indeed often are superior to professionals. As (a maybe rather gross) generalisation Professionals do it for the money, Amateurs do it for the love. (I'm ignoring the idea of 'professional' in the highly educated sense as that seems way out of left field for playtesting 40k, and you do seem focused on pay)

In the context of playtesting toy soldiers I'd argue you have it backwards. The people who playtest something like 40k for free are effectively self selecting to those who are really dedicated and steeped in the game, you probably can expect a significant portion of them to provide high quality feedback. If you went professional you are getting people who are in it for the money with no real filter on those who will do it well, or do no more than the absolute barest minimum to meet some contract.

Money is simply a convenient proxy for value. From a value point of view perks may actually be far more valuable to playtesters than mere money and hence compensation. E.g. I'd playtest for 40k, and whilst I'd accept money I'd rather have certain perks which money can't readily buy, some perks may have no value to me and other certainly would. The idea that perks encourages a 'company line' and money doesn't is bizarre - in both cases the person playtesting stands to lose out on the 'value' they do it for if they 'lose the job'. Indeed to think that is the case you are in essence accepting that perks are far more valuable than money in order to have that hold on the tester that money wouldn't. When you give perks not readily available for money you also give a signal you value what that person is doing, indeed one could argue more - you have something that others can't and rarity is often value.

Seems GW is doing, or maybe even did it before, the gaming industry. they often don't test their games, or test them get feed back and ignore it anyway, because the assumption is that maybe the game is going to be fixed after people pre order or buy it.

GW is the same 8th seems to be just a phase to get to 9th, and I have a feeling that 7th, 6th etc were the same. They are always working on the next rules and next edition. I mean mid edition they are starting to work on testing rules for next edition, they don't have the time or men power to test and check stuff. I think they assume that if something really bad happens, they will let it stay for anywhere between few weeks to 9 months, and then change it.

I don't think anyone who played any game or sports doesn't imagine how bad two back to back activation phases are. Specialy in a IGUG game. Yet GW let Inari be a thing for years. They only react fast if there is a chance that something they missed may endager the sells they have at the moment. If other marines may not be buying a new codex and models because IH are "too strong", then GW reacts fast. Castellans making other vehicles, which don't fly, worthless is on the other hand okey for almost a year.

When you use unpaid labor, you can't put the same conditions on it you can with paid labor. The way the playtesting program is set up right now is the perfect example of this. They select people not based on their ability to playtest according to a structured, uniform method, but their willingness to donate their time for free in order to pad GW's profits, or, as is increasingly common these days, their willingness to trade their labor for access that will help them with their social media business. The people involved definitely mostly do their best, but they're limited by the amateurish way the program is set up.

GW doesn't do what's necessary to get good, valuable feedback, because doing so would transform playtesting into a job, and people wouldn't want to do it for free any more. It's a lot less fun to play a game where you're logging every interaction for the database, and where you're given strict parameters about what to test with what, in what conditions. So they're stuck giving out books to youtubers, having them play random games based on the stuff they happen to have access to, and then having them fill out basic feedback forms, sometimes even after the thing in question has gone to print, with everyone involved knowing perfectly well that the playtesters involved are placed in the weird position of being unpaid volunteers working completely on sufferance.

Volunteers are great for some things, but they're absolutely not good for running structured, rigorous testing.

Using your logic that volunteers are likely to produce better material than professionals, GW should probably fire its rules development staff instead and have volunteers do it, shouldn't they? Because right now they're just in it for the money.

yukishiro1 wrote:
20 playtesters playing a game a day is 300 games a month. That's a very significant sample size to draw conclusions from, especially if you had a professional playtesting approach where all the data from every match is getting fed into a database, instead of just asking random people to play a few games and say what they thought.

20 playtesters playing a game a day would be about 210 games a day. You only get 1 game per two playtesters since they need an opponent. And they only work 5 days a week, which averages to 21 days a month with variance due to the exact placement of weekends and holidays. And that is before you factor in vacations

Not that you need to play full games to playtests most rules. Serious tournament players often play partial games to gather much of the information they need. Sorta like how chess players often setup end game scenarios to refine their skills. So they could get a lot of information from dedicated playtesters.

That being said, why do that when you can outsource the labor for free. They just need to do a good job of directing those players, gather data, and utilizing that data. Hopefully, they have learned something from the whole Space Marine debacle.

I assume you meant 210 games a month, but that's fine, we can use those numbers instead if you prefer (full-time playtesters could also obviously play more than one game per day, if they were actually full-time). 210 games a month is a hugely significant number, just like 300 is. The point was just to illustrate that even a single game played per day per two playtesters adds up to a very significant number per month.

The reason they shouldn't outsource the labor for free is (1) that it's wrong and (2) that it produces bad or at least mediocre results, as we've seen. GW obviously doesn't give a fig about the first except in their PR marketing, but they should care about the second. As I said, I know for a fact some players that do have the rigorous approach required to playtest systematically have turned them down precisely because they don't believe in working for free to pad GW's profits. If you rely on unpaid labor, by definition you're relying on people who don't value their own labor.

Now maybe GW can't run a good playtesting program period because it can't get it together to do so, whether people are paid or not. But it certainly isn't going to be able to do so as long as it doesn't consider the work playtesters do serious enough to deserve payment.

Well things like open source software exists written by those who volunteer time and I think there are a couple of community based games (wasn't there an old WFB done via some community after AOS).

However, play testing does not need play testers to collaborate; each group of play testers just needs to play/test what they've been asked to and report back.

Game designers do need to collaborate a lot more and agree to a shared long term vision probably for many years over multi versions, which adds a degree of structure/control as you put it that playtesters do not need. Play testing needs structure and control, but that has to come from the company/person in charge of that and not the testers for the most part. It isn't necessary for the testers to share some group vision etc, they just need to test the rules for ambiguity/balance etc.

It is that collaboration and group goal/shared focus that is the weakness of community based stuff; a group of highly talented and motivated individuals can achieve nothing as a group due to disagreement. Anecdote, but I was briefly involved in an open source computer game years ago, it was quickly obvious that most of the time was spent debating how thing should work and rewriting code etc. The game was a few years old when I got involved and was clearly never going to be completed (it wasn't).

That is not an issue for play testing. As I noted earlier, I've play tested mini games before and there are plenty of debates and arguments about the rules and balance and intent etc etc - but it doesn't matter so long as each tester reports back their data and opinions to the person/designer/company who will actually make the decisions.

PS In the sense you talk of you can put exactly the same conditions on testers even if paid for through 'perks'. Money is not needed to form a contract which stipulates said conditions, only consideration is needed and that can be pretty nominal so long as something is offered in return. The tester is no more working under sufferance than the paid person then - they are working under a contract. Whether you'd want to do that is another matter.

Equally if the program is amateurish then that has nothing to do with the testers - and applies no matter whether they pay people or not.

Tyel wrote:

 DeffDred wrote:
Reading this nonsense has brought me to a basic conclusion.
Too many of you think that GW is in the business of making you happy. When in fact they are in the business of making money.
They care not for tears of sorrow or joy.

They do want to keep me (and the broad playerbase) happy so I keep buying their stuff. Which they've broadly managed.

I used to think a dedicated team of playtesters would help, but realistically 20 odd people wouldn't get many games done, and they would have varied views on what is and what isn't balanced (never mind more axiomatic questions of "how should the game play").

Much like every forum discussing 40k ever.

Eh 20 people would do plenty if their opinions was listened rather than ignored.

40k balance issues aren't subtle or hard to spot. Put in semi active player(99.9% posters here　sufficient level) read through codex and he will spot pretty much all glaring　issues.

It's not it would be hard to make more balanced 40k. It's gw doesn"t want to do so that's issue.

The truth is that GW doesn't set out to create an unbalanced game, but it also doesn't really set out to create a balanced game either. It's agnostic on game balance. It's just not a big priority one way or the other. If the game happens to be balanced in a given moment that's fine; if it isn't, that's fine too, unless the balance is so egregiously bad that it causes people to stop playing.

The fact that they don't pay playtesters is a symptom and sign of this. A company that did care deeply about game balance would have paid playtesters (along with a bunch of other things - not suggesting paid playtesters is the be all end all or most important thing they'd change if they cared more about balance).

The reason they shouldn't outsource the labor for free is (1) that it's wrong

No it's not.

That's fine if you think that. We can disagree. It's not a discussion that's going to lead anywhere useful, so let's not start.

As I've said I've playtested for free before for a game I enjoyed and I could see myself doing it again, there is nothing wrong in that - it is for me to decide and volunteer. As people like me exist there is nothing wrong with GW asking. I'm stumped as to why anyone would think it is wrong. We aren't talking about some crappy company exploiting poverty stricken children for less than minimum wage in dangerous conditions or something (that I'm aware of anyway ).

I don't mind you telling me I may not or have done a good job, or questioning whether there are better methods, but questioning the morality of it!?

See, that's what I meant: this discussion is not going to lead anywhere useful. Discussing the morality of for-profit corporations utilizing free labor is not something that's ever going to be useful here. I probably shouldn't have brought it up, I should have known it would produce that sort of reaction. It's not really important to the point anyway, which is that GW doesn't think playtesting is worth paying money for.

Hi. Did playtesting for years at FFG.

The only payment we received was store credit to buy the book when it came out. Didn't really bother us.

We liked getting to see stuff early and help shape products into something better.

Yeah, and that's fine, and it makes sense for companies without the resources to employ professionals. And I'm not necessarily saying having a wider, unpaid playtesting group is a terrible thing either, for that matter. I think it can be part of a good playtesting program. But it's not the same as a program that pays people for their labor, and in return expects a more rigorous approach. The fact that GW doesn't think running a more rigorous playtesting program is a good use of money is telling re: how much they value balance.

...you're trying to claim, with a straight face, that Fantasy Flight Games didn't have the resources to hire playtesters, but GW does?

OK, now I know this tangent is extracting the urine.

 Dysartes wrote:
...you're trying to claim, with a straight face, that Fantasy Flight Games didn't have the resources to hire playtesters, but GW does?

OK, now I know this tangent is extracting the urine.

Ermm, well, FFG is only like 1/10 the size of GW isn't it? They don't operate B&M so there's less overhead, but last I knew they were struggling.

I didn't say anything about FFG specifically. I have no idea what their circumstances were like when HBMC was playtesting for them. I don't see what it has to do with the point here, every single person in this thread surely agrees that GW could afford to pay playtesters if it wanted to.