Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

i just read this at boardgamenews.com and immediately thought of poor old GW and their unwillingness to bring back outsiders for playtesting.

<h2 class="title">Ask Fantasy Flight: Retesting Playtesting</h2>

Q: People have said that Fantasy Flight has previously brought out games (Warcraft, Doom) without thoroughly playtesting them and have used expansions to fix the potential problems with the game. Do you think this is a fair or unjustified assessment?

A: I would actually not agree with that. There have been instances where we have brought out a game that didn’t exactly match up with what the gaming public anticipated they would do, but we by no means think that they’re “broken.” However, those criticisms seem to have mostly died out recently, I’d say due in no small part to our intensive playtest program headed up by Mike Zebrowski. We by no means have stopped internal playtesting—I’ve been playing StarCraft and Tide of Iron in the office for many many months prior to release—but we now have a network of playtesters around the world that is totally separate from the company. That benefits us by them having to read and understand the rulebook themselves rather than having the rules explained to them by the game developers, as well as giving us a fresh, “unmarried” perspective on new titles.

full article here
http://www.boardgamenews.com/index.php/boardgamenews/comments/ask_fantasy_flight_retesting_playtesting/

That's what Steve Jackson called "blind playtesting" years ago in his "How to design wargames, vol. I".  Pity that GW doesn't use that model more.  The Techpriests are a start, but when you're more concerned, as a company, with the solidity of your NDA's and keeping things to a "trusted" few, the ability to do really solid blind playtesting is almost nil.

I've been involved in playtesting from just about every angle (running a playtesting group, being a playtester, having rules playtested) and blind testing is virtually always where you get your best feedback.  "in house" testing is a good start to things, but it's the blind testers where your rules are really put to the test. 

Conversely, one of the hardest things I've ever seen in the playtesting realm is moderating the expections of the playtesters themselves.   It's very rare to find a playtester who's willing to ID problems in a set of rules and not, at the same time, develop his own (strongly formed) opinion about how they should be fixed.   And I'd be willing to bet that's at least part of GW's hesitation to opening things up.  Look what happened with their Game Design forum... 

Vale,

JohnS