Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

So I was chatting with GMMStudios about painting scores and the like and was thinking about how to refine the painting rubric (checklist) so account for proficiency.

First, I suppose, is why use rubrics at all? Painting an army is an artistic pursuit, and why do you need a checklist to tell which is better than which other? Well, I guess because we're not grading golden daemon entries, so counting the flaws isn't going to work. Also, because everyone at a tournament needs to have a paint score, so you can't just rule out everyone but the top three armies, you have to score them fairly too.

This said, the system I'm proposing is probably more time intensive than some of the rubrics I've seen at GTs (including independent GTs). Finding enough qualified judges is always going to be an issue. There's going to be a balance between how much time you want to spend looking at each army, and how precise the scores can be. For the rest of this discussion, I'd like to make the assumption that you're got enough time/judges to accomplish whatever is deemed the best.

So, the current rubrics I've seen tend to be simple yes/no checkboxes, where you earn points for doing certain things. As I see it, there are two problems with this:

1) People paint to the checklist, which may or not be a problem
2) There's no real distinction made between doing something, and doing it well.

There are some secondary issues that can be discussed as well.

3) Balance between painting and conversion points
4) The difference between character and centerpiece models, and rank&file models.


With these things in mind, here's a first-pass at a rubric that attempts to address these issues.

For each item you get a 0-5 rating, to indicate how well it is done. I'll try to include some examples...


Rank & File points: These points should be assigned on the basis of the majority of the army, focusing on the generic models

1) Army is assembled - A 0 rating here would mean that any of the models weren't completely assembled - missing arms and the like. A 3 would indicate that mold lines had been removed, a 5 would indicate further assembly effort had been made, such as taking the time to drill out bolter barrels.

2) Army is painted to a three-color standard - The 'quality' of this item is reflected in how well the models are painted. A '1' here would mean that there are blatant mistakes (going over the lines), or that the paint is patchy, with primer or bare-metal showing through. A '3' here would mean that there may be some minor mistakes, but for the most part the models are painted decently. A '5' would mean that there are no visible mistakes, the brushwork is smooth, and so on.

3) Army is shaded/highlighted - 0 is not shaded. 1 is a sloppy drybrush or inking job. 3 is standard tabletop shading, while 4 and 5 are reserved for examples of excellent brushwork, blending, blacklining, and the like.

4) Details are picked out - 0 would be no details picked out. 1-2 mean some details are picked out, but not all, and that they're not painted cleanly. 3 means most, if not all, details are picked out, and painted mostly within the lines. 4-5 are for extra work, such as shading/highlighting belt buckles.

5) Artistic effort - This is for freehand work, decals, unit markings, battle damage, weathering, and things of that nature. 0 would indicate that nothing was done here. 1-2 would mean that attempts were made, but that the results weren't that good. Decals that were applied crooked, or with film showing would go here. 3 would be a consistent good job on simpler effects, freehanded unit markings or chapter symbols that are done decently, decals that are applied well, and so on. 4 or 5 would be more intensive markings, camo patterns across the entire army, tattoos or warpaint for all guys, and so on.

6) Conversion work - For the entire army, a 0 would indicate that no visible conversions were done. A 1 would mean a few very straightforward part swaps (including swaps from after-market companies, including forgeworld), a 2 would be a majority of models had basic swaps done. (This might include a counts-as army, where a fantasy range was used as the basis for, say, hruud). A 3 would mean that the whole army was converted in some more obvious way, or a few units featuring heavy hand-done conversions (plasticard/greenstuff), and 4/5 would be reserved for entire armies that were fully redesigned.

7) Basing - 0, models have no basing. 1, basing is basic and done sloppily. 3, basing is fairly standard and done neatly. This can include resin bases or modelling material bases. 4-5, there's something extra about the basing, such as freehand design work on individual tiles...

----

Showcase Points - These additional points would be based on models that stand out from the army standard, with an eye towards characters, showcase vehicles, and monstrous creatures

8) Artistic Effort - Like above, but with showcase models, the base standard is higher. 0 indicates no differences from the rank&file. 1-2 would indicate that efforts were made to have the characters stand out, 3+ would get into the realm of freehanded banners, or murals on tanks.

9) Conversions & Basing - Grouped together for centerpiece models. How much effort went into making the centerpieces stand out? Are they on more ornate bases, with customized poses and equipment? How much extra cool do the centerpieces look.

----

10) Overall impact - This is the final catchall category that allows for a judge to reward those armies that really come together as a whole well, even if they're somewhat technically lacking. Things to consider here include colour choices, model posing, aesthetic choices (were the converted tanks done just to score conversion points or do they look cool?) This is pretty much the place for a judge to reward discretionary points. Lower scores (0 or 1) might indicate that the army doesn't look cohesive; that some models are based completely differently than others, or that models from several different armies were put together to compose this force. A 2 would be an army with a cohesive paint scheme, with uniform basing, that doesn't really do anything special to stand out from the crowd.


So, we've got 10 criteria that we're grading on a scale of 0-5, yielding an absolute score of 0-50. But, in reality, the actual range should be much smaller, as being assembled and painted to a 3-colour minimum is required to enter the tournament.

Grading the 'average' army (based on the majority of what I've seen in two years judging the Adepticon Team Tournament), I'd expect it to see scores like:

1) Assembled: 2 - Most people get their models together, and take care of some, but not all, of their mold lines
2) Painted to 3-color standard: - 3 Most people get the majority of their paint in the lines, and applied well
3) Shaded/Highlighted: - 3 Most people do basic drybrushing, and washes neatly
4) Details: - 3 Most people pick out the details on the models without going over the lines
5) Artistic Effort: - 2 Some people make an effort to include unit markings in some way. Not everyone does. More people do some battle damage than weathering, but the majority of armies feature neither. Few people do much freehand on their entire army
6) Conversion Work: 2 - Most people have some weapon swaps throughout the army.
7) Basing: 3 - Most people have clean-looking bases. More are moving towards resin-bases, but many of these are simple dry-brush work
8) Centerpiece Artistic: 2 - Most people's characters have a little more effort put into them
9) Centerpiece Conversions: 3 - Centerpiece models almost always stand out from the rest of the army as conversions. The exception to this is Special Characters, where most people running a special character are hesitant to convert it, because they want their opponent to know who it is easily.
10) Overall Impact: 2 - Most armies look good when set out for display. Most people run the basic cohesive army.

That means that the average army should score about 35 points. Getting each additional point over these basics is meant to be progressively harder (why there is both a '4' and a '5', to give at least two degrees above average to work with).

I'm torn on the practice of 'capping' scores. On one hand, it means that an excellently painted army with minimal conversion work (lots of eldar armies fall into this category) can still compete for best appearance. On the other hand, if one army is expertly painted and fully converted, and another is expertly painted without any conversions, doesn't the converted one deserve more points? (I'm sure this, alone, is worth a thread's worth of discussion).

This does, however, lead to the question of granularity. If the average army scores 35/50 points, that's only 15 points lower than the theoretical maximum. Assuming that no one in contention for best overall is going to run an army that would score less than 30 points, that's a effective contribution of only a 20-point swing based on painting scores...

This can be manipulated by a tournament organizer in a couple of ways. At some point, you have to establish how many points (of all types) you want there to be available. Let's say that the TO wants battle points to be worth 60%, and painting and sportsmanship to each be worth 20%. If it's a 5 round tournament, with 30 battle points possible each round, that sets the total battle points at 150. In order to meet the TO's ratio goal, that means painting needs to be worth 50 total points... how convenient.

But, let's say it's only a 3 round tournament, and the TO doesn't want to rewrite the missions. Now there are 90 possible battle points. In order to maintain the ratio, painting points need to be worth 30 points. So, take the score from the rubric, divide by 50 and multiply by 30, and now you've got a result on a 30 point scale. Real easy so far.

But, that means that our average score is going to be a 21, and our max score is only 30. In reality, painting isn't counting for 30 points, it's counting for little over 10 points towards the total, with everyone getting about 20 points.

The key to solving this problem is deducting a base number from everyone's score, prior to applying the ratio. For example, if we wanted to make paint scores worth more, relatively speaking, while still having them account for 20% of the total points, we could deduct 20 points from every scored rubric. Now, the average score is a 15/30, instead of a 35/50. Going back to our 5 round tournament, we need to get the total up to 50, so we multiply by 5/3, and end up with a 0-50 range, where the average is worth 25.

Doing this has two added benefits. First, it makes the painting score actually vary the overall a bit more. When actual battlepoints can vary from 0-150, having actual painting points vary from 30-40 has almost no impact, and I don't think that's what is intended at any tournament. Even having them vary from 15-45 is a very small variance compared to the point differences from even one game.

Secondly, it allows the TO to use the same rubric, regardless of how many rounds the tournament is, or what importance they want to attach to soft-scores. Simply apply the desired ratio to the rubric's total and you're good to go.


I got a bit rambly there... what do you guys think to any of this stuff (aside from the "tournament's don't have soft scores" crowd. You're free to completely ignore this, because it clearly doesn't matter to you.)

I personally hate subjective 0-5 type scoring. I've had it personally screw my score more times than I can count. A checklist allows people to have a good idea of what they'll be sitting at prior to the tourney as well as showing them where they can pick up a few more points. It also keeps down calls of shenanigans when paint scores are revealed. My personal opinion and your list would work but based on a lot of tournament going experience I'd say checklists are the better way to work it at that level.

I cannot speak for other organizers, but in the case of our events (SCGWL), transparency and objectivity are our primary concern with all aspects of our scoring. Painting presents a difficulty, because it requires a certain degree of subjectivity and the scores are not something that you get to see openly until the end of the event. As a result, we have always used the GW checklist to limit the amount of subjective influence on the score. In this way, you eliminate the personal preferences of the judge (who may not care for your color scheme or conversion choices) and give the more average painters clear ways to improve their scores.

The problem with checklists, though, is that how do you know what meets a checkbox?

I mean, a quick GIS for free hand shoulder pad turns up both of these:



and



but also



and




If you just have a checkbox, does the Ultramarine shoulderpad get the same checkbox that the Apothecary pad does? Does it get a checkbox that someone who used a decal doesn't get, in spite of the fact that in some cases, the decal may look better than the freehand symbol?

Or do you use multiple checkboxes? (Has freehand, Has good Freehand, has excellent freehand) - In which case, how is having three checkboxes any different than having a single box with a range from 0-3?


Put another way, if the 0-5 scale is defined well enough that it could also be represented as additional checkboxes to show varying skill levels, how do you feel about that?

Lots of good thoughts. One system a couple of well-regarded WH tournaments on the East coast are experimenting with is a panel of painting judges who actually give the player feedback at the time, and explain the score.

http://www.thecolonialgt.com/forums/viewtopic.php?t=2696

I hope you get something good out of this. I have pretty much given up on paint scores.

G

I'd love to be able to have a panel, unfortunately, depending on the size of the event, it just isn't feasible.

Getting four judges to look over 110 teams between them pushes the time limit. Getting four judges to look over all 110 teams would be a logistical nightmare.

On the other hand, I really believe in giving the feedback to the players. People should be able to see how their checklists were scored. I've volunteered to enter that data from the TT this year, but didn't get the sheets, and don't know where they are now. If they were saved at all.

Hulksmash wrote:I personally hate subjective 0-5 type scoring. I've had it personally screw my score more times than I can count. A checklist allows people to have a good idea of what they'll be sitting at prior to the tourney as well as showing them where they can pick up a few more points. It also keeps down calls of shenanigans when paint scores are revealed. My personal opinion and your list would work but based on a lot of tournament going experience I'd say checklists are the better way to work it at that level.

Seconded.

Most important, I'm on the transparency boat when it comes to painting scores. I want my players to know exactly what they're going for and how to earn those points. In longer events, the more points there are to earn (or not!), the more a given category of scoring can skew the results.

And with the example of hand-painted shoulderpad details, what if the two guys on the bottom busted their butts to paint those markings, and that's the best they can do for their skill level, and it took them as long or longer to do that work than the guys with little bits of beauty on the pads? Do they deserve less credit because their ability is at a different level? I guess I'm a bit of an egalitarian when it comes to painting, but I see painting as an integral part of the hobby, and therefore a tournament player's presentation, but it shouldn't be so heavily weighted as to make it an avenue to overall victory.

Back on topic, though, instead of a blanket deduction to keep the painting score in line, why not try a rubric where only a set number of points (depending on the length of the event and the weight of various other categories) counts toward a player's overall score, and any bonus points earned are used to help determine awards for best army/painted? That way everyone gets the points they deserve for completing their army to the best of their ability and no one that makes the effort gets kneecapped for not being at the top of the curve, but you have a tangental mechanism to reward those players who do put time into developing their hobby and producing really beautiful armies.

-Dis.

disdainful wrote:
And with the example of hand-painted shoulderpad details, what if the two guys on the bottom busted their butts to paint those markings, and that's the best they can do for their skill level, and it took them as long or longer to do that work than the guys with little bits of beauty on the pads? Do they deserve less credit because their ability is at a different level? I guess I'm a bit of an egalitarian when it comes to painting, but I see painting as an integral part of the hobby, and therefore a tournament player's presentation, but it shouldn't be so heavily weighted as to make it an avenue to overall victory.

Are we judging effort, or judging results? Just because something is the best someone can do doesn't mean they should be winning best-painted awards.

Maybe I'm old fashioned, but I prefer rewarding talent, and don't buy into the whole "everyone's a winner" mentality that seems to have been bred into the Gen-Yers.

No way in hell to judge effort. Judging the results is hard enough.

"E for Effort!"

"Um, that's below D, sir."

"E-xcellent."

"Still below D."

"Above F, though?"

"Noted, sir."

Redbeard wrote:

Are we judging effort, or judging results? Just because something is the best someone can do doesn't mean they should be winning best-painted awards.

Maybe I'm old fashioned, but I prefer rewarding talent, and don't buy into the whole "everyone's a winner" mentality that seems to have been bred into the Gen-Yers.

I never said they should win best painted awards. They should not, however, be relegated to a lower finish overall because of their painting score. Nor did I say I think 'everyone's a winner'. If someone paints an army to the best of their ability, and is the best sportsman at the event with the best army to play against (if you're scoring comp and sports), and performs the best on the tabletop, then they deserve the top prize. Regardless of whether or not their hand-painted details are masterworks.

All of this discussion hinges on how painting and other scores are weighted in the event anyway. Your rubric is fine, albeit unquantifiable, by itself. Now, plug it into the standard 3-round, one-day GW RTT points scale: all of a sudden the subjective painting score is triple-weighted respective to other scores besides battle points and a player with an amazing looking army can easily take first despite shortcomings in other areas. If we use the rubric as a part of a larger multi-day event, where there's hundreds of points at stake, then a 30-point scale will probably dissolve into the overall score much better.

As I said before, if you cap the number of painting points that contribute to the overall score, while allowing further points to contribute to a 'best presentation' or 'best army' award, you've got a level field where anyone can win the tournament, based on their performance as a whole, and the players who excel in the hobby side have a standalone prize they can compete for with their paintbrush. Otherwise, you're creating a bias toward players who paint well.

-Dis.

Looking at this from the opposite angle...

What if each contestant that wanted to have their army judged on painting, were provided with, and expected to fill out, a painting score for their own army exactly as if they were a judge... with added areas to give a brief justification for the scores they gave themselves? That way the real judge can look the sheet over, will have the presenters notes on what they think has qualified them for each score, and their justification for it. The judge can of course, them make their own calls, but if a major the aim of the judging is to help give direction to lesser skilled painters, what better way to do that then to view their own opinions/misgivings about their own work? I think it would make life easier for the judges and would allow people to point out (in their own judging form) where their conversions and hand painting is in the army... which a hurried and harried judge might miss. Also makes people do a reality check about their own painting and modeling skills before the competition.

I say this because I've seen cases where the judges have clearly missed criteria that were fulfilled by the army presented, or didn't realize the extent to which figures were converted or free-hand painted (I'm not blaming the judges or claiming mis-deeds). The above system puts the onus is on the presenter to make the judge aware (in writing) of their extra work, thus allowing the judge to give them an appropriate score. One or two minutes of reading might make a big difference in understanding the army that you're presented with.

Otherwise, you're creating a bias toward players who paint well.

I think that is EXACTLY what the point of having a painting score is. Similarly we are even more biased towards players that play well, as battle points are generally several times the points you can get for painting.


Automatically Appended Next Post:

Gitsplitta wrote:What if each contestant that wanted to have their army judged on painting, were provided with, and expected to fill out, a painting score for their own army exactly as if they were a judge... with added areas to give a brief justification for the scores they gave themselves? That way the real judge can look the sheet over, will have the presenters notes on what they think has qualified them for each score, and their justification for it. The judge can of course, them make their own calls, but if a major the aim of the judging is to help give direction to lesser skilled painters, what better way to do that then to view their own opinions/misgivings about their own work? I think it would make life easier for the judges and would allow people to point out (in their own judging form) where their conversions and hand painting is in the army... which a hurried and harried judge might miss. Also makes people do a reality check about their own painting and modeling skills before the competition.

An interesting idea, but I don't think it's workable. Just getting people to bring their army lists and copies for their opponent, rulebook, and their own dice, is tough enough. Expecting people to do their homework and fill out a painting criteria would get mixed results at best. Also puts a lot more work on a painting judge/judges the day of the tournament. You'd have to simultaneously take people seriously, and also lookout for the BS masters blowing smoke about how good their army is. You'd just build in new bias based on how good of an advertising campaign someone could write up.

Re: the freehand shoulder pads.
It should just a checkmark, 1 point kind of thing. With the figures that are shown, the freehand quality is a symptom of larger (or smaller) talent. The GREAT free-handers will have top marks all through the painting score, while the ones that are of lesser qualities will have junk scores from other areas.

Re: checklist in general.
The best toury that I have ever been to had a check list for painting scores. I knew what to expect before I ever walked through the door because of the checklist. The checklist had 7 areas, with a max of 5 points (out of 7 or 8 options) in each area. There was a cap of 30 points max for painting, so that excellent painters were not out of the hunt for top spot if they could not convert worth beans. Since it was just a checklist, there was no issue about what constituted “well painted”.

While it would be amusing to have a master painter's tournament where every model
is moved at a snail's pace because varnish ruins the finish and so nobody wants to scratch the
paint on their models, if you're going to add painting score towards the final result then
go whole hog. There are (usually) separate awards for all the categories anyway.

disdainful wrote:
I never said they should win best painted awards. They should not, however, be relegated to a lower finish overall because of their painting score.

What exactly do you define overall as? If a golden daemon winner painted an entire army, and "did their best" on the tabletop, would they deserve best overall? Do you believe battle points should be capped as well, so that those people who can't play well still have a chance at the overall prize?

If someone paints an army to the best of their ability, and is the best sportsman at the event with the best army to play against (if you're scoring comp and sports), and performs the best on the tabletop, then they deserve the top prize. Regardless of whether or not their hand-painted details are masterworks.

This can easily be turned around to read if someone plays to the best of their ability and is the best sportsman at the event with the best army to play against (if you're scoring comp and sports), and has the best painted army, then they deserve the top prize, regardless of whether or not they won any games. It sounds pretty stupid doesn't it. If you're going to award best overall to someone regardless of how well they do in one area of the tournament, you might as well not have that scoring element.

As I said before, if you cap the number of painting points that contribute to the overall score, while allowing further points to contribute to a 'best presentation' or 'best army' award, you've got a level field where anyone can win the tournament, based on their performance as a whole, and the players who excel in the hobby side have a standalone prize they can compete for with their paintbrush. Otherwise, you're creating a bias toward players who paint well.

You make it sound like decent painting scores are a right, not something that should be earned. We don't award best general to the person who downloaded the best list off of YTTH, we reward the person who played the best against the opponents they faced that day.

I mean, god forbid that there is a bias in the overall scoring for people who can do all aspects of the hobby well. What was that definition of overall again? Again, let's just spin this a bit..., If you put a cap on battlepoints, you've really got a level playing field where anyone can win the tournament based on their performance as a whole, and the players who excel at making good army lists have a standalone prize (best general) they can compete for with their dice. Otherwise, you're creating a bias toward players who roll well.

Gitsplitta wrote:
What if each contestant that wanted to have their army judged on painting, were provided with, and expected to fill out, a painting score for their own army exactly as if they were a judge...

Clipping the rest of your post to save space, I think this is possibly a very good idea. It goes a long way to helping remove judge error, but it can also be prone to inflating scores, especially in a simple check-box situation. I've seen people who believe that slapping FW shoulder pads on a model counts as conversion work (really? you just spend more money and get parts that fit perfectly and require no effort, and think you deserve extra points for this?), or that believe that the sort of painted shoulder pads above should earn them freehanding points. If there are checkboxes, I don't think they should be checked for making an effort, they should be checked for achieving a positive result.

In my ork army, almost every boy has some bitz swaps, but they're all from other ork kits, and seriously, is it worth points that one of the footsloggers has an arm from a biker and one of the bikers uses a loota's head? Technically those are swaps, and technically, every model in the army has at least one component from a kit that it didn't come with, but really? I mean, they're all parts from the ork range, and the casual observer would probably never know. I'd never fault a judge for not counting those as conversions.

Redbeard wrote:
In my ork army, almost every boy has some bitz swaps, but they're all from other ork kits, and seriously, is it worth points that one of the footsloggers has an arm from a biker and one of the bikers uses a loota's head? Technically those are swaps, and technically, every model in the army has at least one component from a kit that it didn't come with, but really? I mean, they're all parts from the ork range, and the casual observer would probably never know. I'd never fault a judge for not counting those as conversions.

But see you've already negated your scoring by this statement. Part swaps are conversions. Just because the models are from the same range doesn't mean it wasn't a conversion. In fact you could almost say that unless the person judging knows all the kits from all the armies then they will never be able to tell what a conversion is or the extent without massive green stuffing/rebuilding. I personally think Painting (due to it's complete subjectivity) should be a seperate prize that doesn't add in to overall.

I've noticed that my marine armies recieve much better paint scores vs. my Nids or Orks. The reason seems to be that clean is better but bugs and orks aren't clean armies and therefore don't look the same. Bias. Which a list with 0-5 scoring throughout only enhances. Basically keep it as open and easy to understand as possible. If you want to keep a 3-5 point swing for the most amazing armies there then feel free. Basically the best painted recieves 5 extra, 2nd best painted recieves 4, and on down to the 5th best painted. This way excellent painters are rewarded but it's not a ridiculous swing. But an open 0-5 is a recipe for non-clarity and future issues.

How 'bout you simply have different levels of entry for tourneys:

-Tournament Only: one fee, you play in tourney, top prize is "Best General" and based solely on battle points
-Painting and Modeling Only: one fee, you do not play in the tourney (but perhaps there is a display games area where you can play for fun while the competitive tourney is playing), and are judged on painting and modeling by a judge panel, top prize is "Best Painted"
-Overall: pay both fees, you play in tourney and are ALSO judged on painting, eligible for both "Best General" and "Best Painted," plus a separate "Best Overall" prize

Prizes for each are paid for out of their respective pools of entrants. This way, all players can choose what they are interested in and eligible for their respective top prize(s). If there are a large number of players for the tournament and few in the painting competition, for example, the prizes would slant heavily towards the Best General award. If there are few players going for Overall, the prize will be relatively small.

I am assuming that sportsmanship type stuff is REQUIRED, not scored, and if scored, it's minimal and probably only a penalty for being a d-bag. Similarly, comp is not scored and at most, used for first round pairings (though I would favor simply random draws).