Based on a lot of bickering about unit quality, (mostly in the Space Marines thread,) I thought it might be useful to implement a 'Tier' list, stolen blatantly from fighting games.
My proposal is that four categories are considered for every unit: Durability, Offense, Mobility, and Force Multipliers. Each of these are ranked from F to S, with S being exceptionally good. (This will be mostly subjective, but hopefully if enough people give numbers then the average will be generally accurate.) Durability and Offense would both by calculated by comparing points to points - If it can kill its points back in two turns, for example, it's above average.
Once all the factors are taken into account, the highest two are averaged. If the third best result is as good as the first two, it gets a +. If all four options are equally high, it goes up a full letter ranking. If the two lowest options are both D or F, then it instead gets a -.
Once this is worked out, an extra line for subfaction bonuses can be worked out. (For example, most dreadnoughts are a full letter grade more durable when in an Iron Hands army, and this could be noted in a sidebar.)
The categories would generally go like this:
Durability:
F - This unit will die quickly, even when targetted by bad matchups. (A tank which can be killed by equivalent points worth of anti-infantry guns, for example.)
D - This unit will die easily, but subpar matchups will not kill it quickly.
C - This unit will be killed in three turns by an equivalent amount of weaponry designed to kill it. (A tank killed in three turns by average anti-tank weaponry.)
B - This unit requires dedicated firepower to kill.
A - This unit requires a considerable amount of focused firepower to kill, and will absorb far more than its points worth in focused fire in order to die.
S - This unit is almost indestructible. Killing it requires significant luck, special strategies, or dedicated counter-picks.
Offense:
F - This unit cannot hurt other units, or the firepower is so negligible as to be effectively nonexistent.
D - This unit's firepower is either difficult to use (IE a slow melee unit) or well below average, but can hurt the right target in certain circumstances.
C - This unit will kill the equivalent points cost of what it is designed to kill in about three turns, or deals some damage against many types of units.
B - This unit will either kill what it is designed to kill in about two turns, or is an effective weapon against many types of units.
A - This unit can kill equivalent points cost of what it is designed to kill in one turn, or is very damaging to many types of units.
S - This unit will kill equivalent points or greater against almost anything it can target, and the best way to counter its damage is to keep from getting hit.
Mobility: (Note - Units with any sort of Deep Strike ability increase their mobility by a full letter rank.)
F - This unit cannot move
D - This unit is slow enough that its ability to be used effectively is hampered
C - This unit could move 48" before the end of turn 4 without losing the bulk of its offensive capability. (For example, a dedicated melee unit can advance without losing offensive capability as long as it doesn't advance on a turn where it would want to charge.)
B - This unit could move 48" before the end of turn 3 without losing the bulk of its offensive capibility.
A - This unit can move 48" before the end of turn 2 without losing the bulk of its offensive capability.
S - This unit can move freely to almost anywhere it wants without restriction. Players have to be ready for this model to be almost anywhere on any turn.
Force Multiplier:
F - This unit provides no benefits except for the damage it can deal
D - This unit provides a small buff, debuff, or benefit to the army it is in, but this buff is minimal or can be obtained cheaply elsewhere. (For example, most Troop choices provide Command Points and can hold objectives.)
C - This unit provides a buff which is either niche and of average utility, or broadly applicable but minimal.
B - This unit provides a buff which significantly alters the units around it, and can be taken only for its buffing ability.
A - This unit provides a buff which makes otherwise ineffective units effective, or which would increase the effectiveness of multiple units by a full letter grade.
S - This unit is so beneficial that it singlehandedly makes an army, faction, or playstyle competetive. (Think early-edition Guilliman.)
Note that universally accessible faction stratagems should be taken into account, (such as Da Jump, which all orks can use,) but faction-locked abilities should only be mentioned in a sidebar, and unit buffs should only affect the score of the unit who provides the buffs.
Automatically Appended Next Post:
To add a few examples:
(Edited to add this note: I MIGHT BE WRONG ON THESE. Getting multiple opinions is the point! I might think too highly of Ork Boy durability, but if we get enough people to add their thoughts on various units, we can get a broadly accurate final score. I'm going to try and avoid arguing about other people's scores unless there is a massive blatant error, (Like how I put ork boy mobility at B
) since the point is to get an accurate score by comparing many results.)
Tactical Marines
Durability - C (60-80 points of average anti-infantry fire will kill Tactical marines in three turns.)
Offense - C (Their weaponry will kill 60-80 points of infantry in three turns, depending on specific gear and matchups)
Mobility - D (They will struggle to move across the board without giving up shooting and melee)
Force Multiplier - D (Providing Command Points and Objective Secured)
Total score: C-. With Combat Doctrines and Chapter Tactics in place, their durability and offense increases make them a C+.
Ork Boyz
Defense - C
Offense - B
Mobility - C (I originally posted this as B, which was pointed out as way wrong)
Force Multiplier - D
Total score: B (With the right Kultur, namely Evil Suns, this improves to a B+ or even an A-.)
Leviathan Dreadnought:
Durability - A
Offense - B
Mobility - D
Force Multiplier - F
Total score: B+ (With Iron Hands, Mobility becomes C, Durability becomes S, and Offense is somewhere between B+ and A-, so the total becomes an A+.)
~16000 Astra Militarum: 
~1200 | Imperial Knights: 
~2300 | Leagues of Votann: 
~1300 | Tyranids: 
~3400 | Stormcast Eternals: 
~5000 | Kruleboyz: 
~3500 | Lumineth Realm-Lords: 
~700
