Tournament Analysis for 10th

Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you:

No adverts like this in the forums anymore.
Times and dates in your local timezone.
Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance.
Email notifications for threads you want to watch closely.
Being a part of the oldest wargaming community on the net.

If you are already a member then feel free to login now.

**Dysartes**

Morning all - I had been putting some legwork in the Prediction Time thread to look at 9th ed events and provide some analysis of them. Unfortunately, I was using quite a manual method - many Notepad files - so I'm looking to revise this methodology ahead of 10th ed events beginning later in the year.

On the assumption that this sort of analysis does prove useful to people, I wanted to take a bit of time to figure out the sort of metrics that people were interested in before I started to put anything together (and test it, using 9th ed events).

The things I was looking at in the other thread include:
Per event
- Number of entrants
- Number of entrants with no recorded results
- Number of entrants who dropped during the event
- Submitted faction (and subfaction), as per the BCP Roster tab*, with final places
- Breakdown of factions by codex (so consolidating sub-factions into factions), with final places
- Participation %, by codex
- Top n slots, and my thoughts as to who is over/underperforming (where n varies by event size**)

Per period of events (usually a week)
- Total count of sub-factions for SM (with top n spots highlighted)
- Total count of sub-factions for other factions (with top n spots highlighted)
- Breakdown of combined "participation vs. top n slots", by event size
- Space Marine Win Records (as that was a focus of t'other thread)

Given what we know of 10th so far, I'm going to assume that knowing which detachments are taken would be useful.

It'd probably be useful to capture which "period" of the edition each event (or set of events) falls within, probably defined by the most recent Codex (or FAQ/errata/dataslate) to be released, so it'd be possible to see the impact of these changes as the edition moves on.

That leaves me with three main questions at this point, before I start going to poke spreadsheets with a stick:
1, Are there any other metrics people think might be interesting to try to pull out of the BCP data?
2, Is there a cost for signing up with BCP? I've been pulling info without an account, but any info regarding specific lists is going to need me to sign up.
3, Is there such a thing as an event too small to be worth looking at?

* - I'm really hoping this is accurate, as I don't want to have to be double-checking it for each player at each event...
** - I'll need to codify the top n for each breakpoint for event sizes - given a Swiss system, that means messing with powers of 2. I'll have a think, and post an initial stab at it in this thread later.

**Slipspace**

There's already a guy who collates tournament data in a Meta Monday thread on the WarhammerCompetitive Reddit, which also gets linked in lots of FB groups. I believe there's now a website where he collates all the data too. He used to just post it all on Reddit. The data tracks faction/sub-faction play rates and win rates and is broken down by "season". The only useful info it doesn't automatically capture is non-SM subfactions (I think this is actually captured, just not reported) and the participant info about number of tournaments and number of drops. The plan for 10th seems to be to do the breakdown by detachment rather than sub-faction.

BCP sub-faction is notoriously poor data, sadly. It's self-reported for the most part, so you sometimes end up with someone putting their sub-faction as, say, Tzeentch, when they have 1800 points of Thousand Sons and a single Daemon unit, which I don't think is helpful. For event size, probably anything below 20 is not worth bothering with, but it might depend on the number of events we see in a typical week/month.

**Dysartes**

Just because one person is doing it, though doesn't mean others can't as well - and depending on what you look at, you might find something that the other hasn't considered.

I definitely think any events of 16 players or fewer are too small to probably include. I guess the big question mark is about events of 17 to 32 players - I'd rather use one of the power of 2 values as a cut-off, rather than splitting one section in half.

In terms of top n places, I think I'd want to revise how I was doing it a bit, but it might mean that the top n range ends up being quite wide as events get larger. Something like this...
17 to 32 players - top 4
33 to 64 players - top 8
65 to 128 players - top 16
129 to 256 players - top 32
etc - fortunately, I don't think we see too many events over 256 players. Note that in this context "player" would refer to entrants with at least one recorded game.

As for factions and sub-factions - and it is an assumption that sub-factions will be there in 10th, but I'm not ruling it out just yet for non-SM factions - I know I've seen instances where what was displayed on the Roster tab is then contradicted by what is shown on the Results tab. I don't think your example of a Tzeentch army is a problem, though. 1800 pts of Thousand Sons and 200 points of Daemons will be using Tzeentch as their Faction Keyword, as far as I'm aware, so that makes sense to me. For me, it would only become a problem if people pick Tzeentch for an army that was just Tzeentch Daemons.

**Aash**

Dysartes wrote:
Just because one person is doing it, though doesn't mean others can't as well - and depending on what you look at, you might find something that the other hasn't considered.

I definitely think any events of 16 players or fewer are too small to probably include. I guess the big question mark is about events of 17 to 32 players - I'd rather use one of the power of 2 values as a cut-off, rather than splitting one section in half.

In terms of top n places, I think I'd want to revise how I was doing it a bit, but it might mean that the top n range ends up being quite wide as events get larger. Something like this...
17 to 32 players - top 4
33 to 64 players - top 8
65 to 128 players - top 16
129 to 256 players - top 32
etc - fortunately, I don't think we see too many events over 256 players. Note that in this context "player" would refer to entrants with at least one recorded game.

As for factions and sub-factions - and it is an assumption that sub-factions will be there in 10th, but I'm not ruling it out just yet for non-SM factions - I know I've seen instances where what was displayed on the Roster tab is then contradicted by what is shown on the Results tab. I don't think your example of a Tzeentch army is a problem, though. 1800 pts of Thousand Sons and 200 points of Daemons will be using Tzeentch as their Faction Keyword, as far as I'm aware, so that makes sense to me. For me, it would only become a problem if people pick Tzeentch for an army that was just Tzeentch Daemons.

I suspect that subfactions as they stand in 9th won't translate directly into 10th, and detachment names will be the most useful metric for differentiating between one army and the next.

**Dysartes**

If they don't end up being there, they don't end up being there - I'm not expecting them in the Index Era of 10th (outside of SM), but they may crop up as factions start to get their Codex released.

And assuming a spreadsheet is being used, we're just talking whether a column is used or not.

**Bosskelot**

I'm curious as to why you want to do this when not only does the aforementioned meta mondays exist, but you've also got stat-check and goonhammers own stats site too.

Stat-checks meta data dashboard is basically everything you were looking for.

**Canadian 5th**

Bosskelot wrote:
I'm curious as to why you want to do this when not only does the aforementioned meta mondays exist, but you've also got stat-check and goonhammers own stats site too.

Stat-checks meta data dashboard is basically everything you were looking for.

I compile my favorite hockey team's stats into tables and post them to a forum I frequent even though there are already plenty of places where people can go to see said stats. I enjoy doing it and I compile things in a way that is, hopefully, relevant to my audience and not exactly the same as what can be found just by typing in "[player name]" into Google.

I can see Dysartes being much the same as I am.

**Dysartes**

I mean, you look at this post and tell me I don't like messing around with data.

**JNAProductions**

Dysartes wrote:
I mean, you look at this post and tell me I don't like messing around with data.

You don't like messing around with data.

In all seriousness, as someone who likes doing probability math, you go nuts. Have fun! And hopefully you'll glean some good insights.

**vict0988**

Have you considered looking at how unit choices affect win rates?

**Daedalus81**

I have previously built a script that calls into BCP and queries all tournaments. Once I recover my old PC I'll kick it over if you want to tinker. Fairly basic cURL stuff. Meta Mondays are great, but they generally focus on bigger games as does the dashboard so there's some data at "the bottom" that remains uninspected.

You may not need to sub to BCP, but if you do it's $5/mo iirc.

I am not sure how fast, but FLG will have to reconfigure the options for detachment selection so that we see the right allocations.

As for army lists - that's the holy grail. I didn't spend much time troubleshooting the issue I had, which I think is a permissions restriction until the tournament is done to prevent other players snooping on lists. Obviously the data can vary so a really competent parser will be needed, but there is a framework for tackling Battlescribe format.

**Dysartes**

Yeah, getting army lists seems to need a subscription - which is annoying, when I initially just want to verify if people are playing the faction/subfaction they signed up as.

Looking at units would be interesting, but I guess there's a question of how granular you go - units with options for different loadouts, for example, or different unit sizes. Even if multiples of units change things.

In theory, I'd consider looking at it for the top n of an event, but for the really big events it starts to become less manageable.

Daed - What number of entrants would you estimate as the lower threshold that's currently ignored? I was initially considering avoiding events smaller than 33 players, but if that's an unexplored niche maybe I'll consider it after all.

**Daedalus81**

I believe you can read the faction without a sub, but it's limited to a certain time period.

I can't remember the numbers, but the bigger tournaments playing 5+ games sometimes quickly out-weight the smaller 3 game tournies with fewer people -- by games, but not necessarily by people. When I get my stuff back I can give you a better idea.

Another factor is being able to follow trends by individuals since they all get an ID.

**alextroy**

That could be an interesting statistic, Faction Win Rate excluding Top Players. It would require deep stats, but imagine the information that could be gained if all the games involving top players were removed from the stats? If you take all players winning 70%+ of their games over multiple tournaments, their impact on all factions win rate will warp the results based merely on their selection of faction(s).

**Dysartes**

That's an interesting idea, alextroy, though you might have to wait two-three months for it to kick in for a new edition - got to build the dataset to allow for such exclusions, after all.

**Dysartes**

With Dark Angels (and other significant SM sub-factions) appearing to regain Codex status in 10th, that's going to make things a bit simpler in terms of faction/sub-faction.

Some questions still arise as to how to classify forces with allies vs. those without, and I kinda hope BCP do a refresh of their faction options for 10th.

**Slipspace**

With sub-factions no longer being a thing I think the problem with BCP (which is essentially that players self-report their faction/sub-faction) should go away. Hopefully BCP will introduce something that tracks detachment type instead as that's likely the most useful data point.