Variance Hammer: Are Later Round Tournament Games Closer?

Variance Hammer LOGO COL

So after reaching out to the good folks over at Frontline Gaming, I have indeed gotten some more detailed data from the Las Vegas Open. One of the analyses I have planned is a bit of a beast, so rather than wait for that to be finished, I thought I’d put up a little side thought I had: How are the scores in tournament games distributed? Do they get closer in later rounds?

Read on to find out.

This post was a little bit inspired by another blog musing about how to prevent really ugly first-round match ups in fun/narrative tournaments, because nobody wants to start their day of a friendly even by getting their teeth kicked in. I may have some more detailed musings on that later, but it prompted another, more fundamental question: What does the distribution of round-by-round scores actually look like? Is a tournament dominated by nail-biters, or blowouts?

One might think, because of the way Swiss pairing works, that the scores narrow over time. The first round should be all over the place as lower-tier players (yours truly) get matched up with people who have the chance to go all the way, etc. But later rounds are matching up people with similar scores – presumably those games should be somewhat tighter. This potentially has a big influence on the way you plan for a tournament: if the scores narrow over time, the relative contribution of early rounds to a player’s score is greater, making early morning games more important to carry. In this way it would work a little bit like the adaptive scoring system for the GRE test one takes to get into graduate school in the U.S. – the first few questions are massively influential on your score, while the last are at best going to change things by a point or two. This would also have implications for those tournament players who enjoy an adult beverage the night before, and might not be at their best for the first few games.

But, as it turns out, this is not the case.

To look at this, I took the difference in scores between paired opponents, broken up by round, for the first six rounds of the LVO. The later rounds were small, and largely determining the winner from a much smaller pool, so I threw them out. Here’s what those differences look like:

RoundDiffs

Lets break this figure down a little bit. The odd shape for each round is the distribution of the differences in scores in that round – fatter areas indicate more games fell into that category. So, for example, that big bulge around 10 in round one? Indicates that a large number of the round one games had a total difference in scores between the players of 10. And overall, that’s a pretty consistent trend – every round has a hump at the “blowout” end of the spectrum, while really only Round 3 has a substantial number of tight games, where the difference in scores is quite close. Also interesting is that the “average” score (a ~7 point difference) actually happens quite infrequently: the average tournament game never really happens.

What’s this mean? First, that those who are dragging a bit in the early morning need not worry so much – there’s not a significant trend for narrowing scores as the day goes on. It may also mean that the multiple mission objectives from the ITC aren’t working as intended. For those who are curious:

  • Players score each mission in the ITC Scenarios separately, which then generate Mission Points to determine who wins the scenario.

  • The Dawn of War and Maelstrom missions are worth 4 Mission Points if won, 2 mission points to each player if the mission is tied, and 0 mission points if the mission is lost.

  • Each Bonus Point is worth 1 Mission Point. This generates a score of between 0-11 Mission Points.

The preponderance of 10 and 11 point differentials in the results indicates that, generally, the army that carries Dawn of War is also carrying the Maelstrom missions and picking up bonus points. If the intention of the different objective types are to have different armies claiming different aspects of the mission, followed by a showdown to see if being stronger in one area or another is enough, that seems to only happen in a minority of games.

Alternately, it could simply mean that tournament-level Warhammer 40K isn’t actually a game that produces close results, but is rather dominated by a “go big or go home” paradigm where there’s always a very clear winner. There are likely other explanations that I’m not seeing at the moment, and I’d be interested to hear them. Determining which is the case is, in the immortal words of any number of academic papers, left as an exercise to the reader.

Variance Hammer’s ad-free content is provided through the generosity of our Patreon sponsors. If you’ve enjoyed what you’ve read, and want to support the creation of new, quantitatively-driven Warhammer 40K content, please consider joining them.

Tags:

About Reecius

The fearless leader of the intrepid group of gamers gone retailers at Frontline Gaming!

31 Responses to “Variance Hammer: Are Later Round Tournament Games Closer?”

  1. Adam
    Adam (TheDiceAbide.com) April 13, 2016 10:06 am #

    What’s interesting is that since each round is tied to a specific scenario, it would appear that the third scenario makes for the closest games. Perhaps the overall scenario design, or at least all but the third, might benefit from a redesign to make more interesting game results.

  2. Fagerlund April 13, 2016 10:36 am #

    I don’t think the closeness in a game of 40k can be determined by the points alone actually. How often haven’t you had that game where the game is super tense from the start and then this one thing happens which makes the points spiral out of control?

    • artfcllyflvrd April 13, 2016 10:42 am #

      So true. You can barely lose all 3 ITC objectives and get goosed. The final score isn’t really representative of the closeness of the game.

      One of the things the Adepticon mission scoring tried to do was make it so losing 3 objectives barely still put up a considerable number of points. It didn’t quite work out that way (for lots of reasons) but I think it’s a goal worth striving for.

    • Reecius
      Reecius April 13, 2016 12:01 pm #

      Precisely, games can be close and the scoreboard can show it quite differently in a game of 40k.

  3. artfcllyflvrd April 13, 2016 10:39 am #

    “it could simply mean that tournament-level Warhammer 40K isn’t actually a game that produces close results, but is rather dominated by a “go big or go home” paradigm where there’s always a very clear winner.”

    This is truer than ever and has been my experience playing ITC, ATC, and Adepticon missions.

    • Reecius
      Reecius April 13, 2016 12:11 pm #

      Again though, the score may not represent how close a game actually is in terms of player experience. For example, I was getting creamed in an Adepticon mission by my Tau opponent, but still won 30-0. The game was close in play experience, but the score was lopsided. Another example, my round 1 game at Adepticon was very close but I was racking up points all game, my opponent wasn’t. I won by a large margin on the scoreboard, but the game felt fun and close the entire time while we were playing. Conversely, we’ve played ITC missions where it was super close but then the last minute there’s a big swing and my opponent steals both Eternal War and Maelstrom from and puts up a big score that doesn’t show how close the game actually was while playing it.

      My point I suppose, is that focusing on the score may be missing the forest for the trees. The numeric value representing the outcome of the game doesn’t always accurately express how the game felt to play (which to me, is a far more important metric, although clearly harder to track).

      • asmo April 13, 2016 7:15 pm #

        Sounds a lot like the delegate system. Really it is a delegate scoring system when you think about it. Each objective is a winner takes all type of thing. It doesn’t mean it is wrong it just means the system is a take all points type of thing. A rework in scoring or mission design would change that situation. We all know that in the last 2-3 presidential elections the delegate count suggested a blowout but the popular vote was close.

        An example for using on crusade with 5 objectives. Instead of having players be awarded 4 points and 0 points for a loss just give each player 3 points for each objective which goes into their overall score regardless of whether they had less objectives than their opponent. Anyway just an example of something that hasn’t been done for a while that would help make scoring more analog vs the digital system we have now. Of course kill points would need some thought but that as well isn’t an unbreakable obstacle. Some thought and careful mission design could make it possible for all missions to be worth the same amount and achieve a closer analog scoring system.

        • Julnlecs April 14, 2016 1:18 am #

          You should look into Nova missions. It’s pretty much what you are talking about.

        • Lord Krungharr April 14, 2016 5:16 am #

          Definitely my personal preference for 40k scoring. Then a game win can add bonus points to a players score for overall reasons , so someone who loses every game but scores high can’t win the event.

        • abusepuppy April 14, 2016 6:09 am #

          Part of the issue with that is balancing it across different missions. How do you ensure that The Relic and Purge the Alien end up contributing the same amount to your score when you get 16VP on one of them and 3VP on the other? (Missions like Relic and Emperor’s Will that are almost inevitably either ties or complete blowouts are also an issue.)

        • Reecius
          Reecius April 14, 2016 9:32 am #

          Good ideas asmo, but the reason we do it as 0-2-4 for loss/tie/win on objectives is to have a uniform amount of points from round to round. In order to weight each missions equally for pairings and scoring in overal scores, you have to have the same range of points available each round. Otherwise, you’d have an uneven distribution of points throughout the tournament.

  4. Codi April 13, 2016 10:40 am #

    Mission 3 was purge right? I can see how that would create closer games.

  5. Venkarel April 13, 2016 11:14 am #

    “The Dawn of War and Maelstrom missions are worth 4 Mission Points if won, 2 mission points to each player if the mission is tied, and 0 mission points if the mission is lost.” was not true at LVO. A tie was worth zero.

    • Reecius
      Reecius April 13, 2016 11:59 am #

      That was the 2015 format, we changed it for 2016 which is where the confusion is coming from.

      • Venkarel April 13, 2016 12:25 pm #

        Huh all of the missions in my 2016 LVO 40k Championship Player’s Packet have 0 pnts if lost or tied. If this is incorrect, then I definitely scored at least two rounds incorrectly.

        • Reecius
          Reecius April 13, 2016 12:30 pm #

          Yeah, the LVO 2016 was no points for a tie, the 2016 season through, starts the day after the LVO 40k champs, so currently you get half points for a tie. LVO 2016 used the 2015 season rules. Our calendar begins and ends at the LVO, so we’re now in the 2016 season, starting right after LVO 2016 Champs warps up.

  6. bogalubov April 13, 2016 11:46 am #

    The “closeness” of the third game perhaps indicates that the Euro’s are right with their approach to including kill points in every mission as part of the ETC.

    I know that sometimes a close, fun game won’t end up having a close score. But I think that shows that perhaps we are not scoring the right things. You’d never walk away from a basketball game saying “Oh man what a close game! They lost by 100, but man was it close!”.

    A close basketball game can end up with not a particular close score as one team just piles up free throws at the end, but it won’t be the equivalent of 10-0.

    As is the results show that whoever wins the maelstrom mission is likely to win the dawn of war mission. Which begs the question, why bother keeping track of maelstrom? You could play straight dawn of war and you’ll get the exact same distribution of results.

    • Reecius
      Reecius April 13, 2016 12:01 pm #

      I think you’re reading too much into the results of this analysis. In our games, very frequently, one player wins one and the other wins the other, the game coming down to bonus points. We’d have to dig deeper to determine the why of the lopsided scoring in this study.

      • asmo April 13, 2016 7:25 pm #

        The data does not suggest what you just said. It suggests that players were generally winning both in all missions except #3. With the scoring system at LVO this year a 8-10 differential is impossible with one player winning primary and the other winning secondary. The highest differential there can be with one player winning either primary or secondary is 3 (winning player max can get 7 points and losing player min can get 4 points). In fact every single data point on the chart that is above the red dotted line by definition means that the winning player won both. It is not uncommon though for people to observe something in practice that is not backed up by real data as the lopsided events are not often talked about or memorable where the games where it came down to bonus points are some of the most memorable and talked about. I don’t often see people talk about and celebrate their games where they didn’t score closely. The losing player generally wants to forget the experience and the winning player doesn’t want to “rub it in” and make the losing player feel worse than needed. The winner tries to be a good sport. If the winner talks about it normally it is in a quick hush hush manner or later when their opponent is not present.

        • Reecius
          Reecius April 14, 2016 9:34 am #

          Oh, I agree that anecdotal evidence doesn’t represent a trend. However, in the scoring system used this year, there were no points for ties, either, which skews the data a bit. I will be interested to see the point distribution now that we’re scoring points for ties as well.

  7. Reecius
    Reecius April 13, 2016 12:05 pm #

    Great article as always from Variance Hammer. A few things:

    At the LVO this year, the scoring was not points for a tie, just FYI. That is a component of the new season’s scoring structure.

    I think their may be a false premise at play here. When we wrote the missions we wanted to create multiple paths to victory, and wanted to stay as close to the book as possible (thus the Maelstrom and Eternal War missions). Of course we want the games to be close and fun, but, balanced missions that avoid lopsided outcomes based on the mission itself is more important form our perspective then necessarily have a close score. As noted, the score doesn’t always represent how close a game is in actuality. The score is used to stratify the field and make next round pairings possible. The play experience should feel close, but the numeric representation of it at the end doesn’t have to be (although that can help break up the field for ranking purposes).

    • Variance Hammer April 13, 2016 10:59 pm #

      Reecius,

      Some thoughts, on your thoughts:

      1. “Balanced missions that avoid lopsided outcomes based on the mission itself is more important form our perspective then necessarily have a close score.” One might reasonably think that “Balanced missions that avoid lopsided outcomes” would result in closer scores.

      2. “As noted, the score doesn’t always represent how close a game is in actuality.” True. Sadly, I don’t think you guys will pay me to hover around LVO going to each table asking people how close the game is, then correlating those with the actual score. Or more accurately, paying someone else to do it so I can play too 😉

      None of the numbers in here are definitively saying “X thing is True”. What they are saying is that closer games, especially closer games as time went on, meant tense games were happening. That the data doesn’t reflect that doesn’t mean tense games *aren’t* happening, just that they *might* not be.

      • Reecius
        Reecius April 14, 2016 9:37 am #

        Fair points. I am not trying to blindly defend our missions either, constructive criticism like this is great for getting better, so I hope I am not coming across as defensive.

        I will be interested to see how scoring ties alters this data for the current season.

        And we’re open to a total reword of missions, too, trying something new to really go for a balanced mission set. Traditionally though, we’ve found positive response to missions players understand instinctually by sticking to the book as closely as we can. That may be holding us back a bit as GW missions are written with fun in mind, not necessarily tournament balance.

    • Variance Hammer April 13, 2016 11:03 pm #

      Also, I super-dig the preview picture on the front page 🙂

      • Reecius
        Reecius April 14, 2016 9:34 am #

        Glad you liked it! It felt appropriate, haha.

  8. Vercingatorix April 13, 2016 12:56 pm #

    This may be showing that missions with a 0-sum tend to be more lopsided. Any time one player gaining points means the other player loses points tends to make larger score differentials.

    • Reecius
      Reecius April 14, 2016 9:37 am #

      True, which is one of the reasons why we now score points for ties.

      • Vercingatorix April 14, 2016 12:08 pm #

        I don’t think that really makes a difference for this scenario though?
        The three results are still 8 point difference, tie, 8 point difference for each primary right?

        I wonder why the kill points one seems so much more likely for one person to win maelstrom and one person to win kill points. Maybe the eternal war end missions and maelstrom actually line up closer than originally thought? As in the player which is in control at the end of the game can usually catch up on maelstrom. So maybe Maelstrom should actually heavily penalize a player for lack of early participation. Maybe making early game maelstrom worth a lot more?

        • westrider April 15, 2016 6:33 pm #

          All the Maelstroms for ITC Purge are Board Control in one form or another. There are a significant number of Firepower-based Armies (Tau and IG come to mind) that can do well at KP, but have to sort of play Keep-Away to do it, which is basically ceding the Maelstrom in that Scenario.

      • Vercingatorix April 14, 2016 12:13 pm #

        Then again, I’m not convinced having one player win one and the other player win the other is a good way to have it either. It could set it up to be the “whoever gets first blood wins” kind of scenarios that were close in points but all but decided after the first shooting phase.

  9. Venkarel April 14, 2016 7:21 am #

    Can you run the same graph but limit it to the top 64 players? It would be nice to see if the trend continues with the “good” players in later rounds when they have to play each other.

Leave a Reply