For a couple of years now I have harped on (to the great boredom of many uninterested listeners) about elevating Zwift racing to the next level. I have written multiple articles here on Zwift Insider, provided numerous forum rants, and have even had meetings with Zwift directly. My general opinion has been that we should empower race organisers with the ability to customise how race pens are defined – and back that up with a ranking system so there is always something worthwhile to race for, even if you find yourself in a tough field and competing for the win is not realistic.
Zwift has made some great progress in the realm of ‘competition’ as they call it. Category Enforcement (CE) has practically removed the issue of sandbaggers (riders racing below their category) and cruisers (riders carefully managing their 20-minute power to stay in a lower category). However, as great as Category Enforcement is, it has two significant problems:
- Race experiences tend to be consistent for any one rider (always competing for the win, holding on but not challenging for the win, or at the back).
- The Category Enforcement system does not necessarily promote successful racers, which I think most of us would agree should be the case.
Zwift is also aware of this, and recent forum posts have shown they are directing their attention away from Category Enforcement and towards a ranking system based on the ZwiftPower/USAC rating model.
I have some concerns with the ZP/USAC ranking approach, and would rather we look to the gaming world to see how effective ranking systems are already utilised. There are many online games that use systems with extremely clever designs that consider the complexities of frequent many vs. many competitions. One challenge (as an example) is how to rank new, returning, or infrequent players.
The good news is, as we have seen recently with other projects like Sauce4Zwift, you can always rely on the community to come out with something incredibly impressive! This time it is the turn of long-time Zwifter Tim Hanson with his ZwiftRacing ranking system (available at zwiftracing.app).
ZwiftRacing – Overview
The ZwiftRacing system uses the Elo-MMR algorithm to rank riders based on their performances against other racers. Many readers may be familiar with Elo rankings from chess. Elo-MMR is an evolution of this logic for the online gaming world, adding much more intelligence to account for the various ways that users interact with gaming software and acting to limit ways the system can be deliberately ‘gamed’ in a way that lessens the experience for other users. Mathematically it is a Bayesian model, which in simple terms means it provides a ‘statistical belief’ of your ranking, which is updated as more certainty is derived based on additional evidence.
Highlights of the ZwiftRacing ranking system:
- Initial seeding of new or returning riders using power metrics so they do not disrupt lower-ranked races
- Confidence factor built into algorithm so new riders are given an accurate ranking as quickly as possible
- Ranking reset 3 times per year to refresh competition
- 8 ranking levels from bronze to grand champion to see how you stack up against the competition (with sub-levels also supported)
- Accurate ranking changes regardless of how pens are split – this effectively ranks racers in power-based or age-group races, as well as pens organised by ranking itself
Just a glance at the rankings reveals some fascinating information. The biggest takeaway for me is that there is such a strong correlation between riders with both a high W’ (anaerobic capacity) and sprint power and their ranking, whereas threshold power is not so impactful. This makes perfect sense when you think about the majority of Zwift races. You need a decent enough threshold to hold on to the pack, but the key moments in a race are decided by surges or short climbs (short duration repeated power) or a final sprint for finishing position.
This ranking system would also allow for more dynamic races even without completely configurable pen boundaries. As each major ranking has 3 sub-rankings (e.g., Gold 1, Gold 2, Gold 3) there are a total of 15 ranking levels. An organiser could, for example, set Pen A to be Grand Champion down to Platinum 2, and Pen B to be Platinum 3 down to Silver 1. The variety in how these pens are configured would allow for Zwifters to experience many different types of race dynamics – sometimes at the front of the race and sometimes fighting for rankings gains at the back.
The Zwift Community has contributed some huge improvements to the ecosystem as a whole – ZwiftPower and WTRL racing being the two biggest examples. I hope Zwift will also consider this ranking methodology as an option going forwards (along with that all-important race organiser toolkit!) I spoke to ZwiftRacing developer Tim to get some more insight.
ZwiftRacing.app – Interview with Creator Tim Hanson
Hi Tim, firstly I am super impressed with ZwiftRacing.app. Could you talk a bit about what led you to put so much work in? What problem were you trying to solve?
There were a couple of things that came together at a similar time that led to the creation of ZwiftRacing. I’ve been a Software Engineer with an interest in UI and UX development for a long time but my recent experience has been using Angular. I wanted to find a project idea where I could use it as an opportunity to learn React.
Separate from that, I started to think that there must be an alternative approach to ranking riders that can take inspiration from other online video games (like Rocket League for instance). In this type of system, a player’s rating changes by some amount after every event. The direction it changes (up or down) and the amount it changes are dependent on the riders’ finishing position and who the competition was.
Diving in a bit deeper it seems to solve for a lot of the concerns I had about a ranking system, such as accounting for new or returning riders in the autumn, or that it only gives an accurate ranking to riders that race very frequently. Can you speak a bit regarding the ranking system (ELO-MMR) and why you felt it would be a good fit for Zwift?
As you mentioned, this system is based on a concept created for the chess world called ELO and has been extended to fit into the massively-multiplayer gaming space. It does a really good job at evaluating a set of results to determine what changes are needed to each player’s rating.
With this approach, every place matters in every race which for me is the number one selling point. I love the added motivation when I’m deep in the hurt locker with a few KM to go. I’m typically in a place where a few extra places at the line might mean the difference in whether my rating goes up or down.
Additionally, once you have riders into ranks, you can look past the A/B/C/D categorization which is tied to a single data point in a rider’s power profile and to a results-based metric where you are assured to be racing against others who are similar in skill level to you. Like any other video game, the others around you might be stronger riders but perhaps you are better at race craft and can beat them anyway. This is an often overlooked aspect to racing on Zwift. It’s the strong riders who also are efficient in the draft who, more often than not, come out on top.
The ranking reset 3 times a year. What is the purpose of this, and how does it impact race experiences early in the season?
This is another concept that is borrowed from other online video games. Each rider (aside from Grand Champion riders) maintains their current rank (Platinum 1 for example) but their rating within that rank is reset. The other impact to a season change is that the first set of races completed in a given season are more impactful to your rating than subsequent ones.
The main reason why this is beneficial is that it allows newer riders (and riders returning from some time away from Zwift) a quicker path to a rating that is a truer reflection of their power and skill.
How could you imagine Zwift using these rankings going forward as a way to improve racing?
I am sure Zwift HQ have plenty of great ideas for improving the racing experience and community that they don’t have the capacity to move forward with at the pace they would like. But, if I had to imagine an (unlikely) world where they would be interested in utilizing this system, they could eliminate the A-E category system and replace it with a single signup where Zwift would put you in the pen of people closest to your rating among those who have signed up.
Are there any areas where you feel like the ranking algorithm doesn’t do a perfect job? What do you wish it could account for that it doesn’t?
The main issue at present is that points events aren’t supported correctly since each race series has their own point definitions. Those events (ZRL for example) are evaluated as normal scratch races currently. I would love to work with event/series organizers to define those points criteria in a central location so that rating changes could be formulated based on points received.
Do you have a vision to develop it further? What sort of things would you like to do?
Of course. I am a father of three young children so I have way more ideas than time. Here are some of them (some more developed than others):
- Achievements – Building on to the (very successful) badge system that Zwift has, I want to introduce a set of race-related achievements that riders can pursue. I have 71 defined so far. Things like:
- Finished a race in every World
- Won a race in Crit City
- Podiumed in a Mountainous race
- Exceeded 1000W over 15 sec
- Club Rankings – Competition is fun so why not provide a way for clubs to battle it out to see which is best? (DIRT of course)
- Looking for Club/Team – There currently isn’t a place for unaffiliated Zwifters to easily locate a club and/or team that works for them when it comes to grouping up to participate in a race series like WTRL’s ZRL or DIRT’s DRS. I’d love to fill that gap and provide a place for club captains to look for new riders that fit their time zone preferences.
Join the ZwiftRacing Ranks Trial Race!
We are hosting a trial rankings-based race on Wednesday 19th October at 19:00 UTC. You can sign up here. Please ensure you enter the correct pen based on your ZwiftRacing ranking.
Sign up at zwift.com/events/view/3179031
The race is being held on 2 laps of Richmond’s 2015 UCI Worlds route.
Your Thoughts
Please note that ZwiftRacing.app is in BETA. The plan is to assess the feedback and data from each season to make improvements for the next season. Check out ZwiftRacing.app and give your comments below!
Love it! I’m curious as to how well the zwiftracing ranking translates to the original pens as we know from other series that people struggle to enter the correct pens once the traditional A to D is not in order. Sure you can simply DQ everyone but with enough people getting it wrong the race becomes seriously impacted. And of course as you alluded to, applying scratch race rankings to a points race is asking for some ‘creative’ race performances.
But this is very much appreciated and i’m one of the people you certainly didn’t bore. We need a system like this and we need it 3 years ago.
Also i’m just 200 points away from Platinum and i was never platinum in any game ever so that’s exciting.
I think the biggest issue is simply that you have to self-select pen, full stop. Zwift engagement to force the pen selection would be the dream 😀
Zwift should support external pen splitting logic. E.g. Zwift asks ZwitfRacing how entrants should be split and ZwitfRacing answers with racers splitted by ranking.
Looks Like fun as a C Rider to start Cat A
Pen A… (wink with tongue stuck out)
i will be too exhaustet after ZRL to Race on wednesday
I think Bastini needs Pen E!
Basti not Bastini, sorry, don’t know how to edit!
yes Pen E is right but would also be Impossible to ride
Pen A… (wink with tongue stuck out emoji)
OK – let’s try that the other way around. Let’s imagine what an A rider racing in Cat C would look like.
An A rider racing Cat C for ZRL would likely win every single segment – both FAL and FTS on every one of 3 very different segments – and then win the overall race too. All in Division 1 of that category.
I think its fair to say that every other rider who was racing in Cat C might think this is a much more fun and fair way to allocate categories. Do you disagree?
PEN E is full of A+ riders. is there any reason to start there for a Cat C or B rider? 😹😹😹
Yes Basti, and you know why.
no, i think theres a failure in algorithm.
Riders with 3.1,3.2,3.3 wkg in same Pen with 5.5, 5.6 wkg.
I dont have an ideo how this will work Outside the warm -up Pen.
20m w/kg is largely irrelevant for zwift racing, as your continuous wins and podiums attest to. You know this, although it is also obvious that this system does not suit you as the Cat C King. You have the short duration power to easily compete at this level. Give it a go and see. There is no failure in the algorithm, if you keep winning you will face tougher opponents.
easily. yes of course. im going easlily in the Pack with A+ 🙂 i like your kind of humor:-)
It’s an interesting concept. But I don’t like it. From real life chess games through World of Warcraft PvP, League of Legends games. I feel like for every balanced ranked game I’ve ever played I’ve had another five where I’ve either been completed screwed by the match making or it’s been remarkably easy.
That’s in general, when you apply it to Zwift it becomes even more complicated. As you said all race series score points a different way. Which would need to be ironed out.
I don’t see the point of having different “race scenarios” based on different pen allocations. Surely the point of entering a race is to win it? Or work to put a teammate in the best position to win it. If I don’t win under the current system I can look at the riders who do win and see what I need to do in order to improve. But if it’s possible to move the goalposts of the individual pens for each race. Then it just adds complexity to me understanding why I fared differently in different races.
Speaking of complexity. You mention ZRL, probably currently the biggest team racing event on Zwift. With the current system you need 6 riders in the same letter category. Do you think that increasing the ranking system to 15 ranks instead of 5 is going to make it easier or harder for riders to find and manage teams?
Your first points are very interesting, and I’ll address them separately later as just heading out on a run. For ZRL, there is a plan too look at a team ranking system. Otherwise they could just use power like today, they could the CE logic, or they could split the rankings 4 ways. The ranking system works in the background, so it does not have to be the master for pen assignment.
Thank you for the reply and look forward to the discussion. Enjoy your run!
From my experience (same games and some others like Starcraft, Trackmania, CS:GO) there is a huge variance because players rarely play on the same level game after game. There are games where I have the brilliant idea to towerdive and force my botlane to overcommit and we all die and the game is over 4 minutes in and there are games where the enemy mid is tilting off the face of the earth because his jungler does not give him blue buff. But those behaviors are not consistent from game to game.
I believe there could be a lot less variance in Zwift.
I like this in many ways. I’ve long been advocating for results-based ranking, so you’ve already got my support when you say that. I’ve also thought the initial ranking should be based on power (CP is fine and better than 20 minute, but I’d take something more heavily weighted toward short term power with a smaller peak of weighting at threshold as even better). I do think I prefer the ZwiftPower method of results just expiring after 90 days rather than a reset to within your same level because the ZP method allows for a bigger drop after an injury (something you don’t really have to worry about with Rocket League – a game I used to play before Epic bought it and something I’ve thought about when considering Zwift rankings as well).
One other thing I really like about the proposal is the varied pen definitions. I finally worked my way up to B this past May, but I’ve been a very, very low B ever since then (so low, in fact that my Cat Enforcement category is still a C after the Cat Enforcement categories got redefined a month or two back). As a result, every race, I’m off the back before the halfway point or earlier (sometimes much ealier). This weekend in Harrogate, I was off the back for the last ~45 minutes of the 49 minute (for me) race of 2 laps of the UCI World’s Course.
In theory, I don’t mind being off the back and am working to try to last longer before going off the back every race (that’s my new standard of success), but that means I haven’t done any real sprinting in the last 5-6 months and my 15s power has dropped ~100W (~1 W/kg) or more as a result. Sure, I could do sprint-focused workouts, but I find workouts boring – racing is my workout. I’ve occasionally thought about doing a C race just to work on my sprints – I am a Cat Enforcement C after all – but the B that would show up next to my name in ZwiftPower that would mark me as a sandbagger (even though I’d be there legally according to the rules) has kept me from doing that.
Something that would vary so that I’m usually off the back but am occasionally present to compete for an intermediate sprint would be great.
A completely different system like the categories listed here (taken from Rocket League) to divorce the rankings from the ZP rankings is a great idea as that way we’re not thinking “Zwift Race Rankings says I’m a this, which means I’m really a __ in Zwiftpower” the way that at least I did when Cat Enforcement first started and I got promoted there much earlier than I did in Zwift Power and always felt I was racing over my head.
I don’t think it needs to be that complicated.
Even taking these two steps would improve the racing situation greatly in Zwift.
Good point about normalized power.
Appreciate the write up and detail James! We’re asking ourselves a lot of these same questions at the moment and it’s clear to me we and the community are more closely aligned on this than ever.
That’s super to hear Flint! Really exciting times ahead. 👍🏼
It’s showing as 8pm UK time in companion app, is that right?
No, 1930 BST, at least according to https://zwiftpower.com/events.php?zid=3179031
Oh, web sign up page conflicts with Zwiftpower race page and does say 2000 BST!
I’m sure it’ll all be sorted 🫣
That’s because ZwiftPower caches its event data…
Should be 8pm UK time. It was updated as to not clash with a major race series.
Looks like a very interesting way to categorize racers, and interesting to see how different riders might fall because of their strengths.
Just curious – what is the minimum to obtain a ranking? I see that my name doesn’t bring up anything in the search despite having competed in several ZRL races this round
It should be there. There seems to be a few riders missing. What is your Zwift ID?
1301472 — I see myself and a few others in results on the site that have no ranking. Thanks for taking a look!
Once you have done another race you will appear. There was an issue with some early September races.
Great stuff and impressive tech – really smooth site.
Posting the below just as a thought starter, not really a serious concern.
I have found the current algorithm as set up still tends to favor people at the top of their category over the people at the bottom of theirs
Take this race for example: https://www.zwiftracing.app/results/3170918
I finished 14/18th, not a big deal – however, interestingly, I beat everyone rated lower than I did and everyone who finished ahead of me was ranked higher. In fact, that’s generally the case throughout the result – which speaks to how predictive the algorithm is within a category! Awesome! However, despite finishing in largely expected order, the people at the bottom all took hits to their ranking, and the people at the top all gained ranks.
The risk here is the boundaries between categories and is the algorithm really predictive across cats. Will be interesting once the categories reset, but I worry you will end up with people constantly bouncing between categories, unless, as you have suggested this is paired with dynamic or adjustable categories so the boundaries are different every race. Might be ideal to set the boundaries actually right at the start time to equally allocate signups, rather than allowing racers to pick and choose a race where they are right on the bubble.
Lastly, after paying attention to my rank for a week or two, I have noticed a new voice in my brain 30min into the race after I’ve been dropped: “should I just DNF and to avoid the inevitable ranking hit?” (per the above). This wasnt really a concern with the Zwift Power ranking system as it only takes your top performances.
If this is implemented where every race impacts your score, DNFs need to be counted in the results.
Some great comments and a few points we have discussed in detail, join the discord (linked at bottom of website).
Sort of to go along with your “bouncing” comment, as a very low B, I notice several people (very high Cs) who I beat well over half the time when we’ve raced head to head (not very common) but who are ranked ahead of me due to beating people they’re supposed to beat in the Cs while I lose to the people I’m supposed to lose to in the Bs. I suppose, if we get put in the same categories together, that will either work itself out or it will show that they were off their game (injured, raced too many days in a row, not putting as much emphasis on the race as I did?) on the days I beat them.
Big fan of the idea, but without category enforcement it will be hard to implement I fear. Simply because a big part of the community isn’t aware of all of this.
More categories might be needed though. As a “Challenger 2” racing against “Grand Champions” is way out of my league.
I signed up for the test on Wednesday but doubt I’ll make it after the suffer fest in ZRL tomorrow :).
Challenger 2 is great! The split I have used for Wednesday’s race is just a line in the sand. Pens could be carved up any way in the future.
My first thought this appears skewed to those I know race a lot. That might be fine in gaming, but Zwift isn’t just a straight game, we’re athletes. I think the ZP rankings have the better riders top ranked. It might help the arguments in C and D, I don’t know, but it seems a lot of work for something that seems sub optimal to the original.
Interested as to why you think it’s suboptimal. Zwift Power rankings require 5 races to give you an initial rank, as opposed to 1 with this system. Zwift Power ONLY works for frequent top racers. If you are genuinely ranked lower than 550 on ZP, you would be racing high quality riders in every single race.
Everything depends on one’s POV, I am in situation you describe, so to me I see the floors there. However, as I said, it may well work for lower categories. But not quite (not better anyway) for the best in the world, and the unfortunate thing about ranking is the best are the ones that are easy to see.
Surely this is a system for the community made up of 10,000’s of racers who want a good/fair experience and not the top 1% in world. I’m sure they know how great they are and don’t need a rank to confirm it.
This is fantastic work, thank you @Tim Grose Hanson. I’m really excited to see that it reflects the results analysis I published a few months ago (ie. 20 minute power is not a good predictor of category based race outcome), and that it goes beyond that into a demonstrable ranking system! Let’s keep pushing these boundaries forwards. Like you – I have more idea than time, and you’ve got way further than me! Love it.
YES. A thousand time YES.
Getting off the 20-minute standard is a great first step (says the man whose max wattage is less than 2x FTP).
I’ve always felt that any ranking system should take into account not only race wins, quality of the field in the races, and an individual’s power profile but also race length and course difficulty. Right now with the current ZP ranking system you can simply climb up the ranks by doing a bunch of super short sprint races. Winning a 4km sprint race after 8 minutes of racing should not be weighted the same as winning a KISS 100 race after 2+ hours of racing. Likewise, winning a race on Surrey Hills is significantly different and most would agree much more difficult than winning a race on Tempus Fugit.
I’ll agree that it’s significantly different (and that it burns more calories), but “more difficult” can also mean likelihoood of doing it and that comes down to number and composition of the participants and your particular skillset. A heavy diesel engine racer with no sprint might find a long TT on Tempus Fugit easier to win than would a light. climber-type or a fast-twitch sprinter-type. Those short races that you say are easier to win often have fields 2-3 times (at least) the size of the 100 km races, decreasing the odds of winning for those who make it to the finish by a fair bit.
That’s a great development.
One criticism would be resetting of rankings. I don’t see the point if doing that. The longer they stand, the more accurate the system if it is mathematically based on predictions from numbers.
A suggestion..perhaps there could be a minimum number of races per month you have to enter in order to maintain yiur ranking.
It shouldn’t be positively influenced just because you do many races as too tgere shoukd be a penalty to ranking if you enter very few races just to sit on a ranking.
It would be nice if the older the results were, the less they counted over extended time.
If soneone did not race for a long time, they could just be given a preliminary ranking that was at the low end of their level and work up again (or down) from that.
I tried so hard in D pen as a good C. I couldn’t even stay on the group on the flat towards the sprint.
I have a 3707 rating but I was absolutely no where near.
I love the idea but that was awful.
My rating is high as I do TTT and ZRL and Tiny series. I am a decent sprinter so I am not sure how long it would take for me even to have a decent race
TTTs aren’t counted in the rankings. I’ll look at your data soon, but it was not hard on the flat and you should be able to stay on… it is possible that your rating is high if there isn’t much variety (always doing sprint races for example)
Yeah I do a lot of sprint races. I think the ZRL tired me also last night. But the pace was too much for me, I was hanging on for sure!
My job is to sprint and I love it, I find it very unfair that riders like me get labelled. I can share my stream to show how hard I ride with my heart rate, video etc.
I will give it another go, but I think doing well in the races I do will artificially raise my rating
Thanks for reply mate, well appreciated
I’ve played many competitive online games. Can’t wait to see an MMR system into zwift and a more centralized way for event points that could relate to your zwift category/ranking.
OUTSTANDING.
It seems like a good idea. But how does it work out that #3 rider overall enters a not necessarily supercompetitive B-race (although with a lot of riders), ends third and gains 58 (!) points. If you are the best in the world then ending third in a B race is not an outstanding performance I would guess?
Currently it considers a podium a strong performance regardless, as it is based on riders being grouped by rank, however Tim is looking at solving that particular issues as it doesn’t work well for blatant sandbaggers.
Have to admit, I am not a fan.
This system encourages people to drop out of races when doing badly as it then does not count towards your score giving you negative points.
I guess you are a highly ranked C like me Dave
What I dont understand is until the game changes the way the pack dynamic works (i think this is being trialled) if the races comes to a sprint and people like us have rode well, we will contest the sprints. Thats how it works in real life and on Zwift.
Interesting! I’m not entirely sure how I feel about the system as such, but appreciate all the effort and innovation on the matter!
Did a hobby comparison today (October 27th) between ZwiftRacing [BETA] and ZwiftPower, based on the top 20 ranked on ZR. It shows relatively clearly what was also mentioned in the article, about the importance of high W’ and sprint power. It undeniably seems that some B riders from one and the same club have managed to crack the code for high ranking on ZR. 🙂
Need more race times for the ranking events maybe run them between ZRL ? At early evening slots and early morning slots for all zones ? Hopefully zwift will play ball with this ????
Not zwift playing ball, but a new race organiser toolkit is on its way soon, so hopefully we will see far more races using the platform.
Lots of great brainstorming in this article. I would suggest keep the A-E rating component, as an easy to gauge entry-point especially for new Zwifters. It does not have to take away at all from the “intelligent” dynamic ratings discussed, but if you remove the A-E rating, then those that need a structure to start in will likely find themselves lost on how to start and how to categorize themselves. I do think the dynamic rating as described is an excellent idea as it addresses multiple existing problems, and as it appears to be viable, I think Zwift should move forward with it, yet keep the A-E as well. I’m also looking forward to more with the zwiftracing.app.
Is there any value to analyzing the outdoor rides of racers as well? I know that’s a double edge sword with lack of, or different power meters, but it seems it could be applicable.