Just Released: A Fresh Batch of Zwift Speed Test Data
Approximately two months ago we jumped into a new project with the goal of providing more accurate data on the performance of bike frames and wheels in Zwift.
Our previous dataset was quite accurate, but had a few issues:
- Margin of error: our data is based on Strava segment times, and those don’t record fractional seconds. Often we would run two identical tests, but segment times would be 1 second apart. Which one do you pick?
- Outdated: Zwift has tweaked the way avatars move side to side (pack dynamics), and this affects our speed tests even though the tests are done in isolation with a solo rider. This made it impossible to compare old and new test data precisely. (The difference is most noticeable in our new climb test data, where 101 of the 124 tests turned in slightly faster times – probably due to changes in how hairpin turns are made.)
- Stealth Updates: occasionally Zwift will tweak a frame or wheelset without telling us, and the only way we find out is if we re-test that item. We figured a complete re-test would turn up a few of these instances (and we were right).
Updated Test Methodology
To address the margin of error issue we modified our testing protocol so each flat and climb test was run twice. That way, if they turned in times that were 1s apart, we could average those times for improved accuracy. So you’ll see a lot of times in our test data which include a half-second, which we never used before.
We still used a 183cm, 75kg rider at 300W steady for all of our tests. For our flat test, our test bot completed two laps of Tempus Fugit, which takes around 50 minutes. (And we did that two times, so two separate rides of two laps). For our climb test the bot rode up Alpe du Zwift, which takes around 50 minutes. (And did that twice, so two separate ascents of Alpe du Zwift.)
Calming the Combinatorial Madness
There are currently 124 items for us to test:
- 35 Wheelsets
- 62 Road Frames
- Zwift Concept Z1 “Tron bike”
- 16 TT Frames
- 5 MTB Frames
- 5 Gravel Frames
In the past, when a new wheelset was released, we would test it on multiple frames because our early test data indicated that a given wheelset might make (for example) one frame 2s faster, but another frame 4s faster. This never made sense to us, but that’s what the data showed, so we had to test various combinations in order to be accurate.
Thankfully, the performance of items on Zwift no longer works this way. So we can now test a new wheelset, compare it to the “baseline” wheelset (we use the Zwift 32mm Carbon), and know that whatever advantage that wheelset has over the baseline will hold across all frames.
It works the same way with frames: if a new frame is released, we can compare its performance to our baseline frame (the Zwift Aero) and know that we can put that frame on any wheelset and it will deliver the same boost in performance across the full range of Zwift wheelsets.
So we tested every frame in game using the same wheelset (32mm Carbon). Then we tested all the wheelsets in game using the same frame (Zwift Aero). As you can imagine, this was a much smaller undertaking than attempting to test various combinations!
But it was no small undertaking.
If you do the math, that’s 124 * 2 Flat Tests * 2 Climb Tests = 496 total tests, each lasting approximately 60 minutes. Almost 21 solid days of testing!
Flat Test Results
103 tests were between 1s slower to 1s faster than the old data, falling within an acceptable margin of error due to Strava rounding to the nearest second. 9 tests had no old times to compare to (new additions to Drop Shop) or were limited-time items (Zwift Big Wheel) that aren’t available for new tests.
That leaves 12 items with more substantial changes:
- Zwift 50mm carbon wheels were 3.5s faster
- Zwift Buffalo Fahrrad is 15s slower
- Lauf True Grit is 4s slower
- Focus Izalco Max 2020 and Pinarello Dogma F were 1.5s slower
- Roval Alpinist CLX wheels were 2s slower
- 6 frames were 1.5s faster than before:
- Cannondale Synapse
- Giant Propel Advanced SL Disc
- Giant TCR Advanced SL
- Ridley Noah Fast Disc
- Canyon Lux
- Specialized Epic S-Works
Climb Test Results
46 tests were between 1s slower to 1s faster than the old data, falling within an acceptable margin of error due to Strava rounding to the nearest second. 9 tests had no old times to compare to (new additions to Drop Shop) or were limited-time items (Zwift Big Wheel) that aren’t available for new tests.
That leaves 69 items with more substantial changes. 50 of those items are now faster by 1.5s or more. Here are the notable items which received speed improvements:
- Felt AR frame is 6s faster, putting on the same level as the Specialized Venge S-Works which has the same climb time, but is just 0.5s faster on the flat test.
- Cannondale EVO frame is 3s faster, moving it just barely into 2nd place behind the Specialized Aethos in the fastest climbers list.
- Lightweight Meilenstein wheels are 3s faster, putting them on par with the Roval Alpinist wheels which turn in the same climb time and are 0.5s slower on the flat test.
- Trek Madone is 3s faster, moving it up the all-arounder rankings just a bit.
Three items received substantial slowdowns:
- Zwift Safety and Buffalo Fahrrad bikes are both 17.5s slower. These were already bad bikes for climbing. Now they’re worse!
- Lauf True Grit is 24s slower. We aren’t sure why this bike’s performance was nerfed from its original, because now it is much slower than all other gravel bikes in game. Our guess is Zwift will adjust it back.
What ZI Posts Are Updated… and Which Aren’t
We’ve updated all the charts sitewide with the new data, including TT frame, road frame, and wheel charts, plus the popular Tron vs Top Performers chart.
Our various “Top” lists, such as the “fastest bikes and wheels at each Zwift level” post, have not been updated, because those require manual updates. We don’t anticipate any major changes to these lists, but items may get shuffled around a bit and we’ll summarize those changes in the Changelogs at the bottom of the posts.
Do Brands Pay for Zwift Performance?
Over the years, Zwift CEO Eric Min has said in a few different interviews that companies don’t pay in order to show up in Zwift. But with so many new frames and wheels arriving in game and companies appearing to jostle for position at the top of the Zwift performance heap, the community has been wondering: are companies paying for their products to perform well in Zwift?
I reached out to Zwift HQ with that simple question. Here’s what they replied:
Zwift has not commercialized partner products in the game as these products assist the in-game experience. What we ask in return is that brands bring additional value to our community – whether that be through partner-led events, community engagement, pro rides or other elements.
Additionally, there is no pay-for-performance element to these partnerships. Performance and ratings are determined by taking available information about bike weight and aerodynamics to place the bikes and wheels in a reasonable relative position compared to other items in-game.
There you have it. Companies aren’t paying Zwift to have their frames and wheels perform well in-game.
Can I See the Raw Data?
Sure you can. Here’s the Google spreadsheet.
Questions or Comments?
Share below!
