Just Released: A Fresh Batch of Zwift Speed Test Data

Just Released: A Fresh Batch of Zwift Speed Test Data

Approximately two months ago we jumped into a new project with the goal of providing more accurate data on the performance of bike frames and wheels in Zwift.

Our previous dataset was quite accurate, but had a few issues:

  • Margin of error: our data is based on Strava segment times, and those don’t record fractional seconds. Often we would run two identical tests, but segment times would be 1 second apart. Which one do you pick?
  • Outdated: Zwift has tweaked the way avatars move side to side (pack dynamics), and this affects our speed tests even though the tests are done in isolation with a solo rider. This made it impossible to compare old and new test data precisely. (The difference is most noticeable in our new climb test data, where 101 of the 124 tests turned in slightly faster times – probably due to changes in how hairpin turns are made.)
  • Stealth Updates: occasionally Zwift will tweak a frame or wheelset without telling us, and the only way we find out is if we re-test that item. We figured a complete re-test would turn up a few of these instances (and we were right).

Updated Test Methodology

To address the margin of error issue we modified our testing protocol so each flat and climb test was run twice. That way, if they turned in times that were 1s apart, we could average those times for improved accuracy. So you’ll see a lot of times in our test data which include a half-second, which we never used before.

We still used a 183cm, 75kg rider at 300W steady for all of our tests. For our flat test, our test bot completed two laps of Tempus Fugit, which takes around 50 minutes. (And we did that two times, so two separate rides of two laps). For our climb test the bot rode up Alpe du Zwift, which takes around 50 minutes. (And did that twice, so two separate ascents of Alpe du Zwift.)

Calming the Combinatorial Madness

There are currently 124 items for us to test:

  • 35 Wheelsets
  • 62 Road Frames
  • Zwift Concept Z1 “Tron bike”
  • 16 TT Frames
  • 5 MTB Frames
  • 5 Gravel Frames

In the past, when a new wheelset was released, we would test it on multiple frames because our early test data indicated that a given wheelset might make (for example) one frame 2s faster, but another frame 4s faster. This never made sense to us, but that’s what the data showed, so we had to test various combinations in order to be accurate.

Thankfully, the performance of items on Zwift no longer works this way. So we can now test a new wheelset, compare it to the “baseline” wheelset (we use the Zwift 32mm Carbon), and know that whatever advantage that wheelset has over the baseline will hold across all frames.

It works the same way with frames: if a new frame is released, we can compare its performance to our baseline frame (the Zwift Aero) and know that we can put that frame on any wheelset and it will deliver the same boost in performance across the full range of Zwift wheelsets.

So we tested every frame in game using the same wheelset (32mm Carbon). Then we tested all the wheelsets in game using the same frame (Zwift Aero). As you can imagine, this was a much smaller undertaking than attempting to test various combinations!

But it was no small undertaking.

If you do the math, that’s 124 * 2 Flat Tests * 2 Climb Tests = 496 total tests, each lasting approximately 60 minutes. Almost 21 solid days of testing!

Flat Test Results

103 tests were between 1s slower to 1s faster than the old data, falling within an acceptable margin of error due to Strava rounding to the nearest second. 9 tests had no old times to compare to (new additions to Drop Shop) or were limited-time items (Zwift Big Wheel) that aren’t available for new tests.

That leaves 12 items with more substantial changes:

  • Zwift 50mm carbon wheels were 3.5s faster
  • Zwift Buffalo Fahrrad is 15s slower
  • Lauf True Grit is 4s slower
  • Focus Izalco Max 2020 and Pinarello Dogma F were 1.5s slower
  • Roval Alpinist CLX wheels were 2s slower
  • 6 frames were 1.5s faster than before:
    • Cannondale Synapse
    • Giant Propel Advanced SL Disc
    • Giant TCR Advanced SL
    • Ridley Noah Fast Disc
    • Canyon Lux
    • Specialized Epic S-Works

Climb Test Results

46 tests were between 1s slower to 1s faster than the old data, falling within an acceptable margin of error due to Strava rounding to the nearest second. 9 tests had no old times to compare to (new additions to Drop Shop) or were limited-time items (Zwift Big Wheel) that aren’t available for new tests.

That leaves 69 items with more substantial changes. 50 of those items are now faster by 1.5s or more. Here are the notable items which received speed improvements:

  • Felt AR frame is 6s faster, putting on the same level as the Specialized Venge S-Works which has the same climb time, but is just 0.5s faster on the flat test.
  • Cannondale EVO frame is 3s faster, moving it just barely into 2nd place behind the Specialized Aethos in the fastest climbers list.
  • Lightweight Meilenstein wheels are 3s faster, putting them on par with the Roval Alpinist wheels which turn in the same climb time and are 0.5s slower on the flat test.
  • Trek Madone is 3s faster, moving it up the all-arounder rankings just a bit.

Three items received substantial slowdowns:

  • Zwift Safety and Buffalo Fahrrad bikes are both 17.5s slower. These were already bad bikes for climbing. Now they’re worse!
  • Lauf True Grit is 24s slower. We aren’t sure why this bike’s performance was nerfed from its original, because now it is much slower than all other gravel bikes in game. Our guess is Zwift will adjust it back.

What ZI Posts Are Updated… and Which Aren’t

We’ve updated all the charts sitewide with the new data, including TT frame, road frame, and wheel charts, plus the popular Tron vs Top Performers chart.

Our various “Top” lists, such as the “fastest bikes and wheels at each Zwift level” post, have not been updated, because those require manual updates. We don’t anticipate any major changes to these lists, but items may get shuffled around a bit and we’ll summarize those changes in the Changelogs at the bottom of the posts.

Do Brands Pay for Zwift Performance?

Over the years, Zwift CEO Eric Min has said in a few different interviews that companies don’t pay in order to show up in Zwift. But with so many new frames and wheels arriving in game and companies appearing to jostle for position at the top of the Zwift performance heap, the community has been wondering: are companies paying for their products to perform well in Zwift?

I reached out to Zwift HQ with that simple question. Here’s what they replied:

Zwift has not commercialized partner products in the game as these products assist the in-game experience. What we ask in return is that brands bring additional value to our community – whether that be through partner-led events, community engagement, pro rides or other elements. 

Additionally, there is no pay-for-performance element to these partnerships. Performance and ratings are determined by taking available information about bike weight and aerodynamics to place the bikes and wheels in a reasonable relative position compared to other items in-game.

There you have it. Companies aren’t paying Zwift to have their frames and wheels perform well in-game.

Can I See the Raw Data?

Sure you can. Here’s the Google spreadsheet.

Questions or Comments?

Share below!

About The Author

Eric Schlange

Eric runs Zwift Insider in his spare time when he isn't on the bike or managing various business interests. He lives in Northern California with his beautiful wife, two kids and dog. Follow on Strava

Subscribe
Notify of
guest
31 Comments
oldest
newest most voted
Inline Feedbacks
View all comments
A O
A O
6 days ago

Has there been any testing of the different gear? We all know of aero helmets, and skin suits, it wouldn’t surprize me if there were a aero benefit of those terrible helmets and gloves we all get on our quest for the lightweight wheels up le alpe

dan
dan
6 days ago
Reply to  Eric Schlange

does that mean you can tell all those people wearing aero lids on a road bike to take them OFF!

A O
A O
5 days ago
Reply to  Eric Schlange

Thanks Eric – Glad to hear it’s been tested, it’s been something I’ve wondered about
So I can continue to wear my paux hawk and tall socks, and non arero helmet

Max
Max
6 days ago
Reply to  A O

+1 interesting to see if there has been any change here. Great effort doing all of this testing!

Matthew Key
Matthew Key
6 days ago

That is a huge amount of work Eric, very much appreciated by many of us.
The True Grit being even slower makes me a bit sad, since I just got one IRL, but hey. I can still use it, it just lets me make a pace partner a bit harder or an excuse for getting dropped by other gravel bikes 🙂

Sam
Sam
6 days ago

How are these tests conducted, some automated bots?

CouchTo1200K
CouchTo1200K
6 days ago
Reply to  Eric Schlange

This is a huge amount of work, even if you used a bot to do the “riding”. Did you do anything else to automate these tests (for tasks like starting/finishing rides, changing frames/wheels, recording times, etc.)? Did you have to do them one at a time or could you run multiple tests simultaneously using virtual machines?

Jaizki
Jaizki
6 days ago
Reply to  Eric Schlange

chapeau!
Thank you very much.

Carsten Re. from Germany
Carsten Re. from Germany
5 days ago
Reply to  Eric Schlange

So, you have slept 21 days in your computer-pain-cave … with the alarm clock ringing every hour 😴😳… puuuuh, that’s tough man 🙈

abcd
abcd
6 days ago

Wilier、Lapierre、Time、Look、CEEPO、Isaac、Merida——not famous enough to join in Zwift game

Jimmy
Jimmy
6 days ago

Glad I stuck with the Cannondale Evo and Trek Madone as my climber/all rounder set ups now. Seem to be long term solid performers and still haven’t been nerfed (yet).

JayDee
JayDee
6 days ago

“Performance and ratings are determined by taking available information about bike weight and aerodynamics to place the bikes and wheels in a reasonable relative position compared to other items in-game.”

And the new TCR is below average when climbing. Unbelievable.

Carsten Re. from Germany
Carsten Re. from Germany
6 days ago

Wow … that’s a lot of time ! The bot should be exhausted until X-Mas now ;-).
I’m very thankful that you have retested the whole Drop-Shop stuff for the big indoor community. I appreciate your work and will dive deep into the data now :-).

Zach S
Zach S
6 days ago

Guessing some tabs on the raw data spreadsheet are hidden unintentionally? Looks like only frame differences on the flat are shown, and only wheel differences on the climb.

Temps 09
Temps 09
6 days ago

It seems the Felt AR and the new Cervelo s5 2020 are tied with S5 better for the flats and the Felt for rolling/hilly terrain. Tough call between them depending the course. Felt is looking close an all arounder now. Glad I did not get a new climbing out the drop shop and the Cannondale Evo has lots use still. Now if I could get lucky with alp spinner.

Thomas Christiansen
Thomas Christiansen
6 days ago

Thanks Eric!
Much appreciated!

Thomas Christiansen
Thomas Christiansen
6 days ago

Any change on the dirt? Have you re-tested the jungle circuit as well?

b.schraven
b.schraven (@b-schraven)
6 days ago

Nice work as always Eric. I’m curious though. Would the order of fastest frames/wheel be the same at, for example, 200W or 250W compared to the 300W test?

Evan
Evan
6 days ago
Reply to  b.schraven

I think so. The results would be a lot closer together, I believe. @eric Schlange correct? btw, have you done tests with fastest frames at, say, 2000 watts? Frames would be farther apart, making it clearer whichever is faster.

Rob GZ
Rob GZ (@robgrootzwaaftink)
5 days ago

It’s posts and dedication like this why I love this site so much! Keep up the good work Eric!

robertpaulson
robertpaulson
5 days ago

you know, I’ve always wondered if the tron has an advantage in the draft – maybe there is an actual zwift implication from the rider never getting out of the drops?

Bas de Vries
Bas de Vries
4 days ago

Thanks for researching this and I am especially happy to learn that the Lightweight Millensteins are as fast as the Roval Alpinists for climbing. That means that my many attempts to win those wheels (twice, but sometimes you get lucky in life), were not for nothing!

Andrew
Andrew
2 days ago

Nice work doing a full update of all Frames and Wheels, your bot deserves a day off…but afterwards:
Would be great to see some comparative testing on popular (or eventually ALL) routes whereby you take the Tron, Fastest Flat combo, Fastest Climbing combo, and Fastest All-Rounder combo and put all 4 through a 2-lap or whatever bot text on popular routes like Richmond UCI, Lotta Lava, Pretzel, Greatest London, Champs Elysée etc to see how they stack up against each other in the “mixed” terrain of actual up-and-down routes that aren’t virtually flat or virtually all uphill 👍

Free Zwift Trial

Newsletter Subscription

31
0
Would love your thoughts, please comment.x
()
x