How fair is Zwift racing?

Virtual racing has grown at a rate that's taken much of the cycling world by surprise. Like Jack's enchanted beanstalk, all too suddenly treading a pathway to a hidden universe, this new genre within an often antiquated sport has progressed from a first National Championship event in 2019 to the planned UCI World Championships in 2020.

Regulation has had to be drafted up at a similarly whirlwind pace.

Most of us hopping on Zwift for a Tuesday night smash up in Watopia don't expect clinical accuracy. Anyone suspected of adjusting their weight favorably is more likely to receive a gif of a flying pig than a genuine reprimand, but as the stakes rise, so too does enforcement (especially after Britain's first male e-racing champion was stripped of his title).

In 2019, Zwift set up its own organisation designed to spot data hackers. The ZADA (Zwift Accuracy and Data Analysis) team is a progression from a community group which disbanded when the growing popularity of e-racing made the job insurmountable for volunteers. The new policing system is made up of a group of Zwift consultants and employees.

So how big is their job, and how fair is the current world of cycling e-sports? We took a look at some of the key factors...

Power meter accuracy

Newsflash: 250 watts on your mate’s power meter isn’t the same as 250 watts on your power meter. Even if you’re both using an SRM. And just because Billy from the cycling club found that a Wahoo Kickr read 20 watts higher than his Stages crank, it doesn’t mean Sally also using a Wahoo Kickr needs to subtract 20 watts from her FTP.

The number a power meter spits out for a given effort will deviate between units and, by a small amount, on the same unit from one day to another.

A 2017 study looked at 52 different power meters, assessing them for ‘trueness’ (reliability of the number given versus actual power produced) and ‘precision’ (variability between results from the same power meter). The researchers concluded that power meters ‘vary considerably in their trueness’ whilst precision was ‘generally high but differs between manufacturers.’

This has always been the case. When a power meter says it’s accurate to ‘+/-1%’ what it means is that, on average, across all the power meters of that class, you can expect the unit to read within one per cent of itself from day to day.

Sports scientist and coach, James Spragg, explains: “Before virtual racing come along, power meters needed to be relatively accurate to themselves, to use as a training gauge. As a training tool they work fine. But when you start comparing the values from a Stages crank to an SRM then you start to get issues. Because they were never designed to do that.

"Between two SRMs you could have an eight per cent swing. Between two Stages power meters you could have a 12 per cent swing," he says, referring to data from the 2017 study.

Zwift has practices in place to ensure fairness across the top tier of races, with the level of scrutiny increasing with the professionalism of the race.

Zwift's senior PR manager, Chris Snook outlined: “For Pro-Am competition, we ask that riders record multiple power sources and ensure that their power meter or trainer is calibrated or zero offset before they race. Multiple power sources help to validate performances after the race and help to highlight any discrepancies or miscalibration.

“In Pro-Am races like the Zwift Classics, the top three and two randomly selected riders are asked to submit their power data after the race. This data is analysed by ZADA, a team of data scientists. If required, riders may be asked to submit additional outdoor data, or perform a series of test to validate performances. If a rider is riding a poorly calibrated power meter, they will be disqualified from the race.”

All of this is outlined in Zwift's own rules and regulations. Things become more serious when there are jerseys to be won – such as the British National Championships and the upcoming UCI events.

Snook explains: “For races with titles on the line, like the British Cycling eRacing National Championships, these are held in live environments. This ensures we can control all variables.  In the instance of the BC nationals, all riders were using Wahoo Kickr trainers as the power source, and all were calibrated before the race to ensure complete fairness."

But what about amateur racing? Put simply, if Zwift wants to be accessible, this level of scrutiny can't be applied.

"For community racing, it is much harder to police at this time. It's important to remember that community racing is meant to be fun" Snook says, adding "and it's important that racing Zwift remains accessible."

This said, there are various rule sets for different levels of racing - and community organisers can choose to enforce their own race rules, too. Organisers choose, for example, if riders are required to use a heart rate monitor in order to have their results appear on the official rankings in Zwift Power -  the community driven website which tracks race results.

To keep it fun, Zwift is working on reducing ‘sandbagging’, where stronger riders enter races designed for those in lower categories. Snook added: “This is still in an experimental phase at the minute, but those in the wrong category will be flagged, and will be slowed in game - effectively restricting their watts to match the category they are in. Moving forward, we are looking into machine learning that will help us police race results using AI.”

Of course, racers themselves can improve accuracy, by completing a zero offset and calibrating their power meter before each ride or race.

Accounting for drag

Aerodynamic drag is by far the greatest limiter of speed for any cyclist, with about 80 per cent of it coming from the rider’s body (next up is rolling resistance but that’s another conversation).

The best way to limit drag is to draft in the peloton, and Zwift, like all e-sports cycling platforms, mirrors this by having the power required to move at a given speed drop in line with the number of riders ahead.

The level of relief is affected by the number of riders, and indeed Zwift confirmed – the size of those riders. Quite incredibly, if you’re in a two-up break with a rider who has entered their height as 6ft4, you’ll get more protection than if your partner in escape was 5ft4.

On an individual level, things get harder to predict. In the real world, a rider’s Coefficient of Drag area (CdA) is affected by factors including their height, weight, proportions, position and equipment.

How does Zwift work it out? “We actually use the approach from a peer-reviewed study for correlating height and weight to an approximation for surface area for cyclists,” Snook says. “In-game aerodynamics do consider both height and weight. This is something we are always reviewing, and it's likely that it will develop.”

Spragg refers to this method as assessing a rider’s ‘body surface area'.

“What you don’t get in Zwift is any positional effects," he notes. "If you take two riders who are 68kg, one is 176cm and the other is 180cm, the guy that's 180cm does have a bigger body surface area. So that would translate in Zwift into a higher virtual CdA. In the real world, it may be that the taller chap is more flexible, and can get into a more aero position.

“Body surface area isn’t the only thing that affects CdA, there's shoulder width, position, proportions, all those factors aren’t taken into account in the Zwift world.”

>>> Are aero sensors the tech of the future?

So, if you’re a tall rider with a snake like spine and a low front end? You might be being diddled by the algorithm. This said, unless personal CdA measuring tools develop at a rapid rate, there’s not much anyone can do about this; “I think Zwift does a great job,” Spragg says. “Aside from putting everyone in front of a green screen, I don’t see how they could do this any better.”

The effect of equipment is of course taken into account, with Zwifters able to access bikes with improved aerodynamics and a lower weight via time spent in the games universe; attempts to cheat this are of course an infringement.

Weight

The most widely referenced method of ‘Zwift race doping’ is of course a rider adjusting their weight to unlock a superior power to weight ratio.

This obvious hack is well policed from the Pro-Am level – with Zwift outlining a very specific ‘race weight video’ protocol. This includes the rider weighing themselves on camera, as well as holding up a piece of media displaying the day’s date, and demonstrating the reliability of the scales by weighing a ‘heavy weight, approx. 10kg.’

For amateurs? Well, according to ZwiftPower's FAQ, if you're churning out 5w/kg, you can expect to be disqualified. Ditto if your w/kg figures are well in excess of the category race you've entered.

There is a flagging system - if riders break the rules of what is deemed to be 'humanly possible' they'll receive a pop-up notification ('Uh oh! You've missed your calling as a pro...') before their avatar is restricted to a realistic speed and they're removed from leaderboards.

Snook explains: "This tool really is designed for the extremes. We are also looking into machine learning tools that will be able to auto-detect weight fluctuations or unrealistic behaviour."

Aside from that? All rider weights are presented on Zwift Power for the world to see. If your online profile suggests you come in at 45 kilograms and a sideways glance would suggest otherwise, then you're only cheating yourself.