Cycling Zwift

ZwiftPower – The coding behind Zwift Community Racing

Zwift has grown from the brainchild of Jon Mayfield into an arguably recognised cycling powerhouse which, I don’t think it is an understatement to say, has revolutionised indoor cycling. In the early days of Zwift, the priority of the company was on making a great user experience. As cyclists, our priority was going fast, which meant races, which early Zwift users created their code to facilitate. This system is now known as ZwiftPower and underpins the majority community of Zwift race results. So let’s take a look under the hood

ZwiftPower is now a core feature of the Zwift Community, and anyone who races even casually on Zwift is aware of this vital racing results system. It can’t really be underplayed how important this system has been to the community of races and riders over the few two years or so. In fact is it is not an exaggeration to suggest that many of the current race teams of Zwift, such as Team Experimental, or KISS Race Team, would not have formed up as they have done without the data and the connections which have been made using ZPTitaniumGeek Screen-Shot-2017-01-17-at-14.31.31-1024x539 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

ZwiftPower is now administered by three lunatics chaps; Glen Knight, Christian Weidmann and James Hodges (Who is a little camera shy!)

TitaniumGeek Screen-Shot-2017-01-17-at-15.32.37-1-1024x236 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

To put in perspective of how much work this has entailed, Glen had a quick look at their previous conversations, and there have been approximately 21,000 messages regard Zwift power…

So we understand that ZwiftPower is important now, but where did this site and the code come from, and who is behind it? I think to look at that we need to understand what came first and where the need was.

Pre-Zwift Power – Human Power

In the VERY early days of Zwift races, such as the Zwift Training Races, hosted by Christian Weidmann and (myself for a short period), results were done manually. Either by looking at Strava segments, which provided to be highly unreliable or MANUALLY watching a rider cross the finish line on a video, and the more accurate times taken from that! Here is the type of finish video which would form the backbone of Christian’s results

You can see, at this point, we have 80 riders on THE WHOLE of watopia here – these were massive numbers back in 2015, whereas 80 riders are now considered reasonable attendance for an event or set ride! Also, all riders needed to have their race and category in the name, for Christian to allocate them positions and times. These rides were then posted on FaceBook and a few hours later when the numbers had been ground by hand (To say crunched by hand vastly underplays Christian’s efforts) as you can see below

TitaniumGeek Screen-Shot-2017-01-17-at-14.28.30-519x1024 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

Pre-Zwift Power – Rise of the code

From there a coder by the name of Jonathan Lemon wrote the original Zlogger Utility, which has been the coding foundation which ZP is based. This is as aspect which I have found very interesting in looking at the Zwift community. So many of the surface aspects of the projects and programs run by the Zwift riders themselves are based on layers of other projects which have come before. Many of which rely on this original code by Jonathan Lemon.

TitaniumGeek 40002_1406368033489_5514870_n ZwiftPower - The coding behind Zwift Community Racing Zwift cycling   Zlogger replaces the human viewer, and “watches” the Zwift riders on the courses and reports every cyclist which cross specific lines in the road, such as the start/finish lines and KOM lines. In doing this Jonathan off loaded a colossal amount of grunt work required in producing race results to lay the foundation for ZwiftPower. Zlogger automated the process of getting race data, but still required a lot of manual work from the race organisers requiring a different configuration for each different course and race length to run reports.

TitaniumGeek 419584 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

Pre-Zwift Power – Making connections

Zlogger has now been replaced for many people by ZwiftPower but is still vital work on it’s own, as that code is still used to retrieve the data, which is then pushed to ZwiftPower. It is that interface which is the part of the project focused on by Team Experimental rider, and Zwift Community Elder, Christian Weidmann

TitaniumGeek 15909508_10209375429637508_180617696_n-2 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

Originally Christian needed some extensions for the ZTR race series, to facilitate the continued race reports, as the ZTR series began to develop a more formal rule set, and in doing so Christian took over from Jonathan as the most active contributor to the data coming into ZP. Glen Knight had the idea of storing data from multiple watchers in a common database and turned to Christian to write the script that transfers the data from zlogger into the ZwiftPower database infrastructure.

ZwiftPower is now based on the idea observing riders performance from multiple watchers placed around the tracks, rather than having computers monitoring riders crossing road markers. Hence why many of the photos showing the inner workings of the KISS Race series, which rely completely on ZP, have many, many instances of Zwift running at the same time, to facilitate this tracking

TitaniumGeek 15817689_10154926364502558_990539990_o-2-1024x571 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

In addition to recording performance, which is kind of vital to a race series, Christian has built other scripts that retrieve rider profiles and scheduled events directly from Zwift which Zwiftpower is then able to populate a lot of the structure you see on the website

TitaniumGeek Screen-Shot-2017-01-17-at-16.17.47-1024x417 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

Zwift has not released a public API. Thus the biggest challenge in working on Zwift power has been figuring out how to access Zwift’s internal API. Several community members independently discovered how to do this. Christian was able to figure a route to the API by looking at Zwift;s dashboard and see how the login worked and how it retrieved information from Zwift’s servers.

TitaniumGeek Screen-Shot-2017-01-17-at-16.21.38 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

ZwiftPower is born

TitaniumGeek cWvZSmc-1024x562 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

If Zlogger is attributed to Jonathan, access to Zwift data, and it’s integration by Christian, then ZwiftPower itself can be attributed to James Hodges. ZP began as a personal project to track the data and relative power output of other racers. That data being weight, w/kg, np, power zones and 20min/5min/1min best efforts. Initially, it was only intended to cover the 2 or 3 races a week James actually took part in himself; then KISS Race Boss came calling…
TitaniumGeek 15129696_1000878250023933_60591677052844412_o-2 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling
Glen Knight pitched James some crazy ideas about race monitoring and the idea of building the huge data farm that we know as ZP – that’s when the real work began!!
TitaniumGeek 16106819_10154980315087558_1913008130_o-1024x768 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling
As the sole website developer for ZwiftPower James Hodges is responsible for keeping the site running and implementing new features. ZwiftPower takes the automation a step further than it’s Zlogger underpinnings by using the Zwift event module configuration to determine course and time, then automatically creating race entries based on this information. It was this tracking riders via LIVE data which proved to be one of the biggest challenge. Mainly as the data is often incomplete and has required weekly updates to the site to keep everything running smoothly. But in the same way, as the live data gave James the biggest challenge, so it has been one of ZP’s biggest successes. Now Zwift events are scanned, created and results collated with no human intervention, in turn providing the race data, both in terms of races, but also personal results for rides we have become accustomed too in the Zwift Community
TitaniumGeek Screen-Shot-2017-01-17-at-16.55.03-1024x676 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

Where does KISS feature in all of this?

TitaniumGeek 15820642_10154931643537558_1662747629_o-2-1024x653 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

In the early days of the Zwift Community, things were a little like the Wild West; people were learning, and doing, and building practically at the same time. Regarding race series, the central events were ZTR, ZHR, ODZ, KISS & Asia. Glen Knight approached Jonathan Lemon and Christian Wiedmann about centralising multiple watchers over multiple courses, so the data was all consistent but also available all of the time.

With data available all of the time, it was only a small step to start a conversation between James and Glen over the idea of taking this continually available data, and making it “live” data, and thus Zwiftpower LIVE. If the work by Jonathan, Christian and the Zwift pioneers is the bedrock of process, the current ZwiftPower collaboration is the end product, and is now used by most race organisers.

Glen has tracking data on nearly 3000 different events. To put that into context, the 17th of Jan had 33 different races listed as going through ZP. TitaniumGeek Screen-Shot-2017-01-17-at-17.47.36-885x1024 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

But what does that actually take? – Well, Glen hosts 20 physical PCs running various instances of Zwift, Zlogger and their own scripts for data transfer/conversion. On top of that are 5 servers which serve ZwiftOnBoard a feature released at the end of 2016 for Nathan’s live stream allowing you to view riders as they compete in races on Zwift

From a technical side this involves a MySQL server, on the local network so that the internet connection does not effect the transfer of data to the database from the 40+ instances of Zwift. This then replicates to 2 cloud-hosted MySQL servers. One of these servers is used to run scripts for event and rider profile information. The other is used for direct access by the ZwiftPowerfront-end Web server, which is also hosted by Glen – which is should be noted is doing so without charging the Zwift Community anything. This whole enterprise is free for any Zwifter to use!

Things are not finished, every week there seems to be a new challenge for the ZP team. Although in many ways, it seems the biggest challenge that James faces is Glen coming up with another feature request or idea, which I’m told comes genuinely occurs about weekly! However, a very pressing concern for ZP is simpy infrastructure and growth of Zwift. As there have been major changes on Zwift, from when times with 41 riders on Jarvis Island were peak traffic…

TitaniumGeek 2015-04-03_1804302-copy-1024x550 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling   To today when it is considered a “quiet time” with 1400+ riders on the island
TitaniumGeek 2016-11-21_193940197-copy-1024x550 ZwiftPower - The coding behind Zwift Community Racing Zwift cycling

ZwiftPower currently monitors over 100 data sources across all three courses, and that monitoring is 24/7 – however you chose to look at is, that’s an awful lot of incoming data especially when on peak times Zwift is pushing towards 3000 riders. To put that in context ZP is currently storing records of 41,000,000+ line crossings since March 2016… and before you ask, yes, they have a backup!

When pushed for which part of the project the team are most pleased about there is a bit of a split. Some feel that the technical side, moving from a single server to a data cluster and all of the IT coding is the greatest achievement. Glen, on the other hand, is more impressed with the community result and how ZP has benefitted the race organiser but also the individual user regarding their results.

The team do chat regularly with the Zwift creators Jon Mayfield and Roberto Duarte regarding data and game setup, which is been particularlly imporrtant with live tracking riders on the course, but also gives them ideas about future projects. Given the team are as surprised as anyone at where they were a year ago compared to know, they are very bullish about what they can do in the future!

So what does the future hold for ZwiftPower?

The guys behind ZP still have plenty of ideas they would like to implement given time. These range from tweaks, too much larger changes – sometimes that related to actually implementing a function- ZwiftPower already have the ability to add rider telemetry, live chat data.
Streaming is already happening with ZPOnBoard, but the team want to focus next on stability and integration with ZwiftPower, to allow then additional course information to be displayed.
TitaniumGeek 15170978_1004150219696736_6233625007999496452_n ZwiftPower - The coding behind Zwift Community Racing Zwift cycling
We’ve been working on the way to adding segment PR as a ranking system which can also be used as part of the race setup. We will be adding the ability to support the KISS Zwift Quest event as well as other event types to spice it up a bit
ZP is probably one of the most vital resources to the Zwifting community, and as new system such as the Cycligent Virtual Ranking tool come online, relying on the data from  ZP, that importance is only going to increase!

The future is obviously ultimately affected by Zwift itself. As a result, a major from hope is that Zwift will publish an API that does the equivalent of zlogger without having to run a copy of the full game. This would allow the ZP team to run the infrastructure in the cloud and might go some way to saving Glen Knights electricity bill, but more importantly for the community, would hopefully greatly increase reliability too!

PERSONALLY, I think the throwaway comment about Glen’s power usage is very important. ZP is one of the cornerstones of the Zwift Community. It is easy to say someone will fill the gap if it closed, but that would still be a hugely difficult task. ZP has literally hundreds of man hours in it, a significant investment of time, but also funds for the infrastructure behind it. This is an aspect that is frequently not talked about, upkeep. ZP requires actual hard currency to run. I for one was surprised at the cost, which is being shouldered here by the team, for a community of people they will in reality never meet face to face. Let’s hope that the team keep up the great work, and continue to power the Zwift community for years to come!
From one rider – Thanks, guys!
James Gill

Author of TitaniumGeek, which started after smashing off my RIGHT elbow. Feel free to drop me a line about sports tech, medicine, or frankly anything that you want to chat about!!