It is important to trust your tools.
More and more coaches and teams are adopting Protern.io because it provides insights they can not get from optical timing systems. They are also finding it easier to use and the instant setup frees up a significant amount of a coach’s time.
Trusted by the world’s best
Protern.io’s customers include the world’s best athletes and teams, such as Swiss-Ski, Swedish Alpine Ski Team, Alpine Canada, and the German Ski Federation.
None of these teams blindly trusted Protern.io’s accuracy and value to their athletes. They did their own tests and in many cases, the Protern.io team worked closely with the teams to improve the technology.
The teams have regularly run Protern.io alongside their existing optical timing solutions to compare the accuracy of timing 1.
After performing tests in varying conditions, different locations, multiple disciplines, they have come to the point where they trust Protern.io to give them information they can use with confidence when coaching an Olympic gold medalist or World Champion.
Comparing optical timing with GNSS timing requires a methodical process
Protern.io uses an entirely different approach to capturing times than optical timing.
The Protern.io Sensor connects to four different GNSS constellations (Global Navigation Satellite System – often referred to as GPS, which is the name of the US satellite constellation) to know where the skier is at any given point. Unlike a laser, splits in Protern.io do not exist in real-life, but are virtual lines drawn on a map.
Matching the exact location of a virtual split and physical laser is impossible.
This means that Protern.io’s total times will never be exactly the same as the optical time. Instead, there will be an ‘offset’ between the two timing systems. So if the optical time says 11.5 and 11.3 seconds for a section, Protern.io might say 12.1 and 11.9. In this case, the offset is 0.6s.
To control for this offset in comparing the two systems, we set one athlete as the “reference” and look at the difference in time from that athlete.
Our method of validating Protern.io’s timing precision
Ensure proper setup
- Protern.io sensors are fully charged and in a proper fitting Protern.io gilet on the outside of a ski suit
- Optical timing is set up correctly on the hill as per FIS guidelines
- Ensure the weather won’t introduce unnecessary errors from either the optical or Protern.io system (i.e. not too cold, heavy snow, strong wind, etc)
More data points, the better
- One optical split is the absolute minimum to test, although it is best to use more optical splits for additional data points
- Two skiers, each with Protern.io sensors, are required as one skier’s time will be used as a reference to determine the offset time between the optical and Protern.io systems
- More runs and skiers with Protern.io sensors will give more data to give a better confidence in the results
Testing scenario should be real
- We use a real training day and course
- We avoid ‘test courses’ or specialized testing, because we want to make sure the tool works well in the same conditions it will be used
- Instruct the skiers to ski through the finish rather than standing up early
Marking optical split and finish locations for Protern.io
- With a Protern.io Sensor ski the course to set a reference run to build a quantriq
- At the first optical split, ski the length of the laser in as straight and accurate of a line as possible to be used in the Protern.io map
- Repeat the above step for all optical splits and finish
- For testing it is not important to capture the start location
Building the quantriq to get the data
- Use the reference run with the optical split and finish locations to set the Protern.io splits
- Ensure the Protern.io splits are exactly aligned with all the splits – zoom in on the map and adjust Protern.io split handles as necessary
- For the finish, align the Protern.io split so that it is exactly parallel to the finish, but place it 1-3 metres before the actual finish 2.
Setting a reference athlete and comparing times
- Open your favourite spreadsheet program
- Define which athlete will be your reference
- For each section there will be 5 columns – Optical Section Time; Optical Difference; Protern.io Section Time; Protern.io Difference; Optical and Protern.io difference
- Optical Section Time – For each athlete and each of their runs enter their section times (time between splits) from the optical unit – ignore the first section (i.e. start to first split)
- Optical Difference – For each athlete, subtract their section time from the reference athlete’s section time. This will tell you how much faster in time each athlete was relative to the reference.
- Protern.io Section Time – Repeat the same process as done for the Optical Section Time
- Protern.io Difference – Repeat the same calculation as done for the Optical Difference
- Optical and Protern.io Difference – Subtract the Optical Difference from the Protern.io Difference. This number will be the variance between the two systems.
- Use the STDEV.S formula in Excel to calculate the standard deviation of the variation between the two systems to determine the difference between the two systems
The sharp edge of the knife
Protern.io’s testing has seen timing variance between 0.05s and 0.03s in tests from normal training sessions, to World Cup training runs.
There will always be a difference in time between Protern.io and an optical system. That difference will be made up of a combination of variances within the optical timing, Protern.io timing, as well as any errors in the test itself.
A mistake in the testing process can easily make very accurate timing data look wrong.
Not setting the Protern.io splits in the exact the same locations, or completely parallel to the laser can easily give differences of up to 0.2s. Slow speeds, or a sensor that doesn’t have a clear view of the sky will also impact the results.
In the end, doing a proper comparison is a methodical process, especially considering the degree of error is less than 0.04s.
Protern.io has the trust of the best in alpine skiing – we aim to have yours.
Notes
1 What about speed validation?
Protern.io measures speed differently than an optical system. Optical simply takes the average speed of the skier between two lasers. Whereas Protern.io measures the speed of actual skier.
Because of these two different approaches, it is not possible to get an accurate comparison.
Federations and institutions have instead used industry grade differential GPS mounted on a skier using a backpack and external antenna mounted to the helmet. The data is then processed for later comparisons.
2 Why place the Protern.io finish split early?
The errant absolute positioning error of GNSS tools means that Protern.io’s split will probably not actually be on top of the real finish. It will be within 1.5 to 3 metre range. If it the system places the Protern.io finish after the actual finish it is possible that Protern.io time takes into account a skier standing up and starting their braking. Similarly, if the Protern.io finish is placed too far before the finish and the skiers have a habit of standing up before the finish, it is possible Protern.io will measure a time when the skier is tucking and the optical will measure when they have stood up. These situations will show up as large differentials between the two systems.
