How Protern.io validates its timing accuracy

It is important to trust your tools.

More and more coaches and teams are adopting Protern because it provides insights they cannot get from optical timing systems. They are also finding it easier to use, and the instant setup frees up a significant amount of a coach’s time.

If you want a quick overview of what Protern supports beyond timing, start here: Capabilities overview.

Trusted by the world’s best

Protern customers include the world’s best athletes and teams, such as Swiss-Ski, Swedish Alpine Ski Team, Alpine Canada, and the German Ski Federation.

None of these teams blindly trusted Protern’s accuracy and value to their athletes. They did their own tests and, in many cases, the Protern team worked closely with them to improve the technology.

Teams have regularly run Protern alongside their existing optical timing solutions to compare the accuracy of timing ¹.

After performing tests in varying conditions, different locations, and multiple disciplines, they have come to the point where they trust Protern to give them information they can use with confidence when coaching an Olympic gold medalist or World Champion.

Related reading: Start split location and the impact on timing accuracy.

Comparing optical timing with GNSS timing requires a methodical process

Protern uses an entirely different approach to capturing times than optical timing.

The Protern Sensor connects to multiple GNSS constellations (Global Navigation Satellite System, often referred to as GPS) to determine where the skier is at any given point. Unlike a laser, splits in Protern do not exist in real life. They are virtual lines drawn on a map.

Matching the exact location of a virtual split and a physical laser is impossible.

This means Protern total times will never be exactly the same as the optical time. Instead, there will be an offset between the two timing systems. So if the optical time says 11.5 and 11.3 seconds for a section, Protern might say 12.1 and 11.9. In this case, the offset is 0.6s.

To control for this offset when comparing the two systems, we set one athlete as the reference and look at the difference in time from that athlete.

Our method of validating Protern timing precision

Ensure proper setup

1. Protern sensors are fully charged and in a proper fitting Protern gilet on the outside of a ski suit
2. Optical timing is set up correctly on the hill as per FIS guidelines
3. Ensure the weather won’t introduce unnecessary errors from either the optical or Protern system (not too cold, heavy snow, strong wind, etc)

More data points, the better

1. One optical split is the absolute minimum to test, although it is best to use more optical splits for additional data points
2. Two skiers, each with Protern sensors, are required as one skier’s time will be used as a reference to determine the offset time between the optical and Protern systems
3. More runs and more skiers with Protern sensors provides higher confidence in the results

Testing scenario should be real

1. Use a real training day and course
2. Avoid “test courses” or specialized testing, because you want to confirm the tool works in the same conditions it will be used
3. Instruct skiers to ski through the finish rather than standing up early

Marking optical split and finish locations for Protern

1. With a Protern Sensor, ski the course to set a reference run to build a quantriq
2. At the first optical split, ski the length of the laser in as straight and accurate a line as possible to be used in the Protern map
3. Repeat the above step for all optical splits and the finish
4. For testing, it is not important to capture the start location

Building the quantriq to get the data

1. Use the reference run with the optical split and finish locations to set the Protern splits
2. Ensure the Protern splits are aligned with all splits. Zoom in on the map and adjust split handles as necessary
3. For the finish, align the Protern split so it is parallel to the finish, but place it 1 to 3 metres before the actual finish ²

Setting a reference athlete and comparing times

1. Open your favourite spreadsheet program
2. Define which athlete will be your reference
3. For each section there will be 5 columns: Optical Section Time, Optical Difference, Protern Section Time, Protern Difference, Optical and Protern Difference
4. Optical Section Time: For each athlete and each of their runs, enter section times (time between splits) from the optical unit. Ignore the first section (start to first split)
5. Optical Difference: For each athlete, subtract their section time from the reference athlete’s section time. This tells you how much faster or slower they were relative to the reference
6. Protern Section Time: Repeat the same process as for Optical Section Time
7. Protern Difference: Repeat the same calculation as for Optical Difference
8. Optical and Protern Difference: Subtract Optical Difference from Protern Difference. This number is the variance between the two systems
9. Use the STDEV.S formula in Excel to calculate the standard deviation of the variance to determine the difference between the two systems

The sharp edge of the knife

Protern testing has seen timing variance between 0.05s and 0.03s in tests from normal training sessions through to World Cup training runs.

There will always be a difference in time between Protern and an optical system. That difference is made up of variances within the optical timing, Protern timing, and the test itself.

A mistake in the testing process can easily make very accurate timing data look wrong.

Not setting Protern splits in the same locations, or not setting them parallel to the laser, can produce differences up to 0.2s. Slow speeds, or a sensor without a clear view of the sky, will also impact results.

In the end, doing a proper comparison is a methodical process, especially considering the degree of error can be less than 0.04s.

Protern has the trust of the best in alpine skiing. We aim to have yours.

If you want to talk through how to run this test with your setup, contact us here: Talk to us.

Notes

1. What about speed validation?

Protern measures speed differently than an optical system. Optical timing takes the average speed between two lasers, whereas Protern measures the skier’s speed along the run.

Because of these different approaches, it is not possible to get a direct comparison.

Federations and institutions have instead used industry grade differential GPS mounted on a skier using a backpack, with an external antenna mounted to the helmet. The data is then processed for later comparisons.

2. Why place the Protern finish split early?

Absolute positioning error in GNSS tools means the Protern split will likely not be exactly on top of the real finish. It will typically land within a 1.5 to 3 metre range.

If the system places the Protern finish after the actual finish, Protern timing may include a skier standing up and starting to brake. If the Protern finish is too far before the actual finish and skiers stand up early, Protern may measure a tucked phase while optical measures a stood-up phase. These situations show up as large differentials between the systems.