Study shows energy readings off over 90%
“The heart rate measurements performed far better than we expected,” said Ashley, “but the energy expenditure measures were way off the mark. The magnitude of just how bad they were surprised me,” said Euan Ashley, co-author of the paper and professor of cardiovascular medicine, of genetics and of biomedical data science at Stanford.
Seven devices were tested: the Apple Watch, Basis Peak, Fitbit surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2, on 60 volunteers.
All in all, the Apple Watch came out on top and the Samsung Gear S2 was last as it had the largest error rates. The Samsung Gear S2 was the only gizmo to overstep the 5 per cent error percentage for the pulse measure by 0.1 per cent.
A closer look at the results shows that this varies wildly for calories burned. None of the fitness trackers were accurate enough. The median error rate for the best performing device for 27 per cent for Fitbit Surge, but a whopping 93 per cent for PulseOn.
The average percentage of error increased slightly for men, darker skin tones, higher BMI and when participants were engaged in less intensive exercise such as walking instead of running or cycling.
The test group included 31 women and 29 men; they wore the each device whilst walking, running or cycling. An electrocardiograph tracked the volunteer’s pulse, and the metabolic rate was calculated by measuring oxygen and carbon dioxide levels in breath to estimate the amount of calories burned during exercise.
Experimental data taken from gas analysis from indirect calorimetry and an electrocardiogram were used as the “gold standard” to work out the accuracy of each device. The precision for finding out energy expenditure and heart rate varied across different exercises, gender, body mass index (BMI) and skin tone.
“People are basing life decisions on the data provided by these devices,” said Ashley. Although users can rely on using a fitness tracker for heart rate, “basing the number of doughnuts you eat on how many calories your device says you burned is a really bad idea,” he concluded.
The study shows that although results from these fitness trackers could give physicians or doctors a way to test a patient’s cardiovascular health, they should think twice before accepting them as valid medical data. ®