Many of us are interested to see the number of calories burnt during a physical activity or the amount of energy spent on a specific activity, also called the energy expenditure (EE).
For some people, it has an emotional value associated with a reward after a hard workout. For others, having accurate estimates is important for weight and nutrition management.
However it turns out that estimating EE is not that simple. There are 3 different methods used in laboratory settings to do it. For those interested, I added a short description at the end of the post.
Given that we typically don’t have access to a lab, we can only rely on wearables and sport devices to know how many calories we spend during exercise. But how reliable are sport watch’s estimates?
Before we dive into comparison between different watches, let’s see what factors are used in EE calculation. Your age, weight, height, sex, maximal heart rate (HR max), and maximal oxygen uptake (VO2peak) are the main inputs to calculate the individual EE. And then you can play with different predictive equations, and there are plenty of them!
All sport watch developers are using proprietary algorithms to estimate EE, and they mostly keep their algos secret. Well, it seems that almost all of them are using the same software developed by a 3rd company called Firstbeat, based in Finland. Unfortunately, there is not much research comparing the validity and reliability of EE estimates of these algorithms.
Originally Firstbeat's algorithm for calorie computation was based on the recorded heart rate during an activity, beyond your gender, weight and height measurements. This algorithm used heart rate and the user profile to calculate calorie readings. There are a lot of issues with this approach, because it tends to overestimate the calorie count when the heart rate is still elevated but the intensity is low (training interruption, stops at red light, stress). However, their recent model seems to address this error by incorporating other variables. I’ll explain it a bit later.
So how big is the error in EE estimates?
In 2017, Roos et al (2017) compared Garmin Forerunner 920XT, Suunto Ambit 2 and Polar V800.
Of course, those are older models, and most of us have already switched to the latest technology.
But let’s see what was reported in this study.
The accuracy of EE estimates is intensity dependent. For high intensity activities, the anaerobic proportion of EE is the most challenging to estimate accurately.
For aerobic running (4-11km/h) the error values were between – 25% (for Garmin) to + 36% (Suunto). Polar did a better job by showing only -12%/-4% error values.
For high intensity, all 3 watches significantly underestimated the EE. The error was proportionally increased as the running speed was increasing.
I was surprised to see such a discrepancy between Garmin and Suunto knowing they both use Firstbeat algo. And yes, dont be surprised, Suunto is also using Firstbeat analytic algorithms.
Only Polar is developing their own algos, and they seem to be fully in control of their data!
We are now in 2022, and logically, the technology should have been improved. Not so fast, the white paper that Firstbeat published in 2012 vaguely describing the proposed EE estimation model hasn’t changed.
But I hope the new variables incorporated into this model are more accurate, even if the model itself is still the same.
In fact, Firstbeat now combines not only the heart rate, but also the respiration rate derived from heart rate variability (HRV) and a third variable provided by the Firstbeat analytic engine that allows the model to measure oxygen consumption (VO2) as it changes.
The respiration rate is another algo based on the phenomenon, called respiratory sinus arrhythmia. As you inhale, your heart rate increases slightly, and when you exhale your heart rate decreases slightly. Firstbeat is extracting this data from HRV, but its accuracy is dependent on the accuracy of the heart rate data. Don’t be surprised, but all this oxygen consumption data is also included in your calorie count, seemingly improving the accuracy.
Firstbeat claims that the accuracy has been improved by 20-60% over original conventional heart rate base EE calculations. If that's the case, we can hypothesise that the error margin is now lower but still potentially in the double-digit figure. I haven’t seen any new study validating the accuracy of the Firstbeat model versus the gold standard such as doubly labelled water or even indirect calorimetry.
Recent watches are literally loaded with various algos. To give you an example, on the Garmin Fenix 6 there are 18 different Firstbeat licensed algorithms being used to cover all sorts of features from training load to VO2Max and respiration rate.
If you want to have more accurate readings, there are few nuances to consider.
The error is probably still large, especially during high intensity workouts.
Without wearing the chest strap heart rate monitor, the data is even less accurate, because the EE estimate heavily relies on various heart rate parameters.
Make sure all your training zones are well defined, including your max heart rate, and all your personal data is correct. If you have recently gained a bit of weight indulging in Christmas cakes and chocolates, don’t forget to update it.
In 2020 Garmin did acquire Firstbeat consumer product licensing part of the company, effectively becoming the dominant player also controlling the software. Very smart move! Garmin might have the largest database in sport wearables. Well, I don’t include Apple here, because it’s not strictly speaking a sports watch, although the data is still very useful.
Polar seems to be an outlier, with potentially more accurate data, at least for the EE < 75 % VO2Max.
And Suunto is now sourcing all the algos from Garmin.
Not sure, knowing it all will help you to burn more calories, but let’s hope all this accumulated analytical data might get us better calibrated underlying algos in future.
Thanks for reading.
Note
3 methods for EE estimates.
Direct calorimetry, when we measure the heat emitted by the human body at rest or in activity in a special room. Highly accurate method.
Indirect calorimetry, when we measure the concentration of inhaled or exhaled gases - currently the most used technique. But it doesn’t correctly estimate anaerobic metabolism.
Doubly labeled water, when the participants ingest the water with traced elements. This method is very expensive, but it’s considered to be the gold standard.