Not very – according to a review recently undertaken by CNET.
Senior Editor Sharon Profis collaborated with Cardiologist Dr Jon Zaroff to compare some popular health tracker devices to medical ECG equipment at rest and during treadmill exercise at quite high heart rates.
Dr Zaroff explains that whilst heart rate monitor chest strap devices are essentially a single lead ECG in a sweat proof packaging, and with very good accuracy, wrist worn optical pulse measurement sensors depend on patterns of light absorbed & reflected from blood vessels beneath the skin surface. These light patterns are sensitive to skin colour, blood vessel density and any relative motion between the blood vessels & the device itself.
The table below is taken from the article and shows significant errors for all the devices, but most especially the wrist worn ones at typical exercise intensities.
| Device | Error (%) at 80-90 BPM | Error (%) at 160-170 BPM |
|---|---|---|
| Garmin VivoFit | 10.7 | 0 |
| Withings Pulse O2 | 5.3 | 57.1 |
| Basis Carbon Steel | 10.2 | 57.9 |
| Samsung Gear Fit | 4.2 | Unable to read |
| Samsung Galaxy S5 | 3.1 | 0.2 |
As the article also points out though, measurements taken by passing light through the finger tip are inherently more accurate, and this is the principle we chose for the ithlete finger sensor. To put the needed accuracy for HRV into perspective, the ithlete app measures each heartbeat interval to a precision of 1-2 milliseconds which equates to <0.2% error at a resting heart rate of 60 bpm. When validated by the University of Sydney, the average heart rate derived from the ithlete finger sensor was essentially identical to that from the reference ECG. The dedicated pulse sensor of the Samsung Galaxy S5 also measures pulse at the fingertip and acquitted itself quite well.
Although many of these devices carry disclaimers about accuracy, they are not cheap toys and I think consumers have the right to expect reasonably informative (ie accurate) heart rate data out of them both at rest and during exercise. Needless to say, none is close enough yet to use for HRV!
The question is what accuracy we need for ithlete’s purposes. Maybe even a ten percent measurement error evens out over a large sample? Put differently, what level of heart rate meter accuracy is needed to be confident that a day-to-day change in HRV is not measurement error?
There’s a guy on another blog who tested RMSSD variability on the Mio alpha, Polar H7and Armour 39 vs an ECG. The Mio was quite inaccurate, while the other two were spot on.
http://www.marcoaltini.com/blog/heart-rate-variability
Hi Jens. Thanks – I saw this one, and I also read that Mio would not recommend using the wristwatch products for HRV, which begs the question as to why they still send the RR intervals over Bluetooth. I wish the BT4.0 spec stipulated a minimum accuracy of ±4ms for sending RR intervals. I also wish all BT4.0 straps would send RR intervals correctly, since app developers like us tend to get the blame when they either don’t work at all or worse, give misleading readings.
Hi Simon – do you have a link to where Mio stated this. I agree – it’s a big shortcoming of that sensor.
Perhaps a firmware upgrade could help, or is it inherent in the sensor?
Ok… I rtfm’d and found it here:
file:///Users/jeffme/Downloads/Mio_FUSE_Complete-User-Guide_EN_v12.pdf
On page 10, under “Tips for Pairing” (oddly), where it says:
Heart rate calculation of the Mio FUSE is not based on instant R-R interval, and will notwork for apps and devices that require heart rate variability (HRV) data.
Straight from the horse’s mouth!
Hi Jefe,
Good find – from the horse’s mouth indeed.
These kind of wrist worn sensors need to have really good algorithms for detecting motion artefacts, which happen all the time when someone is exercising. Unfortunately these seem to greatly reduce the accuracy of R-R intervals, compared to a dedicated finger sensor which is only intended to be used at rest.
More evidence of wrist worn fitness band inaccuracy – this time in a review by WSJ writer Joanna Stern:
http://www.wsj.com/articles/fitness-bands-with-heart-rate-tracking-are-missing-a-beat-1418759929