Last Updated on September 6, 2022 by pg@petergamma.org
90.01 Introduction
One of the goals of the Meditation Research Institute (MRIS) is to find heart rate monitors which are comforable to wear, can be used for long-term meditation studies, are highly accurate, and have an excellent software support. During the evaluation process for heart rate monitors, we became interested in consumer grade sports sensors.
Doctors use ECG devices for clinical applications, scientists use ECG devices for research, since they are highly accurate, and have other benefits. If sports chest straps where more accurate than ECG, they would use them in the hospital and in their labs, but they don t.
There are several papers available which compare the accuracy of consumer grade sports sensors to ECG devices. The most accurate which where tested where:
1. Polar H7 with an accuracy of 98% compared to a Q-tel RMS ECG device
2. Polar OH1 with an accuracy of 95 % to 99 % compared to a g.Nautilus ECG device
3. Firstbead BodyGuard 2 with an accuracy of 99.95 % compared to an Embla Titanium ECG device
In these scientific papers, no data are available for the Polar H10 chest strap. Rob ter Horst, PhD, author of the Youtube channel «The Quantified Scientist» is a popular heart rate monitor accuracy reviewer with 124.000 subscribers. Rob uses two Polar H10 chest straps as reference device in his reviews.
We (basically) asked Rob via contact form several month ago: Why does Rob not use a high-quality (16 bit or higher 3 channel) research grade ECG device as reference device for his tests? In the mentioned papers, there are no data available for the Polar H10, only for the Polar H7. And the Polar H7 has an accuracy of 98 % compared to a Quinton
Q-tel RMS ECG device.
Soon after we asked Rob, he reviewed a paper about the accuracy of the Polar H10 from the:
Swiss Federal Institute of Sport Magglingen SFISM:
Rahel Gilgen‑Ammann Theresa Schweizer Thomas Wyss
which studied the
RR interval signal quality of a heart rate monitor and an ECG Holter at rest and during exercise
on his Youtube channel. Rob gave the Youtube video about the Polar H10 the title:
«Polar H10 Scientific Review: Best for Heart Rate (99.6% Accurate)»
The paper demonstrates that the ECG device used in the paper looses accuracy with activities at higher speeds, but not the Polar H10 chest strap. The authors of the SFISM paper found out that the:
– accuracy of the AR12plus Holter monitor ECG is 94.6 %
– accuracy of the Polar H10 is 99.63 %.
Rob argues, that his Polar H10 chest strap is also an ECG device, but still it only has two ECG electrodes and one channel. Rob uses the SFISM paper as a scientific validation paper and justification the use of the Polar H10 as reference device for his accuracy studies.
We where confused from these statements. The only paper which we knew to that day which says ECG devices are inaccurate was the is the SFISM paper.
When we look at papers with ECG as a reference, they do only study the accuracy of sports sensors at speeds up about 15 km/h, as we can see later on. We suppose, that the surprisingly strong inaccuracy of the ECG device in the Magglingen paper is due to ECG motion artifacts, which arise only at fast movements.
We found a study with the Adinstruments Equivital revealed problems with ECG motion artefacts, too:
According to the Adinstruments Equivital data sheet:
https://m-cdn.adinstruments.com/brochures/ADINSTRUMENTS_EQUIVITAL_BROCHURE.pdf
the Equivital has noise and movement artifact-free ECG. Whe therefore suppose that the problems with movement artifacts with the Adinstruments Equivital are solved in current Equivital devices. All the other validation papers we studied before with ECG as a reference device mentioned no problems with ECG inaccuracy.
The SFISM paper measures signal quality, but not inaccuracy compared to a reference device. The data can therefore not be compared directly. The SFISM paper could not convince us, that ECG devices are generally not accurate enough for sports sensor accuracy validation, and the Polar H10 is the most accurate «Gold Standard» reference device, as mentioned in the paper. This contradicts findings of the other scientific papers listed here.
The Polar H10 is very popular and often entitled as the best and most accurate heart rate strap. But there are no scientific ECG studies available, which proofs this statement except for the SFISM paper. There is a scientific study which compares the Polar H7 to an ECG device, but not the Polar H10.
Rob ter Host cannot convince us with his choice of the Polar H10 as a reference device for his studies. Rob uses two Polar H10 chest straps. Two chest straps might be more accurate than one chest strap, but he does not use the average of two Polar H10 chest straps. We regret that Rob ter Horst does not use a state of the art ECG device as reference device for his studies, which can be tuned to reach maximal accuracy, as described in this review here later on.
The paper «RR interval signal quality of a heart rate monitor and an ECG Holter at rest and during exercise» which determined the signal quality of heart rate sensors showed the following results:
1. Accuracy of the AR12plus Holter monitor ECG is 94.6 %
2. Accuracy of the Polar H10 is 99.63 %
But these data are about the signal quality of the heart rate sensors, which cannot be compared to papers which measure accuracy compared to a reference device. The Adinstruments Equivital has noise and movement artifact-free ECG, but only 2 channel ECG. Rob ter Horst. PhD argues, that the Polar H10 is also an ECG device with two ECG electrodes, which can be used for scientific studies The Polar H10 was used in several scientific validation studies of consumer grade sports sensors as reference device. In the paper fo the SFISM, it says: «A simple chest strap such as the Polar H10 might be recommended as the gold standard for RR interval assessments if intense activities with strong body movements are investigated».
Gold standard is not a scientifically clear defined term. The term gold standard was used in a validation paper of Bitalino:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6498399/
Biopac uses the term «gold standard» for their Biopac Student Lab as «The Gold Standard for Life Science Instruction»:
Biopac Student Lab was used by Bitalino for their valdiation of Bitalino. Biopac devices are «Gold Standard» also as far as costs of their devices is concerned, which starts from 4000 USD. But the term gold standard says nothing about the accuracy of the devices, except that the accuracy is high. The Polar H10 is used by DCRainmaker, 5K Runner, Rob ter Horst, in one paper of the SFISM, and in several other papers as reference device for accuracy studies. But is it the most accurate reference device?
Rob ter Horst. PhD. argues, the Polar H10 has the «best heart rate», which can be used for scientific studies. But if the Polar H10 would be «the best» device as far as accuracy is concerned, then clinicians would use the Polar H10 in the hospital for their patients, and research scientists would use the device in their laboratories, but they don t. Clinicians and research scientists often use ECG devices with a sampling rate of 16 bit or higher with 3 electrodes.
In a previous study, the group of Cardiologist Milind Desai showed that the accuracy of the Polar H7 is 98% compared to a Quinton Q-tel RMS device. A second paper had an accuracy of 99.6% for the Polar H7 compared to an ECG device. In the discussed paper of the SFISM, the signal quality of the Polar H10 was determined to be 99.63 %. It is hard to believe that the accuracy has risen from 98% for the Polar H7 to 99.63 % for the Polar H10. Is the Polar H10 so much more accurate than the Polar H7? Where new motion artifact removal algorithms used in the Polar H10? The values of the SFISM paper cannot be compared directly to papers which use ECG as a reference, since the SFISM paper determined signal quality. The goal of the following paper is to review these questions and some others concerning accuracy studies of consumer grade heart rate monitors.