090b. Rob ter Horst: issues of the scientific justification to choose the Polar H10 as a reference device for his tests

Issue 1

Rob ter Horst is a scientist who tested 100 smartwatches:

His scientific reference device is the Polar H10 chest strap:

And his scientific justification to choose the Polar H10 chest strap is this paper:

https://pubmed.ncbi.nlm.nih.gov/31004219/

But in this paper of the Cleveland Clinic in Ohio:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6732081/

The cardiologists used a 3 channel medical grade devices as a reference which costs thousands of dollars. And the scientific justification to choose this device as a reference is the following:

In the paper it says:

«

• In order to accurately assess HR in each subject to compare to the wrist worn monitors, a three lead ECG … monitor were used.
• The Mason-Likar electrode placement was used and allowed for the assessment of modified leads I, II, and III.
• The ECG was monitored on a Quinton Q-tel RMS telemetry system and ECG-based HR was determined by visual assessment by trained research personnel.

Using a 3 lead ECG in this fashion is considered the gold standard for HR measurement
«

And the scientific justification to choose this setup is the following paper:

Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989;45:255-68. 10.2307/2532051

This paper has been cited 8580 times to this date:

https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=A+Concordance+Correlation+Coefficient+to+Evaluate+Reproducibility&btnG=

Why has this paper been so often cited? It is a key paper which say something what is required so that heart rate measurement are reproducable, and these requirements are:

• a three lead (or 3 channel) ECG device
• The Mason-Likar electrode placement
• ECG-based HR has to be determined by visual assessment of a trained research personnel

These are the requirements for gold standard of heart rate measurements or the requirements that heart rate measurements are reproducable. We find hardly any PPG devices from iWorx, Biopac, Adinstruments, g.tec medical. All of these manufacturers of devices which are suitable for scientific studies use at least 3 channel ECG devices which cost several 1 000 of dollars.

These are the arguments which made us reatreat from PPG devices for our projects, until someone proof us that PPG are suitable for it. . We decided to choose 3 channel ECG devices instead. We have published in our journal that it is possible to build such ECG devices with 3 channels for much less money than as they are sold from the above manufacturers.

Issue 2

• The paper Rob ter Horst has choosen is partly financed by Polar, and ignores other papers which where done by the Cleveland Clinic in Ohio which use 3 channel ECG devices as reference devices.
• For gold standard or for HR measurements to be highly reproducable very strict limitation are there (see above).
• In the paper partly financed by Polar is as far as we know not indicated how stongly Polar influenced the setup of the paper.
• Since this paper was partly financed by Polar, we name it here as a “paid promotional paper”.
• Rob ter Horst reviewed other papers which where partly financed by manufacturers as for instance about the Dreem 2, or the Verity Sense sleep tracking.
• Where these papers designed to increase sales rate, and eventually designed to be able to increase the price for the devices, as for instance in the example of the Dreem 2 headband.
• The price for the Dreem2 headband had increased from 500 USD to 1400 USD after the paper has been released.

Issue 3

• Rob ter Horst ignores papers which use 3 channel ECG devices as a reference, as for instance those of the Cleveland Clinic in Ohio:

090. Review of Accuracy Studies of Consumer Grade Heart Rate Monitors: Methods, Testing Protocols, Reference Devices

• These ECG reference devices which where used in scientific studies have 3 or more channels and cost thousants of dollars. And there is a correlation between the costs of these devices and their accuracy:

90.02 Research & medical grade ECG devices which some of those were used to study the accuracy of consumer grade HRMs in different papers

• It is hard to believe that ECG devices which cost thousands of dollars are less accurate than the Polar H10 chest strap.
• Rob ter Horst basically claims that ECG devices are bad and the Polar H10 is the best with 99.6 % accuracy.
• What Rob ter Horst claims is only half the truth.
• After studying the papers which are currently available about smartwatches we come to the conclusion that smartwatches are only of limited use for our application, and we do not choose smartwatches for it if possible, but we choose 3 channel ECG devices instead.