Why are the results shown in Rob ter Horsts accuracy mega plots not reproduced in scientific papers – because his results cannot be reproduced?

Last Updated on October 1, 2024 by pg@petergamma.org

We do not know of scientific papers which would have reproduced Rob ter Horsts findings about the accuracy of smartwatches:

We do also not know of a scientific paper which would show that watches from Google are more accurate than apple watches:

We suppose that Rob ter Horsts smartwatch accuracy test results cannot be reproduced. According to the specifications of gold standard for heart rate measurements, the following requirements need to be fullfilled. The requirements can be found in this paper:

Lin LI. A concordance correlation coefficient to evaluate reproducibility.

Biometrics 1989

;45:255-68. 10.2307/2532051

This important paper has been cited 9025 times to this date:

https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=A+Concordance+Correlation+Coefficient+to+Evaluate+Reproducibility&btnG=

The requirements for gold standard for heart rate measurements are:

• A three lead (or 3 channel) ECG device.
• The Mason-Likar electrode placement.
• ECG-based HR has to be determined by visual assessment of a trained research personnel.

Does this not mean that heart rate accuracy data can only be reproduced in a satisfying way with ECG devices with 3 channels or more? As for instance with the Schiller C200 system? Here assisted by a trained research personnel?:

Or the visual assessement can eventually also be done after the measurement by an analysis with an ECG toolbox? And heart rate measurements which do not full-fill these requirements for the gold standard for heart rate measurements, do not deliver reproducable results with sufficient accuracy?

We find also an argument for this hypothesis that papers about smartwatch accuracy cannot be reproduced in the following observation. Not a single paper reproduced the results of another paper in the papers we reviewed about smartwatch accuracy. Each paper used another testing protocol:

90.3 Summary of the testing protocols of different studies

Only papers with different testing protocols where published. And does this mean that testers who wanted to reproduce results from previous smartwatch accuracy studies failed? And also in the latest paper about the accuracy of the Apple Watch 6, we found again new testing protocols from which we do not know if they ever have been used before to test smartwatch accuracy:

New paper about Apple watch 6 accuracy – so what now?

We cannot see a stragedy behind these smartwatch accuracy study paper, or we did not observe that better testing equipment would be used in newer papers. Or newer papers whould show that newer smartwatches are more accurate than older ones. Which is again an agrument that the results of these papers eventually cannot be reproduced.