Last Updated on December 23, 2022 by pg@petergamma.org
Many watches are validated for it s heart rate accuracy, some of them in scientific papers. If Garmin makes firmware updates which affect the heart rate accuracy without communicating it, this affects validations studies and papers, and makes them eventually useless.
Rob ter Horst tested the VenuSQ for heart rate accuracy, and found that it is inaccurate under certain conditions. The Garmin Venu is one of the first watches which monitors respiration rate, which makes the watch interesting for practicing breath mediation:
We thank Rob for his great review. We are surprised by the strong inaccuracy of the VenuSQ. Is it a faulty device? Garmin is in general high quality. We have tested the Garmin Venu respiration rate against the Garmin Fenix 6 respiration rate. We observed that both are correct at around 12 breath per minute, but not in other ranges. The Venu showed much stronger fluctuations than the Fenix 6. Garmin seem not to care so much about the accuracy of their wrist based devices. It is also difficult to find a paper who tests the accuracy of Garmin watch heart rate accuracy. One of the best tests about Garmin watch heart rate accuracy can be found on Rob ter Horsts channel. We currently need a Garmin watch do do some tests, and decided to buy a Venu 2.
But then, in a second test, Rob measured an increased accuracy of the VenuSQ:
But what is the cause of the increase of heart rate accuracy in the second YouTube video of Rob? Rob changed the settings of the VenuSQ before the test. In the middle of his testing, he did a firmware update. But is it a good idea to do a firmware update during a testing series?
The result is confusing. We are happy that the accuracy of the VenuSQ is now better. But is it due to changed settings or due to a firmware update? Firmware updates should be communicated in detail by the manufacturers, if they affect the accuracy. They have an influence on accuracy studies. As on Robs study. In the middle of Robs study, there was a firmware update, and Rob does not know, what exactly was updated by the firmware update. This makes it difficult to interpret the results.
The politics of these firmware updates should be improved. Polar for instance increased the running time of the Verity Sense by a firmware update, which made us retreat from the Verity Sense. If the parameters of a sensor are changed, a new validation paper is necessary. We went back to the Polar OH1, because there is a validation paper with a 3 lead Gold Standard ECG device for the Polar OH1. The reference device was a g.Nautilus ECG device from g.tec medical which costs thousands of dollars. The study was published in PLOS ONE. For the Polar OH1, there is a validation paper with a 3 lead gold standard ECG device, but not for the Verity Sense. Who wants to again and again write papers about watches which change their characteristics again and again by firmware updates?
It is a pity that Rob ter Horst testing protocols are not published. How long was the test, how large was the speed in his test? If Rob would publish his testing protocols, we could reproduce his results, and we could get numerical and statistical data, and an accuracy ranking with numerical and statistical data. It does not need to be published in a scientific paper to start with, it could also be a WIKI page to start with, where users publish their results, we prefer scientifically validated devices, though.
We thank Rob for his testings and YouTube videos, but if Garmin and Polar changes the parameters of watches which are on the market by firmware updates, this makes it difficult to write scientific validation papers about these watches.