Why Apple Watch Will Not Be Accurate on Stress Readings

With the imminent launch of the Apple Watch it is clear that everyone is looking to it for new functionality and inevitably there will be claims you can reduce stress using it. I wanted to look into whether the device could reliably deliver on that promise.

Apple Watch Will Not Be Accurate

The watch reportedly can read heart rate by taking pulse readings from the wrist using pulse oximetry, a method that uses light pulses to read heart beats by measuring the change in skin color due to different levels of blood flow. Pulse oximetry is refined enough for reading heart rate, but Heart Rate Variability (HRV) demands precision that pulse oximetry reportedly cannot deliver.

I am interested in using HRV to improve personal performance in working sessions, face to face meetings, negotiations and public speaking. I thought it would be interesting to be able to use an Apple Watch to read HRV and improve those skills. So I wanted to test pulse oximetry myself.
Apple Watch Will Not Be Accurate

I don’t have an Apple Watch, so I connected a Mio Velo wrist worn band to the SweetbeatLife app on an iPhone. The Mio product claims to deliver “EKG-accurate heart rate data” and uses pulse oximetry, so this would be my proxy for an Apple Watch.

Apple Watch Will Not Be Accurate

To compare this setup with readings from electrical signals I connected a Polar H7 to Marco Altini’s Heart Rate Variability Logger on an iPod Touch. This would allow me to take two readings of a single heart beat and compare the methods. After wrangling settings and conflicting signalling I got them both to work.

My goal was to use my Parasympathetic Flatline method when comparing the pulse oximetry with electrical readings using a heart rate belt. This means I am looking for 10 consecutive heart beat intervals that vary less than 17 milliseconds from beat to beat. When I find these strings of beats I am measuring myself in a fight/flight state.

Researching pulse oximetry I found a research paper that said that physical movement introduced errors in readings making pulse oximetry unreliable for measuring HRV when subjects were in motion. The conclusion was that pulse oximetry “is unlikely to prove a practical alternative to the ECG in ambulatory recordings or recordings made during other activities.”

Apple Watch Will Not Be Accurate

My interest is looking at activities like negotiating, meeting and coding where there is not a lot of physical movement. With the study providing some potential for pulse oximetry to provide some value to my area of interest, it seemed reasonable that readings when relaxed would be similar and when walking very different.

I conducted sessions in a relaxed state, working by myself on the computer, in meetings and while walking. I first conducted the dual measurements while in a relaxed state for ten minutes. I sat and did not move and breathed in an even rhythm. Subjectively I think I was in fight/flight for 25% of the time because sitting motionless allowed me to think about all the things I was not getting done. Here is a graph of the two readings:

Apple Watch Will Not Be Accurate

The pulse oximetry reading was that I was in fight/flight 87% of the time and that is way overstated. The P7 said 32% and that was much closer to my experience. So the relaxed state had a completely different outcome than my hypothesis.

Next I measured myself when I was in a working session, which meant I had structured some time to work on my computer without interruption. I was working on some recruiting matters which meant screening resumes. It was very focused work and I felt relaxed. I would have said I was 10% at most in fight/flight. My session was 16 minutes long, and here are the charts:

Apple Watch Will Not Be Accurate

The pulse oximetry reading said I was in fight/flight 57% of the time. This did not remotely match my experience. The H7 reading said I experienced no fight/flight at all. There were accelerations, but none that were more than 9 beats. So though I’m not sure it was a perfect session it was clear that the H7 more closely matched my experience.

I also took readings during the first and second half of a long staff meeting. I was not the host, I was a participant. There were some controversial things being discussed so I would have subjectively said I was in fight/flight 15% of the time. Here is the chart for the first half of the meeting:

Apple Watch Will Not Be Accurate

You can see the pulse oximetry said I was at 62% fight/flight, H7 10%. Here is the chart for the second half:

Apple Watch Will Not Be Accurate

Pulse Oximetry had me at 63% and H7 at 1%. The H7 seemed low because there were a few moments where I was definitely in a heightened state, but an average under 10% is much close to the perceived 15% than a consistent reading by the Apple Watch equivalent of over 60%. That just made no sense.

I took measurements while walking. I had low expectations because I had taken readings when exercising and know that HRV is low when physically active. Here is the chart as I took my first walk to the train from work.

Apple Watch Will Not Be Accurate

I walk briskly so I expected a 60% to 75% reading here. What you see is Oximetry at 91% and H7 at 67%. Again oximetry was high. Here is my reading for leaving the train and going to the pickup point:

Apple Watch Will Not Be Accurate

What is interesting here is my wife picked me up about a third of the way through the reading and I relaxed in the car chatting with her as she drove me home. The H7 clearly shows me moving from an accelerated state to more relaxed, which was my experience. The oximetry reading shows continued stress. Again the H7 reading matched the experience.

So what conclusion can we draw from comparing pulse oximetry as used by Apple Watch to electrical readings from chest worn heart rate belts? In the range of activity from sitting motionless to walking briskly the pulse oximetry method overstates stress.

So your Apple Watch is not the best tool to measure stress response using HRV. When you read, “Physicians and digital health experts are encouraged by the level of accuracy suggested by the Apple Watch’s sensors,” remember that pulse oximetry will overstate your stress. In a world of stress the last thing you need is to have it overstated.

Sign up for the QuantXLaFont Newsletter
Get our lifestyle tips and studies delivered to your inbox.
Thank you! We don't spam :)

3 thoughts on “Why Apple Watch Will Not Be Accurate on Stress Readings

  1. The fact that the Mio is not designed for high sampling needed for HRV does not mean the Apple Watch (or another optical sensor device) could not be…

    1. Anything could be designed for high sampling and have an algorithmic correction to ensure enough accuracy to be useful. The current state of the market is that the designers are aiming for heart rate readings, not accurate heart rate variability. I have no problem with that. I am calling out that some app and wearables marketers will claim they can measure stress using HRV when in fact the devices are not able to provide that level of accuracy reliably. Specifically, an app marketer can make these claims when the platform is not yet able to provide the underlying data needed.

  2. If HRV is based on a test while stationary surely the Apple Watch data is accurate enough? The data looks the same as my chest strap until I start running – then there’s issues. Or am I missing something?

Leave a Reply

Your email address will not be published. Required fields are marked *