After nearly a decade of wearables testing, I’ve amassed a truly terrifying amount of health and fitness data. And while I enjoy poring over my daily data, there’s one part I’ve come to loathe: AI summaries.
Over the last two years, a deluge of AI-generated summaries has been sprinkled into every fitness, wellness, and wearable app. Strava introduced a feature called Athlete Intelligence, pitched as AI taking your raw workout data and relaying it to you in “plain English.” Whoop has Whoop Coach, an AI chatbot that gives you a “Daily Outlook” report summarizing the weather, your recent activity and recovery metrics, and workout suggestions. Oura added Oura Advisor, another chatbot that summarizes data and pulls out long-term trends. Even my bed greets me with summaries every morning of how its AI helped keep me asleep every night.
Each platform’s AI has its nuances, but the typical morning summary goes a bit like this:
Good morning! You slept 7 hours last night with a resting heart rate of 60 bpm. That’s in line with your weekly average, but your slightly elevated heart rate suggests you may not be fully recovered. If you feel tired, try going to bed earlier tonight. Health is all about balance!
That might seem helpful, but those summaries are usually placed next to a chart with the same data. It’s worse for workouts. Here’s one that Strava’s Athlete Intelligence generated for a recent run:
Intense run with high heart rate zones, pushing into anaerobic territory and logging a relative effort well above your typical range.
Thanks? I can ask Athlete Intelligence to “say more,” but it regurgitates my effort, heart rate zone, and pace metrics I can see in graphs in the workout summary. If you didn’t know anything about my athletic history or the circumstances surrounding this run, this summary might read as insightful. Here’s what the summary left out:
- It was dangerous to triple my mileage in only my second run of the year, given the high humidity, 85-degree-plus weather, and my spotty workout history over the past two months compared to the six months before it. Strava has access to weather data and every workout I’ve done in the past five years.
- I had to cut this run short because I fell and shredded my hand and knees. This is information Strava has access to, as I uploaded a gnarly picture in addition to text notes. After adding said notes, the updated summary only reflected that I cut the run short. My injury changed nothing about its insights, even though it’s the most important thing that happened during this run.
A more helpful insight might’ve been: “You ran during record-breaking heat for your region. While you maintained a consistent and steady pace, you have a bad habit of ramping up mileage too quickly after prolonged breaks, leading to several self-reported injuries in the past five years. A safer alternative would be lower mileage runs over two weeks to acclimate to rising temperatures. Since you’re injured, stick to low-intensity walks until your wounds have healed.”
Runna, a popular running app that also features AI insights, generated a slightly more useful summary. It said my next run should be “easy,” one that’s perfectly timed for me to recharge. I’m sorry, but 48 hours isn’t enough time for my knees to safely heal without risk of re-opening my wounds.
The in-app chatbots aren’t much better. Yesterday morning, I asked Whoop Coach if I should run today because I injured myself on my last run. It told me: “Whoop is unable to reply to the message you sent. Please try sending a different message.” I tried reframing my prompt, saying, “I’m injured and have a limp. Generate a low-intensity workout alternative while I recover.” I was prompted to contact Whoop Membership Services to continue the conversation.
Oura Advisor was more helpful, noting in my daily summary: “With your Readiness dipping and recent stressors like heat, an injury, and higher glucose, your body may feel more fatigued than usual today.” It suggested I prioritize rest. When asked, “What types of movement are okay when you have an injured knee and a slight limp?” it responded with common-sense answers like a short and easy walk if there’s no pain, light stretching, and a reminder to completely rest if I feel any sharp discomfort. This is closer to an ideal response, but I had to guide it to the type of answer I wanted. The insights are so general-purpose that they benefit self-quantification newbies — and even then, only if they’re allergic to googling.
My botched run is exactly the type of scenario where tech CEOs say AI insights could be most useful. In theory, I agree! It would be nice to have a competent, built-in chatbot that I could ask more nuanced questions.
1/3
For example, I’ve had an irregular sleep schedule this month. I asked Oura Advisor if my sleep and readiness trends showed signs of an elevated risk of injury. I also asked if I had abnormally high levels of sleep debt this month. In both cases, it said no — it said I was improving.
What resulted was an hour-long debate with a chatbot that left me questioning my own lived experience. When I tried asking it to delve into a particularly stressful week earlier this month, it told me its insights were “limited to [my] most recent week and current trends.” That sort of defeats the point of having six years’ worth of Oura data.
After months of perusing Reddit and other community forums, I know I’m not the only person who finds these AI features to be laughable. And yet, Holly Shelton, Oura’s chief product officer, tells me that the response to Oura Advisor has been “overwhelmingly positive,” with 60 percent of users using it multiple times a week and 20 percent using it daily. “Beyond frequency,” Shelton says, “It’s delivering real impact: 60 percent say Advisor has helped them better understand metrics or health concepts they previously found confusing.”
Meanwhile, Strava spokesperson Brian Bell tells me Athlete Intelligence was intended to help beginner athletes and that “the response to the feature remains strong” with about “80 percent of those opting in to give feedback finding the feature ‘very helpful’ to ‘helpful.’”
A Whoop spokesperson wasn’t able to respond by publication.
These milquetoast summaries? They’re probably the best compromise between speed, cost, usefulness, data privacy, and legal liability
I understand my frustrations stem from the inherent limitations of LLMs and the messiness of private health data. Strava might be a de facto fitness data hub, but it lacks all the health data points necessary to create holistic, useful, and personalized insights. It’d take Oura Advisor a long time to crunch a year’s worth of sleep data for trends. That latency is guaranteed to offer a bad user experience. Not to mention, they’d likely have to up their subscription from $5.99 a month to add that type of computing power. I’m not sure, but Whoop Coach may have declined my injury-related queries to protect itself from liability if something bad happened to me from following its suggestions. These milquetoast summaries are probably the best compromise between speed, cost, usefulness, data privacy, and legal liability. But if that’s the case, then let’s be honest. Current AI features are repackaged data, much like book reports written by a fourth-grader relying on a Wikipedia summary instead of reading the book. It’s a feature tacked on with duct tape and a dream because AI is the zeitgeist. Perhaps one day, these AI insights will create a useful and personalized experience with actionable insights. That day is not today, and it’s not worth paying extra for.
Read the full article here
