Whose Interests Is Your AI Really Serving?
The more useful an AI becomes, the more context we give it. That makes trust about more than privacy. It is also about whose interests the system is actually designed to serve.
I was listening to a recent episode of The a16z Show with Josh Elman about what makes consumer AI products stick.
There was a lot in it that resonated with how we are building Primed, but one point in particular stood out: trust.
The more useful an AI becomes, the more context we tend to give it. For an AI coach like Primed, that can mean years of training history, sleep, recovery, heart rate, power, activities, goals and conversations. Potentially, it becomes a remarkably detailed picture of you as an athlete.
That creates an interesting bargain.
If you trust an AI with more of your data, it should be able to do more for you. But that only works if you believe the AI is actually working for you.
Key takeaways
- Useful AI needs context, and context requires trust.
- Athlete data should serve the athlete first.
- Business incentives should not quietly distort recommendations.
- Trust has to be earned through product behaviour.
Whose interests is the AI really serving?
I think this is going to become one of the defining questions for consumer AI.
Privacy matters, obviously. But privacy is only part of it. A system can protect your data reasonably well and still use what it knows about you in ways that are better for the company than for you.
For Primed, I want the answer to be very simple: the athlete.
We should not use what we learn about an athlete to manipulate their behaviour for our benefit. We should not optimise recommendations around what makes Primed more money. And we should not make it deliberately difficult for athletes to take their data elsewhere.
The commercial incentive should be much simpler than that: build something useful enough that people choose to keep paying for it.
AI changes the trust relationship
This distinction becomes more important as AI moves from answering questions to becoming an agent that knows your history, understands your goals and increasingly acts on your behalf.
A conventional fitness app might know today's workout and a handful of metrics. A long-term AI coach could eventually understand how you respond to different training loads, what happens when your sleep deteriorates, which sessions you tend to avoid, how close you are to an event, and how your goals have changed over time.
That context is what can make the product genuinely useful. It is also why the relationship has to be different.
The more a system knows about you, the more important it becomes that its incentives are aligned with yours.
Trust is not a feature
You cannot really put trust in a feature list.
It comes from the accumulation of small product decisions: what data you collect, what you retain, how clearly you explain recommendations, whether you make uncertainty visible, what happens when commercial incentives conflict with the user's interests, and whether people remain in control of their own information.
For an AI coach, that includes being willing to recommend less when less is the right answer. It means not inventing urgency where none exists. It means not turning engagement into the goal.
The goal should be helping the athlete make better decisions.
If we are asking athletes to trust Primed with more of themselves, we have to keep earning the right to that trust.
The episode is What Makes a Consumer AI Product Stick? with Josh Elman on The a16z Show. It is well worth a listen.
