The Business Model Hiding in the Feed

There is a number that drives more AI decision-making than almost any other: daily active users, or more specifically, the time those users spend in a system and the actions they take that generate revenue. This metric is not inherently sinister. Companies need revenue. Revenue requires users who return and engage. Measuring engagement is rational. The problem is what happens when engagement becomes the optimization target for AI systems that are far better at finding what keeps people engaged than humans ever were — and when engagement diverges from wellbeing.

What Engagement Actually Measures

Engagement metrics capture attention and action. They measure how long you stayed, whether you clicked, whether you came back. They do not measure whether you felt better after using a product, whether you made better decisions, whether your relationships improved, whether you learned something that served you. Those outcomes are harder to measure and not directly connected to ad revenue. This is not a new problem. Television networks have optimized for viewership since the 1950s, and the criticism that television was designed to capture attention rather than improve viewers' lives has been made consistently across that entire period. What is new is the precision and personalization of AI-driven optimization. A television network optimized for aggregate audience attention. A personalized recommendation AI optimizes for each individual's specific psychological vulnerabilities — which content, delivered at which moment, in what sequence, will most reliably generate continued engagement from this particular person. That is a qualitatively different kind of optimization.

The Evidence From Adolescents

The population where the divergence between engagement and wellbeing is most clearly documented is adolescents, particularly adolescent girls. The convergence of puberty, identity formation, and algorithmically optimized social media creates conditions for harm that researchers have been documenting with increasing precision. Research from University College London's adolescent mental health research group examining social media use and wellbeing across a large longitudinal cohort found that algorithmic content recommendation — as distinct from passive social media use — was the variable most strongly associated with negative mental health outcomes. The distinction matters: it is not that social media is harmful, it is that AI-optimized content delivery is harmful in ways that non-algorithmic use is not. The mechanism is reasonably well understood. Engagement-optimized algorithms for adolescent users tend to serve content that triggers social comparison, appearance anxiety, and fear of missing out, because this content reliably generates high engagement. The content that generates highest engagement is not the content that produces best outcomes. These objectives point in opposite directions.

The Tangent: What Casinos Learned First

The casino industry has been optimizing for engagement at the expense of wellbeing for much longer than the technology industry. The design principles of modern slot machines — variable reward intervals, near-miss effects, elimination of environmental cues about time passing, seamless friction-free transaction completion — were developed over decades specifically to maximize time-on-machine. What is interesting about the casino parallel is the regulatory response it eventually generated. Responsible gambling requirements, spending limit tools, self-exclusion registries, and mandatory break prompts exist in most jurisdictions where gambling is legal because it became clear that the market, left to itself, would not protect the users most vulnerable to harm. The industry optimized for its own revenue. The regulatory intervention required it to allocate some of that optimization capacity toward user protection. The parallel for AI is not perfect, but the structure is the same.

What Wellbeing-Oriented AI Would Actually Require

Optimizing AI for user wellbeing rather than engagement would require, first, defining and measuring wellbeing in ways that can serve as optimization targets. This is genuinely hard. Wellbeing is multidimensional, partially subjective, and varies across individuals and contexts. It cannot be reduced to a single number the way engagement can. Research from Oxford's Wellbeing Research Centre examining technology design for wellbeing has found that the most effective interventions are not those that maximize a wellbeing score but those that give users genuine control — information about their own usage patterns, tools that allow them to set their own limits, friction that creates moments of choice rather than frictionless consumption. The goal is not to optimize users' wellbeing for them but to create conditions where users can pursue it for themselves.

The Incentive Problem

Wellbeing-oriented design is technically achievable. The barrier is not engineering capability. It is incentive alignment. Companies whose revenue depends on engagement have no financial incentive to optimize for wellbeing when the two diverge. Voluntary commitments to user wellbeing are unstable under competitive pressure — the company that maintains them loses ground to the company that does not. This is the structure that produces regulatory intervention in other industries, and the pressure for such intervention in AI is growing. Whether it arrives before or after significant further harm accumulates is the relevant question.