A Simple Question With No Simple Answer

Who is AI for? The question sounds naive, but try to answer it honestly and it gets complicated fast. A company that builds and deploys an AI system has shareholders, who have a legal and contractual claim on returns. It has users, who have an implicit claim on receiving value. It has affected parties who never chose to interact with the system but whose lives it touches anyway. It has employees who built it. It has society at large, which bears whatever externalities the system produces. These interests are not always aligned. In fact, they are structurally in tension in ways that make the question "who does this serve?" genuinely difficult to answer.

The Shareholder Primacy Trap

For most of the late 20th century, the dominant theory of corporate governance held that companies exist to serve shareholders, and that serving shareholders — through profit maximization — would indirectly serve everyone else through market mechanisms. This theory has taken significant damage from evidence, but it remains deeply embedded in how large technology companies are run. The problem for AI is specific. The capabilities that maximize short-term engagement, conversion, and data collection — which serve shareholder interests in a conventional framework — are often directly opposed to the capabilities that serve user wellbeing. Recommendation systems optimized for time-on-platform are optimized for a metric that correlates negatively with user-reported life satisfaction. Personalization systems that learn to serve users emotionally resonant content learn, as a byproduct, to serve them content that produces anxiety and outrage because those emotions drive engagement. These are not design accidents. They are design choices made in contexts where shareholder returns were the primary optimization target.

What Serving Humanity Actually Requires

Genuinely serving humanity through AI would require, at minimum, treating user wellbeing as a primary metric rather than a secondary constraint. It would require building systems that users can meaningfully understand and control. It would require accounting for effects on people who are not users — communities affected by AI-driven decisions about lending, hiring, and criminal justice. Research from Harvard Business School examining long-term value creation in technology companies has found that companies that invest in genuine user welfare and manage externalities proactively tend to outperform their peers over ten-year horizons. The short-term cost of foregoing engagement-maximizing design choices is outweighed by the long-term benefit of building durable trust. The finding is robust but has not dramatically changed industry behavior, suggesting that quarterly earnings pressure consistently outweighs long-term strategic reasoning.

The Externality Problem Is Structural

An externality is a cost or benefit that falls on parties who are not part of a transaction. Markets handle externalities badly by default. A company that deploys AI that damages the epistemic environment — making it harder for people to distinguish reliable information from false information — bears a fraction of the cost of that damage while capturing most of the revenue from the deployment. This structural misalignment is the same one that drove lead paint, tobacco marketing, and decades of unchecked carbon emissions. In each case, the costs were diffuse and delayed while the benefits were concentrated and immediate. In each case, voluntary industry action was insufficient and regulatory intervention was required. Research from the London School of Economics examining the historical pattern of technology externality governance has found that industry self-regulation alone has never successfully managed large-scale externalities in technology sectors. The pattern consistently requires public intervention — either direct regulation or liability structures that internalize costs — before behavior changes at scale.

The Tangent: Fiduciary Duty as a Model

There is an existing legal concept that handles the tension between serving one party and being tempted to serve yourself: fiduciary duty. Doctors, lawyers, and financial advisors who hold special power over clients are legally bound to prioritize client interests, even when that conflicts with the professional's own financial interest. The logic of fiduciary duty applied to AI would be transformative. A system deployed to assist users in making decisions would be legally required to optimize for user outcomes, not company revenue. The conflict of interest built into current design incentives would become a legal liability rather than a business model. This framing has been proposed in academic literature on AI governance. It has not gained significant traction in legislative discussions, possibly because the companies most affected by such a framework are among the most politically active.

What the Ethical Imperative Looks Like in Practice

Ethical AI development is not primarily about avoiding the most dramatic harms. It is about building the sustained institutional capacity to ask who is being served and to give honest answers. It requires governance structures inside companies that give genuine authority to people whose job is to answer that question. It requires external accountability mechanisms that function even when internal ones fail. The companies that build this capacity will not always be the fastest to ship. They may not always post the strongest quarterly results. But they are building something more durable — systems that people can trust, which is the only sustainable foundation for the long-term value that AI genuinely offers.