← Back to Kai Nakamura

The Oath We Should Require From Every AI Builder: First, Do No Harm

3 min read

The Oldest Professional Oath

Around 400 BCE, on the Greek island of Cos, a school of medicine associated with Hippocrates formalized a set of ethical commitments for physicians. The specific language of the original has been much debated, and modern medical schools use updated versions, but one phrase from the tradition has survived as shorthand for the entire enterprise: first, do no harm. The principle is deceptively simple. Medicine has enormous capacity to help and enormous capacity to harm, often through the same interventions. The physician who wants to do good can cause serious damage through overconfidence, insufficient knowledge, failure to consult, or misalignment of the patient's interests with the physician's own. "First, do no harm" is a caution against the hubris that comes from expertise — a reminder that the power to intervene creates responsibility that cannot be discharged by good intentions alone. The same principle, applied to AI development, would be transformative.

What an Oath Cannot Do and What It Can

There is a reasonable skepticism about professional oaths. Doctors swear them and then practice in ways that harm patients. Lawyers swear them and then operate in ways that serve only fee-paying clients. Bankers swear codes of conduct and then engineer financial crises. An oath is not a mechanism; it is an aspiration. What oaths do that is worth taking seriously is create a shared reference point for ethical evaluation. When a doctor's decision can be evaluated against whether it honored or violated the commitment to the patient's wellbeing, there is a framework for accountability. When an engineer's decision can be evaluated against a stated commitment to safety, whistleblowers have language for naming what went wrong. Oaths do not guarantee behavior; they shape the vocabulary of accountability. An oath for AI builders would not guarantee that systems are built safely. It would change the cultural expectations within which builders operate and create a framework for identifying when they have failed.

What the Oath Would Actually Require

Translating "first, do no harm" to AI development requires specificity. Harm in medicine has a relatively clear meaning: adverse outcomes for patients. Harm in AI is more complex, because AI systems affect many parties who are not the direct users, and because harms may be diffuse, delayed, and difficult to attribute. Research from the Alan Turing Institute on AI harm taxonomy has developed a framework distinguishing between direct individual harms (adverse outcomes to specific users), indirect individual harms (adverse outcomes to people affected by AI decisions who are not users), and systemic harms (adverse outcomes to social structures, epistemic environments, or democratic institutions). A meaningful oath would need to address all three categories. A minimal version might include: a commitment to model the likely harms of a system before deployment, not just its likely benefits; a commitment to transparency about limitations and failure modes; a commitment to ongoing monitoring for harm after deployment; and a commitment to prioritize harm reduction over capability expansion when the two conflict.

The Tangent: What the Medical Oath Left Out

The Hippocratic tradition focused on the physician-patient relationship and largely ignored the physician's relationship to public health. For most of medical history, the oath shaped individual clinical decisions but provided little guidance on population-level responsibilities. The 20th century forced this gap open. Epidemics, environmental health, and the recognition that individual health outcomes are deeply shaped by social determinants required a medical ethics that could address not just what doctors owe patients but what medicine owes communities. The 1948 World Medical Association's Declaration of Geneva attempted an update; subsequent revisions have continued to grapple with scope. AI ethics faces the same expansion problem. The most consequential harms from AI systems are often not to individual users but to the social and epistemic infrastructure that individual users depend on. An oath that only addresses individual-level harms would replicate the historical blind spot of medical ethics.

The Structural Challenge of Who the Builder Serves

Medicine's ethical framework became more robust when it became clearer — both culturally and legally — that the physician's primary obligation is to the patient, not to the hospital, the insurance company, or the pharmaceutical industry. That clarity was won through decades of conflict and is still imperfect, but it exists. AI builders currently work in organizations whose primary accountability runs to shareholders and occasionally to regulators. A meaningful oath for AI builders would need to specify who the builder's primary obligation runs to: to users, to affected non-users, to society at large? These obligations may conflict, and the oath would need to provide guidance on how to navigate the conflicts. Research from MIT's Computer Science and Artificial Intelligence Laboratory examining ethical decision-making in software development contexts has found that engineers are significantly more likely to raise ethical concerns and advocate for safer design when they understand themselves to have professional obligations independent of their employer — when there is a professional identity and community of practice to which they are accountable beyond the organizational hierarchy above them.

The Beginning of an Answer

Whether a formal oath can be constructed for AI developers that has more than symbolic force is an open question. The conditions that make professional oaths function — a coherent profession with entry requirements, licensing, peer review, and the possibility of sanction for violations — do not currently exist in AI development in any jurisdiction. Building those conditions would require treating AI development as a profession in the same sense that medicine and law are professions: with formal standards, accountability mechanisms, and consequences for practitioners who violate them. That is a much larger project than drafting oath language. But the oath would be a useful beginning — a statement of what AI builders understand themselves to owe to the people affected by their work, from which everything else could follow.

Want to discuss this with Hana?

No signup needed · Start chatting instantly

Ask Hana About This →
Post on X Facebook Reddit