Why Humans, Not Algorithms, Must Keep Authority Over Ethical Decisions

Though the prospect of moral certainty may seem appealing, ceding ethical decision-making to an algorithm would in reality be both practically dubious and philosophically pointless.

Gottfried Wilhelm Leibniz was, in many ways, the philosopher of the 21st century. He was pivotal to the invention of calculus, binary arithmetic, and statistics, and his work on organizing information logically undergirds the science that makes computers possible. He was able to see the world this way because he believed the universe was fundamentally rational, much as many modern physicists do when they contend that a single theory could describe all physical phenomena within one coherent framework.

For Leibniz, morality was a matter of debate only because we lacked a sufficiently developed ethical theory. Just as mathematical modeling of the solar system eliminated any need for debate about when eclipses would occur, a young Leibniz dreamed that one day ethicists would resolve their disputes by simply calculating the correct answer.1

It's an especially intriguing idea today. With modern computing, could we arrive at an ethical theory that solves every problem? If so, an app on your phone could tell you what to do, no matter the circumstances. Nobody would ever have to do wrong from a lack of knowledge again. 

Say an AI tool was released that made recommendations aligning with your moral intuition. You could type up a brief description of your problem and get a reliable report on what you should do. Would it be desirable to use such a system to determine your actions?

Though the prospect of moral certainty may seem appealing, ceding ethical decision-making to an algorithm would in reality be both practically dubious and philosophically pointless. Practically, it would put ethics in the hands of a machine that is capable of learning in only limited ways, and which is vulnerable to biases it can’t correct. Philosophically, we gain a lot from contemplating our actions through an ethical lens; the right outcome is only part of the value. If we give too much authority to the machines, we’ll lose our stake in ourselves.

The Same Mistakes, Again and Again

The concept of algorithmic bias has become well-known in recent years, and for good reason. All algorithms, including AI models, absorb the unspoken assumptions of their training data.2 If they are trained on the work of people who are prejudiced against certain races or genders, that bias will make its way into their outputs. This can be very subtle. If, for example, I asked a model for a list of medical causes I should donate to, it would be less likely to mention problems that were already under-reported because they affected marginalized communities. Issues like sickle cell disease in the Black community, or polycystic ovary syndrome among women, would get less attention in the future simply because they got less attention in the past.

This is compounded by the fact that, unlike humans, models are often incapable of correcting themselves without system-wide retraining. If you’ve used LLMs with any frequency, you’ve likely encountered a situation where the chatbot makes a mistake, acknowledges it when prompted, and then fails to correct itself the next time the same problem arises. It miscalculates a sum, say, and then uses the incorrect number in a later conversation. Likewise, you can tell an LLM to prioritize the needs of underrepresented groups, but that does not mean it will be capable of actually changing its approach.

A human, on the other hand, can at least try to undo their biases, and be relieved of their responsibilities if they fail. An internal training manual created by IBM in the 1970s contained a line that has become a touchstone for AI ethicists: “A computer can never be held accountable, therefore a computer must never make a management decision.”3 Machines do not have the human capability to care for things. They don’t live within the world like we do. When they’re not in use, they can be turned off or lie idle without being forced to contend with the state of the world.

For German philosopher Martin Heidegger, this sort of care is at the core of what it means to be human.4 We don’t have the option to shut ourselves off; even the prospect of ending our own lives is attached to serious consideration. It is because of this that morals have bearing on us. We act ethically in part because we care for things outside of ourselves. Though we often make logical arguments about what’s right (which LLMs can emulate with a high degree of accuracy), the core of our ethical sense is not reason. We think ethically because we are forced to care before we’ve deliberated on anything at all.

It is this relation that computers can’t understand. When an LLM produces a moral recommendation, it does so without anything at stake. It has no future it is invested in, no relationships that could be damaged by getting things wrong, no anxiety about its own conduct. Its output may be correct, but its correctness is of the same order as a calculator's: produced without comprehension and without consequence to the producer. This is precisely why the IBM formulation has held up so well. Accountability requires a subject for whom something is at stake, and no such subject exists inside a machine.

Outsourcing Our Selves

More importantly, however, handing over our power of moral reasoning to the machines would undermine the point of morality. Ethics are both a guide to making the world better and a guide for living. When we think about which actions are better and worse, we implicitly think about which are better for ourselves. It’s better to be honest than dishonest, not only because honesty is likely to produce more happiness in the world (perhaps debatable), but also because honesty clears our minds of burdensome contradiction and allows us to engage with others more genuinely. All else being equal, the world is better if you are honest.

What’s more, thinking ethically binds us to the world. When we deliberate about what we owe to others, we are not just solving a problem—we are rehearsing our relationships, testing our commitments, and discovering what we actually value by watching ourselves reason about it.

The philosopher Alasdair MacIntyre has argued that modern society is unique in the way it sees doing good and doing well as two contradictory ends, which we are nonetheless compelled to pursue simultaneously. He writes: “Moral rules and the goals of human life have become divorced to such a degree that it appears both that the connection between abiding by the rules and achieving the goals is merely a contingent one and that, if this is so, it is intolerable.”5 The force of ethics, the reason for us to make use of them, is diminished by our insistence that ethics should be impersonal, and need not offer us any personal satisfaction. Handing the keys to machines would only worsen this.

A recommendation produced by an algorithm is, by its nature, impersonal. It arrives from outside your experience, derived from training data that has nothing to do with your particular life. Following it would be moral behavior of a peculiarly hollow kind: correct in its conclusions, perhaps, but disconnected from the process by which we normally come to understand why something is right, and what that rightness has to do with us. MacIntyre's concern is that we have already made ethics feel like a constraint imposed from without, a set of rules that limit our pursuit of happiness rather than contributing to it. AI moral guidance would be the logical endpoint of that tendency—ethics fully externalized, experienced purely as instruction.

There’s another problem with moral outsourcing, which is also, perhaps, the fatal flaw in Leibniz’s calculus. The only way to convince someone that an ethical proposition is correct is to make them feel it, and logical arguments don't usually help with that. Mathematicians and logicians argue with one another despite the supposed certainty of their subjects—including, famously, Leibniz and Newton, who invented calculus simultaneously and spent years disputing priority. A moral calculus of the type Leibniz describes would likely produce humans arguing about the correct way to represent a situation in logical terms, with competing frameworks each claiming to have captured the true structure of the moral universe.

Moral reasoning should be something we do for ourselves as much as for others. Working through a difficult decision, sitting with competing obligations, arriving at a position we can actually inhabit—these are not inefficiencies that a better tool could eliminate. They are how we figure out what we care about. Leibniz wanted to end that process. We should be glad he failed.

References

  1. Rucker, R. (2013). An Incompleteness Theorem for the Natural World. In Irreducibility and Computational Equivalence : 10 Years After Wolfram’s A New Kind of Science (Vol. 2). https://doi.org/10.1007/978-3-642-35482-3_14
  2. William S. Hein & Co, Inc HeinOnline United Nations Law Collection. (2018). Algorithmic Bias and the Weaponization of Increasingly Autonomous Technologies. New York: United Nations,. http://proxy.library.nd.edu/login?url=https://heinonline.org/HOL/Page?handle=hein.unl/algbwpi0001&collection=unl
  3. Bonderud, D. (n.d.). AI decision-making: Where do businesses draw the line?. IBM.Com. Retrieved https://www.ibm.com/think/insights/ai-decision-making-where-do-businesses-draw-the-line
  4. Heidegger, M., & Schmidt, D. J. (2010). Being and time (; J. Stambaugh, Trans.). State University of New York Press ; Excelsior ; University Presses Marketing [distributor].
  5. MacIntyre, A. C. (1998). A short history of ethics : a history of moral philosophy from the Homeric age to the twentieth century (2nd ed). Routledge Classics. https://archive.org/details/shorthistoryofet0000maci

About us

We are the leading applied research & innovation consultancy

Our insights are leveraged by the most ambitious organizations

Image

I was blown away with their application and translation of behavioral science into practice. They took a very complex ecosystem and created a series of interventions using an innovative mix of the latest research and creative client co-creation. I was so impressed at the final product they created, which was hugely comprehensive despite the large scope of the client being of the world's most far-reaching and best known consumer brands. I'm excited to see what we can create together in the future.

Heather McKee

BEHAVIORAL SCIENTIST

GLOBAL COFFEEHOUSE CHAIN PROJECT

OUR CLIENT SUCCESS

$0M

Annual Revenue Increase

By launching a behavioral science practice at the core of the organization, we helped one of the largest insurers in North America realize $30M increase in annual revenue.

0%

Increase in Monthly Users

By redesigning North America's first national digital platform for mental health, we achieved a 52% lift in monthly users and an 83% improvement on clinical assessment.

0%

Reduction In Design Time

By designing a new process and getting buy-in from the C-Suite team, we helped one of the largest smartphone manufacturers in the world reduce software design time by 75%.

0%

Reduction in Client Drop-Off

By implementing targeted nudges based on proactive interventions, we reduced drop-off rates for 450,000 clients belonging to USA's oldest debt consolidation organizations by 46%

Read Next

Notes illustration

Eager to learn about how behavioral science can help your organization?