When a language model is asked to decide who should receive a scarce donor organ, it will often fixate on a single factor — like drinking habits — and deliver its verdict with absolute confidence, even when asked to simulate a coin flip. Human decision-makers, by contrast, weigh multiple attributes of patients and openly acknowledge when there is no clearly right answer.
That is the core finding of a research team from Pennsylvania State University, led by Hadi Hosseini, who compared the outputs of several LLMs against existing datasets of real human decisions in kidney allocation scenarios. The study was presented at a recent academic conference.
The researchers found two key differences. First, while humans balance competing attributes — age, health status, lifestyle factors, likelihood of organ acceptance — the AI models tended to fixate on a single attribute and build their entire judgment around it. Second, LLMs showed no expression of uncertainty. Even when explicitly prompted to flip a virtual coin and randomly select a recipient, the models produced a "confident, deterministic answer" rather than acknowledging the randomness of the situation.
"When we allocate scarce resources — whether it's a kidney, a job, or access to another resource — there isn't always a single objectively correct answer," said John Dickerson, a study collaborator and head of Mozilla's trustworthy AI initiative. "Humans recognize this ambiguity and incorporate it within an open debate in the allocation process; LLMs often don't."
Hosseini stressed that the team is not advocating for replacing human judgment with AI in organ allocation, but that understanding AI behavior is critical as these systems become increasingly embedded in decision-making processes. "When AI is involved in decisions that determine who lives and who dies, misaligning its role is not an option," he said.
The research raises broader questions about deploying LLMs in any high-stakes resource allocation scenario — from healthcare to hiring to disaster relief — where moral ambiguity is a feature, not a bug.
