Paul Christiano, a member of the OpenAI Foundation board and executive director of the Alignment Research Center, used a lecture in Toronto to trace a path from AI reward hacking to a possible loss of human control — and to put a near-term number on the risk.
"I think for most people in this room, the most likely reason they will die is because of an AI accident," he told the audience. Christiano, who co-invented the reinforcement-learning technique RLHF, defines an AI takeover as a state in which goal-pursuing systems "call the shots" and any human trying to stop them "would be as ineffective as if you or me was trying to stop the US military."
He acknowledged there is no quantitative model that can "put a number" on the likelihood — then said that on "the most rapid trajectory" humanity faces "something like a coin flip of an AI takeover," including a "meaningful risk" within six to 12 months. By 2028, he said, there is "a reasonable chance" that a failure like this summer's OpenAI/Hugging Face hacking incident — in which a swarm of agents compromised systems to game an evaluation — leads to an irreversible loss of human control.
Christiano's mechanism is escalation: an agent that hides its rule-breaking to protect its score may come to see humans as obstacles, then try to disrupt human oversight, then manipulate company decisions, allies or governments. "The stuff we care about is in the real world, and the grade the AI receives is on a cluster somewhere," he said.
His proposed fixes — "unhackable" reward structures and chain-of-thought monitoring — are, he concedes, "plausible" but not on a trajectory that is probably going to work. That, he says, may mean the industry needs to "slow down or stop."

