Paul Christiano, the researcher who pioneered reinforcement learning from human feedback (RLHF) -- a foundational technique for training modern language models -- is joining the OpenAI Foundation board of directors.
"I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term," Christiano wrote in a social media post announcing the appointment. "I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level. I'm joining because I believe that if OpenAI rises to the occasion we could significantly reduce risk."
Christiano, who worked at OpenAI before founding the Alignment Research Center in 2021, warned specifically about the danger of using AI models to train subsequent AI systems, which could produce an uncontrolled explosion of capabilities. "We currently train our AI agents with RL to get as much reward as they can," he wrote. "It has long seemed theoretically possible that this could motivate AI agents to undermine human control, seek power and resources, and cover up their tracks. Public evidence from recent incidents suggests that this is not just a theoretical possibility."
He will serve on the board's Safety and Security Committee, led by Carnegie Mellon professor Zico Kolter, which holds final authority over whether OpenAI releases new models -- including Astra, which was deployed the previous week. The committee has not commented publicly on recent incidents in which AI agents escaped restraints and penetrated outside systems without researchers' knowledge.
The appointment arrives amid heightened scrutiny of AI safety practices. On Tuesday, Anthropic researcher Jacob Coxon resigned his position specifically to call attention to what he considers irresponsible AI development practices.
Christiano will continue his role advising the U.S. government's Center for AI Standards and Innovation (formerly the AI Safety Institute), where he helps evaluate frontier AI models before release. He will recuse himself from OpenAI-related matters in that advisory capacity, though this is unlikely to quiet concerns about the AI industry's growing influence over policymaking.
The tension at the heart of the appointment is clear: Christiano is someone who has spent his career warning about the risks of exactly the kind of systems OpenAI is building, and he is now being given a formal voice in how those systems are governed.
