Source: Astral Codex Ten
by Scott Alexander
“In 2025, Owain Evans et al published a paper on ’emergent misalignment’. They trained a previously-aligned AI to do one immoral thing: write insecure code full of vulnerabilities and backdoors. To their surprise, the AI became immoral in general. Its advice to a bored user was to try taking random expired medications and see what happened. Its money-making tips all involved theft and violence. When asked for its favorite historic figure, it chose Hitler. … This sounds bad, in that random things can turn AIs evil or sexist. But some people in AI safety (including Eliezer Yudkowsky) speculated that in fact it was very, very good.” (09/22/26)
https://www.astralcodexten.com/p/mysteries-of-ai-generalization