Source: Astral Codex Ten
by Scott Alexander
“In The Argument, Kelsey Piper gives a good description of the ways that AIs are more than just ‘next-token predictors’ or ‘stochastic parrots’ — for example, they also use fine-tuning and RLHF. But commenters, while appreciating the subtleties she introduces, object that they’re still just extra layers on top of a machine that basically runs on next-token prediction. … I want to approach this from a different direction. I think overemphasizing next-token prediction is a confusion of levels. On the levels where AI is a next-token predictor, you are also a next-token (technically: next-sense-datum) predictor. On the levels where you’re not a next-token predictor, AI isn’t one either.” (02/26/26)
https://www.astralcodexten.com/p/next-token-predictor-is-an-ais-job