Charlie O’Neill

Writing

Essays, mostly about training language models; sometimes about books and cities.

2026
The final form of work
In a post-AGI world, we solve the meta-problem of which problems to solve
Let's bet on this shit
How to act rationally if you believe in RL scaling and envs
Trying to actually define continual learning
Continual learning is not one problem
The Michael Scott Paper Company of AI
I just have to wait out you
Continual learning and the post-monolith AI era
The cost of continual learning scales with generality
San Francisco
Ithaka, California
Rationalism
It will drive you mad (and it's boring)
Books I read (am reading) in 2026
I'm going to be better this year now I'm not a founder
2025
The bitter lesson of LLM evals
Scaling care beats scaling compute
Five hinge questions that decide whether AGI is five years away or twenty
For people who care about falsifiable stakes rather than vibes
Born on third base
The Case for Inheriting Nothing and Building Everything
One-dimensional vs multi-dimensional features in interpretability
How to stop conflating with 768 dimensions
LLMs are really good at k-order thinking (where k is even)
You still need to tell a language model you want to cure cancer before it can help you cure cancer.
Information bounds in quantum gravity
How information theory links quantum mechanics and general relativity
2024
The nihilism of NeurIPS
Some thoughts on our future in AI research
Can quantised autoencoders find and interpret circuits in language models?
Using VQ-VAEs and categorical decision trees to do automatic circuit identification in LLMs.
Learning compressed representations and GPT-5 speculation
Why language models probably get too much from the abstraction we give them for free
2023
What is the future of Rake?
Something I sent to Kirks when we were building