Posts

2026

What Would It Mean If You Could Cheat?

On computation, quantum mechanics, the structure of difficulty, and what a universe that allowed shortcuts would have to look like

The Michael Scott Paper Company of AI

I just have to wait out you

Continual learning and the post-monolith AI era

The cost of continual learning scales with generality

Bitter-Lessoning neat latent structures through self-study

Claude's Constitution

San Francisco

Ithaka, California

Rationalism

It will drive you mad (and it's boring)

Books I read (am reading) in 2026

I'm going to be better this year now I'm not a founder
2025

Anna Karenina and the arbitrary party

From San Francisco

Your MLEs are brilliant, but you're giving them the wrong job

A letter to the C-suite

The bitter lesson of LLM evals

Scaling care beats scaling compute

Five hinge questions that decide whether AGI is five years away or twenty

For people who care about falsifiable stakes rather than vibes

Born on third base

The Case for Inheriting Nothing and Building Everything

One-dimensional vs multi-dimensional features in interpretability

How to stop conflating with 768 dimensions

LLMs are really good at k-order thinking (where k is even)

You still need to tell a language model you want to cure cancer before it can help you cure cancer.

Information bounds in quantum gravity

How information theory links quantum mechanics and general relativity

Books I read (am reading) in 2025

A dynamic list
2024

The nihilism of NeurIPS

Some thoughts on our future in AI research

Can quantised autoencoders find and interpret circuits in language models?

Using VQ-VAEs and categorical decision trees to do automatic circuit identification in LLMs.

Learning compressed representations and GPT-5 speculation

Why language models probably get too much from the abstraction we give them for free
2023

How I'd build the future of news

Notes from Rake

What is the future of Rake?

Something I sent to Kirks when we were building