7 min read
M07

Machines · No. 24

The Bitter Lesson

Seventy years of AI research keep teaching the same humbling thing, and builders keep having to learn it again.

· · 7 min · 1,546 words

In short

  • Rich Sutton argued that general methods that make use of growing computation have beaten hand-built human knowledge across the history of AI.
  • The pattern repeats in chess, Go, speech recognition and computer vision, where clever expert rules helped early and then lost to scale.
  • The lesson is bitter because encoding expertise is satisfying and works in the short run, so researchers keep choosing it.
  • For builders, scaffolding that compensates for a model's current weaknesses tends to become dead weight when the next model arrives.
  • Human knowledge still matters, but it is most valuable in choosing the problem, the data and the test, not in hard-coding the answer.

In March 2019, a computer scientist in Canada posted a short essay on his personal website. It had no images, no charts and no equations. You can read it in a few minutes. It has since become one of the most quoted pieces of writing in artificial intelligence.

The author was Rich Sutton, one of the founders of modern reinforcement learning, the branch of AI that learns by trial and error. Years later he would share the Turing Award, computing's highest honour, with his longtime collaborator Andrew Barto. The essay was called "The Bitter Lesson," and its first sentence did not waste any time:

"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin."

That is the whole argument. The rest of the essay is evidence, and a warning about why people keep refusing to believe it.

The pattern

Sutton's point rests on a simple fact about computers. For decades, the amount of computation you can buy for a given price has kept falling, fast. Any method that gets better when you give it more computation will therefore keep getting better, almost for free, year after year.

Researchers, though, do not usually think on that timescale. They think about the next paper, the next benchmark, the next few years. Over that window, the fastest way to make a system better is often to give it what you know. Teach it the rules of the domain. Encode the expert's tricks. Build in your understanding of how the problem works.

That approach helps, at first. It is also deeply satisfying, because your expertise is visibly inside the machine. But over the long run, Sutton argued, it hits a ceiling, and sometimes it gets in the way. The real breakthroughs eventually come from the opposite direction: methods that need less human insight and more computation, especially search and learning.

Then he walked through the examples.

Chess, Go, speech, sight

Chess. When IBM's Deep Blue beat the world champion in 1997, it did so largely through massive, deep search, looking at enormous numbers of possible moves. Many researchers who had spent years trying to capture human understanding of chess were dismayed. Brute force had won, they said, but it was not how people played and not a general strategy. They were not wrong about how people played. It did not matter.

Go. The same story repeated about twenty years later, with a game long considered far harder for computers. Huge effort went into using human knowledge of Go to avoid search. What finally worked was search combined with learning, with systems improving by playing vast numbers of games against themselves.

Speech. In the 1970s, a competition sponsored by the US defence research agency DARPA pitted different approaches to speech recognition against each other. Some used detailed human knowledge of words, sounds and the human vocal tract. Others were newer statistical methods that did far more computation and relied less on linguistic rules. The statistical methods won, and the field was slowly reshaped around them. Decades later, deep learning pushed even further in the same direction. A joke often attributed to Frederick Jelinek, who led speech recognition research at IBM, captures the mood: every time he fired a linguist, the system got better.

Vision. Early computer vision tried to describe the world in terms humans found natural: detecting edges, fitting simple shapes, hand-designing features. In 2012, a neural network now known as AlexNet, trained on just two consumer graphics cards, won the major ImageNet image recognition competition by a wide margin. Within a few years, most of the hand-built approaches had been abandoned. The networks learned their own features, and those features were better.

Four fields. Four decades. One shape. Human cleverness wins the sprint, and scale wins the marathon.

Human cleverness wins the sprint, and scale wins the marathon.

Why it stays bitter

If the evidence is that consistent, why does anyone still fight it?

Partly because the losing approach really does win in the short run. If you have a deadline next quarter, adding expert knowledge is often the right move for that deadline. The bitter lesson is a statement about trends, and trends are invisible from inside a single project.

Partly because it hurts. Sutton's word was chosen carefully. The success of general methods comes at the expense of something people care about: the idea that understanding a problem deeply, and writing that understanding into the machine, is the noble path. Being told that more computation and a simpler method beat your decade of insight stings in a way that no purely intellectual argument does.

And partly because the lesson cuts against how we see minds. Sutton argued that the real contents of minds are enormously, irreducibly complex, and that we should stop trying to write those contents in by hand. Instead, we should build methods that can discover that complexity for themselves. In his words, we want AI agents that can discover like we can, not ones that contain what we have discovered.

That is a humbling sentence if your career is built on what you have discovered.

The counterargument

Not everyone accepted the lesson whole. Shortly after Sutton's essay, the roboticist Rodney Brooks wrote a reply called "A Better Lesson." Among his points: even the winning methods contain plenty of human design. The neural networks that conquered vision were built with structure that people chose because they understood something about images. And raw scale has real costs, in energy and money, that cannot be waved away.

He has a point. Read carefully, the lesson is about where human knowledge pays off over time, not whether it is worth anything. Knowledge baked in as rigid rules tends to become a ceiling. Knowledge that shapes a flexible, general method, deciding what it should learn from and how it should be tested, tends to last.

A decade before Sutton's essay, three researchers at Google, including Peter Norvig, had made a related argument in a paper called "The Unreasonable Effectiveness of Data": simple models fed with enormous amounts of data often beat elaborate models built on careful theory. And in 2020, researchers at OpenAI showed that the performance of language models improved in a smooth, predictable way as they scaled up the size of the model, the data and the computation. The lesson was no longer just a historical pattern. It had become something you could plan around.

What it means if you build

Here is where the essay stops being history and starts being practical.

If you build products on top of AI models today, you face the bitter lesson in miniature every few months. You find a weakness in the current model. You build a clever workaround: a chain of prompts, a set of hand-written rules, a special pipeline that checks and fixes its output. It works. Customers are happy.

Then a new model comes out, and it simply does the thing. Your workaround is now extra code that is slower, more fragile and no longer needed. Sometimes it actively makes results worse, because it forces the new, smarter model through steps designed for an older, dumber one.

So a few questions are worth asking about anything you build:

  1. Will this get better or worse when the model gets better? Products that improve automatically with each new model are riding the trend. Products that exist only to patch a current weakness are betting against it.
  2. Am I encoding an answer or designing a test? Writing down how the problem should be solved tends to age badly. Writing down what a good result looks like, and how to check it, tends to age well.
  3. Where does my knowledge truly matter? Usually it is in choosing which problem is worth solving, knowing which data reflects reality, and judging whether the output is actually right. Those are the places scale does not automatically reach.

None of this means you should never build workarounds. Sometimes the workaround is what lets you ship this year, and a company that does not survive this year will never see the next model. Just build them knowing they are temporary, and be ready to delete them without grief.

The lesson for people, too

There is a quieter version of this lesson for anyone learning a craft right now.

It is tempting to memorise the tricks of today's tools: the exact prompt phrasing, the right sequence of settings, the workaround everyone shares this month. Those tricks are hand-built knowledge, and they tend to expire with the next version.

What does not expire is the general skill underneath: the ability to learn a new tool quickly, to test whether it works, to recognise a good result when you see one. In a world shaped by the bitter lesson, your own most durable asset is not what you know. It is how fast and how well you can find out.

The lesson only tastes bitter if what you wanted was to be needed inside the answer. If what you wanted was the answer itself, it may be the best news the field has ever had.

Further reading

  • The Bitter Lesson — Rich Sutton
  • A Better Lesson — Rodney Brooks
  • The Unreasonable Effectiveness of Data — Alon Halevy, Peter Norvig and Fernando Pereira
  • Reinforcement Learning: An Introduction — Richard S. Sutton and Andrew G. Barto

More from the shelf

M01 · Machines · 7 minIn Defense of FrictionWe spent decades removing effort from everything, and accidentally removed some of the meaning with it. M02 · Machines · 7 minIntelligence Runs on ElectricityBehind every chatbot is a power plant, a cooling system and a plot of land, and that changes who wins the AI race. B07 · Builders · 6 minDefault AliveOne question decides whether your startup survives, and most founders have never actually run the numbers.

See the whole library →