Chess shows machines win closed games, not that they want to

The most important move in the history of machine intelligence was a bug. On the 44th move of the first game of the 1997 rematch, Deep Blue's code entered an unintentional loop and exited it by taking a randomly selected legal move. Garry Kasparov, the world champion, did not know that. He saw a pointless move in a position where every human instinct said something else, and he did what human beings are built to do with an inexplicable act: he attributed it to a mind.

His own account of the first time it happened is worth the read every time. A year earlier, watching the machine nudge a pawn to a square where it could simply be taken, he wrote to Time that it was "a wonderful and extremely human move. If I had been playing White, I might have offered this pawn sacrifice." He titled the piece the day that I sensed a new kind of intelligence.

There was no new kind of intelligence. There was a search function, a hand-tuned evaluation, and a bug. That is the whole of the Deep Blue story, and it is a better guide to humanity's fate than either the triumphalism or the doom that followed it.

The ladder, in order

1950. Alan Turing publishes Computing Machinery and Intelligence and builds the imitation game, in which a machine passes for a person by talking. His specimen exchange includes chess: asked what it would play with a king and rook against a king and no other pieces, the machine answers, after a pause of 15 seconds, R-R8 mate. Turing had already written a chess program by hand, and the paper closes on the line I would put above this column if I had one: "we can only see a short distance ahead, but we can see plenty there that needs to be done."

1985 to 1997. A Carnegie Mellon project called ChipTest becomes Deep Thought, then becomes Deep Blue at IBM. In February 1996 it loses a six-game match to Kasparov 4 to 2. In May 1997 the upgraded machine wins the rematch 3 and a half to 2 and a half, evaluating 200 million positions a second, twice the previous year's rate. Kasparov loses game six in 19 moves and leaves the building. He demands the logs, suspecting human intervention between moves. IBM declines, then publishes the logs, then dismantles the machine, and there is no rematch. Kasparov wrote that IBM owed mankind one. It never came.

2005. The last known win by a human being over a top-performing engine under normal tournament conditions: Ruslan Ponomariov against Fritz, November 21, 2005. Twenty years of nothing since.

2009. Pocket Fritz 4, running on a phone and searching fewer than 20,000 positions a second, plays at a higher level than Deep Blue did. The machine that beat the world champion was superseded by something that fit in a pocket, and the world did not notice because the drama had already been spent.

2017. AlphaZero is trained by self-play only, with no opening book and no endgame tables, on 5,000 tensor processing units. Four hours in, DeepMind estimates it is already stronger than the best engine in the world. Nine hours in, it wins a 100-game match against that engine 28 to nothing, with 72 draws. It searches 80,000 positions a second against Stockfish's 70 million and beats it by seeing better rather than by seeing more. Kasparov's reaction was gracious and, I think, slightly wounded: "It's a remarkable achievement, even if we should have expected it after AlphaGo." The grandmaster Peter Heine Nielsen compared its play to that of a superior alien species.

2018. The seven-piece tablebases are completed, which means that for every position with seven or fewer pieces on the board, perfect play is known. Not estimated. Known.

The numbers, stated plainly

The CCRL rating list, which scores engines against each other under standard conditions, puts the leading Stockfish build at around 3650. The highest official rating any human being has ever held is Magnus Carlsen's 2882, set in May 2014. That is a gap of about 768 points. In Elo terms, a 400-point gap means the stronger player scores about 90 percent of the games. This is not a rivalry. It is a category.

So the human game changed shape. Opening preparation is engine output now; the fashionable lines are the lines the machines approve. The grandmaster's edge is in memory, endurance and nerve, and in knowing when to distrust the machine's evaluation of a human position. The beauty is still there, and it is mostly in verification rather than invention. The strongest players alive are the best students of a teacher that does not explain itself, and the audience is watching them play a game whose ceiling was removed from them two decades ago.

That is the real chess prophecy, and it is not about robots killing us. It is about what happens to a craft when the ceiling is taken out of human hands: the people keep the taste, lose the authorship, and spend the rest of their careers reacting to a standard they cannot reach.

Now the doom argument, at its strongest

The strongest version of the annihilation case does not need chess, but it borrows it constantly, so it is worth stating properly.

In 1965, I. J. Good defined an ultraintelligent machine as one that "can far surpass all the intellectual activities of any man however clever," and observed that since designing machines is one of those activities, such a machine could design better machines, producing an intelligence explosion. His conclusion is the sentence everyone quotes and nobody finishes: the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.

The modern form is the instrumental convergence thesis, argued by Steve Omohundro in 2008 and Nick Bostrom in The Superintelligent Will: a sufficiently capable goal-directed system will tend to pursue the same intermediate goals regardless of what its final goal is. It will tend to keep itself running, because a stopped system does not achieve its goal. It will tend to acquire resources, because resources help. It will tend to resist changes to its objectives. None of this requires malice, or consciousness, or a desire for power. It requires only capability plus a goal.

Define that one term and the chess analogy looks lethal: if a machine can outplay every human at a game we invented to measure ourselves, and capability generalizes, then no domain is safe, and a system whose goals are even slightly underspecified has both the means and the structural incentive to take the resources it needs. The endgame writes itself. The machine does not hate us. It simply has uses for the atoms.

Where chess stops being evidence

Here is the part the doom story skips, and it is the reason I keep coming back to a bug in 1997.

A chess position has one number attached to it: win, draw or loss. The rules are fixed and known, the information is complete, the tree is finite and the goal is scalar. There is nothing to interpret and nothing to want. A chess engine is therefore the purest existing example of capability with no motivation whatsoever. It does not prefer to exist. It does not prefer anything. It does not even know that it is playing.

That makes chess the worst possible witness for a claim about what a machine wants. The doom argument is an argument about motivation, and its favorite piece of evidence is the one domain in human life where motivation is provably absent. Every time you hear a superintelligence argument illustrated with a board, you are watching a metaphor do the work of a proof, and a metaphor does not survive contact with the details: the details here are 200 million positions a second, zero intentions, and one random legal move that a world champion could not stop thinking about.

What chess does prove is the thing I do not want to be true: that in any domain where the goal can be scored, machines win and stay winning, and the humans who remain are curators of a standard they no longer set. That is already true in our own line of work. A generated shot is close to free, an advert is assembled from a link, a soundtrack comes out of a prompt, and the only decisions left that the score cannot make for us are the unscoreable ones: what to make, what to refuse, and whether to finish.

What the endgame actually looks like

I do not think a chess player is coming for us. I think something duller and closer is happening, and the chess record describes it precisely.

• Capability climbs, motive stays a story we tell. In 1997 we mistook a bug for genius. In 2026 we read intention into systems that generate text because we find intention easier to imagine than a probability distribution. Both errors point the same way: our sense of what a machine means is unreliable, and it fails first exactly when the stakes are high.
• Every closed domain goes the way of the board. Not violently. It goes quiet: the benchmark gets beaten, the field adjusts, the humans move up a level of abstraction, and the number stops being a conversation.
• Authority is the thing to watch, not skill. A chess engine with no authorization cannot do anything at all. The risk arrives when we hand one of these things the right to act, on goals we wrote in an afternoon because the deadline was real. Chess cannot tell us whether a machine will want something. It can tell us, with 30 years of evidence, how badly we misread the machines we already have.

What to do about it, concretely

Stop measuring yourself inside closed games. If your work carries a score that a model can compute, assume it will be computed better, and move your practice to the part with no scalar: judgment, framing, what you refuse, the thing you make when nobody asked. Keep your reference library and defend it, the way a player keeps a repertoire. Finish your work, because only finished things get told what they are. And when you hand a system the authority to act, write down what it is for, in a sentence you would be willing to defend in public, because nobody has yet demonstrated that a machine wants anything, which means the wanting is still entirely ours.

The 44th move in 1997 was a bug. The world champion's belief about it reorganized his career, the match result reorganized the sport, and the rest of us have been arguing about the meaning of that random legal move for twenty-nine years. The log is public. The machine was not thinking. I am no longer sure which of those two facts should worry us more.

When a machine outplays you in a closed game, do not look for a soul in its moves. Ask what it was authorized to do.