A chess problem holds the key to human consciousness?

by Frederic Friedel
3/16/2017 – That, in fact, is what the newly founded Penrose Institute is suggesting. The founder, famous mathematician Sir Roger Penrose, has composed a problem devised "to defeat an artificially intelligent (AI) computer but be solvable for humans". The latter are asked to submit their solutions and share their reasoning. But the position itself and the logic behind the experiment is not compelling. Still, you may enjoy checking it with your chess engine.

The story was broken by Sarah Knapton, Science Editor of The Telegraph, who put it up in her newspaper. In it she reported on the launch of the new Penrose Institute, founded by mathematics professor Sir Roger Penrose, who gained world-wide renown in 1988 by working out black hole singularities together with Stephen Hawking (the two received the Wolf Prize in Physics for that). I was unable to find the original chess artical on the Penrose Institute site, but Sarah Knapton quotes extensively from it:

The chess problem – originally drawn by Sir Roger – has been devised to defeat an artificially intelligent (AI) computer but be solvable for humans. The Penrose Institute scientists are inviting readers to work out how White can win, or force a stalemate, and then share their reasoning. The team then hopes to scan the brains of people with the quickest times, or interesting Eureka moments, to see if the genesis of human ‘insight’ or ‘intuition’ can be spotted in mind.

Can you solve the puzzle?

Scientists from the Penrose Institute want to hear from you if you've cracked it. They write:

The puzzle above may seem hopeless for White, with just a king and four pawns remaining, but it is possible to draw and even win. Scientists have constructed it in a way to confound a chess computer, which would normally consider that it is a win for Black. However an average chess-playing human should be able to see that a draw is possible.

A chess computer struggles because it looks like an impossible position, even though it is perfectly legal. The three bishops forces the computer to perform a massive search of possible positions that will rapidly expand to something that exceeds all the computational power on planet earth.

Humans attempting the problem are advised to find some peace and quiet and notice how the solution arises. Was there a flash of insight? Did you need to leave the puzzle for a while and come back to it? The main goal is to force a draw, although it is even possible to trick black into a blunder that might allow white to win.

The first person who can demonstrate the solution legally will receive a bonus prize. Both humans, computers and even quantum computers are invited to play the game and solutions should be emailed to puzzles@penroseinstitute.com.

Read the full Telegraph article here

The Telegraph report was picked up by a number of media outlets, like this one (in Mashable). There Lance Ulanoff writes:

It’s hard to imagine how the game got here—it's even harder to imagine what happens next, let alone a scenario in which four white pawns and a white king could play to a draw, or even win this game. Yet: scientists at the newly-formed Penrose Institute say it’s not only possible, but that human players see the solution almost instantly, while chess computers consistently fail to find the right move.

“We plugged it into Fritz, the standard practice computer for chess players, which did three-quarters of a billion calculations, 20 moves ahead," explained James Tagg Co-Founder and Director of the Penrose Institute, which was founded this week to understand human consciousness through physics. "It says that one side or the other wins. But," Tagg continued, "the answer that it gives is wrong."

True. Above is the calculation displayed by the oldest engine I have installed on my notebook. Fritz 13 scores the position as 31.72 pawns ahead for Black. On ChessBase India Sagar Shah checked it out with Houdini 5.01 Pro 64 bit, down to 34 ply in a four-line search. Result: 24.91 pawns ahead for Black.

It is true that chess engines will display high scores in favour of Black, due to the material advantage of a queen, two rooks, three bishops and a pawn. What they are saying is that Black has a huge material advantage, one that should result in a win (–+). And they will keep moving their bishops, displaying a high positive evaluation right until the 50-move rule approaches and they see there is no possibility of forcing a pawn move by White. Maybe some of our readers can play out the position and tell us when top engines see the futility of continuing to move and display an eval = 0.00.

Interestingly, when I remove two black bishops my ancient Fritz 13 sees the draw in mere seconds. If I remove just one bishop it does not come up with a 0.00 evaluation in a reasonable amount of time.

But now we come to the humans, who can indeed work things out in a flash: the position is extremely contrived, and so the first thing you do is work out that Black has no legal moves except with his bishops. All White needs to do to defend the position is not to capture a black rook and not move the c6-pawn. He simply moves his king around, mainly on the white squares, and lets Black make pointless bishop moves. Absolutely nothing can go wrong. Once again we ask the owner of very old chess engines to check whether any of them will capture a rook, in order to reduce the material disadvantage slightly – but in the process lose the game.

On the other hand the contention that "it is even possible to trick Black into a blunder that might allow White to win" seems extremely far-fetched. Black would need to move his bishops out of the way, while White advances his king to protect the c-pawn, which then promotes (e.g. 1.Kf3 Be1 2.Ke4 Bc1 3.Kd5 Ba1 4.Ke6 Bec3 5.c7 Kb7 6.Kd7 Bf4 7.c8=Q#), but that is not White tricking Black, it is some kind of pointless helpmate.

Anyway, it is trivially easy for White to hold the draw, and the Penrose Institute will probably receive hundreds of correct solutions submitted by average chess players. The scientists say they are interested in the thought process that lead people to the solution – a sudden moment of genius, or the result of days of consternation? "If we find out how humans differ from computers, then it could have profound sociological implications," Penrose told The Telegraph. Really?

There are much more elegant positions and more profound examples that show the difference between human and computer thinking. Back in March 1992 I published the following study in a computer magazine, as a challenge for any machine to get it right:

[Event "La Strategie / CSS 3/92-29"] [Site "?"] [Date "1912.??.??"] [Round "?"] [White "Rudolph, W."] [Black "White to play and draw"] [Result "1/2-1/2"] [SetUp "1"] [FEN "3B4/1r2p3/r2p1p2/bkp1P1p1/1p1P1PPp/p1P1K2P/PPB5/8 w - - 0 1"] [PlyCount "11"] [EventDate "1912.??.??"] 1. Ba4+ $1 Kxa4 (1... Kc4 2. Bb3+ Kb5 3. Ba4+ Kc4 $11) 2. b3+ Kb5 3. c4+ Kc6 4. d5+ Kd7 5. e6+ Kxd8 6. f5 1/2-1/2

You probably know that you can switch on an engine on our JavaScript board (and move pieces to analyse). You can maximize the replayer, auto-play, flip the board and even change the piece style in the bar below the board. At the bottom of the notation window on the right there are buttons for editing (delete, promote, cut lines, unannotate, undo, redo) save, play out the position against Fritz. Hovering the mouse over any button will show you its function.

Fritz & co. display an eight-pawn disadvantage for White. The correct first move is to sacrifice even more material, which is the only way to secure a draw. This is a much more relevant test, as chess engines, playing the white side, will actually select the wrong strategy and lose the game. In the Penrose position computers with "think that White is losing", but they will hold the draw without any problem (I say this without having tested older engines and trying to entice them into capturing a rook and losing the game).

This little recreational pastime of taking the mickey out of chess playing computers has a long history, which will be told at a later stage. I must admit: it is getting harder and harder as these thing get stronger and stronger.


Topics chess problem

Editor-in-Chief of the ChessBase News Page. Studied Philosophy and Linguistics at the University of Hamburg and Oxford, graduating with a thesis on speech act theory and moral language. He started a university career but switched to science journalism, producing documentaries for German TV. In 1986 he co-founded ChessBase.
Feedback and mail to our news service Please use this account if you want to contribute to or comment on our news page service



Discuss

Rules for reader comments

 
 

Not registered yet? Register

truthadjustr truthadjustr 3/19/2017 12:40
This position, and the computer engines being unable to evaluate correctly reveals a lot to how computer engines are made. I realized, engines are meticulously programmed to evaluate balanced positions and any imbalances in the position would caused it to sway away from the search horizon and never ever coming back. To the engine, the deed is done and nothing more needs to be done - the position is now correctly evaluated. Chess engines are not really optimized to understand a position, but instead it will greedily take on whatever is on the table, it will eat it right away because it looks delicious. So far, the Komodo chess engine is the most positional and less greedy of them all, but it still is unable to correctly evaluate the position. The Penrose institute has correctly nailed it, and only shows that chess engines only rely on brute force with priority on material advantage rather than positional understanding.
Miguel Illescas Miguel Illescas 3/19/2017 01:05
https://twitter.com/illescasmiguel/status/843243317156548608
benedictralph benedictralph 3/19/2017 01:47
@truthadjustr

"The Penrose institute has correctly nailed it, and only shows that chess engines only rely on brute force with priority on material advantage rather than positional understanding"

They didn't "nail" anything. If that's what they are claiming, it's something designers of chess engines and most people in AI have known for years. Programmers have, in fact, tried to code primarily for "positional understanding" but then the program ends up like human masters... often making more mistakes than if brute force searching and material gain is prioritized. The proof is in the pudding. All things considered, chess engines today are simply better chess players than humans, even though occasionally a human player might brag he could solve a very very weird position faster.
polkeer polkeer 3/19/2017 03:03
Here, @benedictralph, the quotation out from the Telegraph's article about from what I'm talking about:

"The new institute, which will have arms at UCL and Oxford University, has been set up to study human consciousness through physics and tease out the fundamental differences between artificial and human intelligence."
benedictralph benedictralph 3/20/2017 02:05
@polkeer

So where does this chess problem they created fit in? I see no connection at all.
Peter B Peter B 3/20/2017 03:40
All this shows is that chess programs are faulty, in the sense that they can't handle a few exceptional cases. It is, in principle, not hard to extend chess programs to handle these situations. And when they do, it won't mean they are conscious.

As for the example, is the requirement a draw or a stalemate - the article is not clear. Draw is trivial (just never move a pawn). I can't see a stalemate.
milignus milignus 3/20/2017 06:27
-.- There are people who often judge chess computers poorly. Conventional engines "are not usually designed" for such problems. For example, Stockfish. The developer community of SF is only interested in the Elo of the engine. These people do not care whether SF solves artistic problems or finds checkmates sooner, if it does not contribute to the engine strength. That even considering SF is FLOSS | Free Software | Open source and its people do not have commercial pretensions, I mean, They are not interested in publicity.

For these kind of engines, fortresses are an Achilles' heel. Getting the engine to understand many kinds of fortresses without negative impact on the Elo is difficult, or in other words, both things may be inconsistent. I know Chiron is one of the best in fortresses, You should give him a try to check whether his assessment is more accurate.

On the other hand, I very much doubt that a decent engine allow to be checkmated with Black here. I speculate that the creators of the puzzle would have used some poor chess app that did commit suicide, in order to be able to say that it is possible to be checkmated due to a computer blunder. If that is not true, then it's yellow journalism. To speak about the limitations of computers intelligence to play chess would require to be "more specific", because in practical chess, the limitations of the human intelligence are considerably disproportionate compared to the first ones mentioned.
milignus milignus 3/20/2017 06:28
-.- There are people who often judge chess computers poorly. Conventional engines "are not usually designed" for such problems. For example, Stockfish. The developer community of SF is only interested in the Elo of the engine. These people do not care whether SF solves artistic problems or finds checkmates sooner, if it does not contribute to the engine strength. That even considering SF is FLOSS | Free Software | Open source and its people do not have commercial pretensions, I mean, They are not interested in publicity.

For these kind of engines, fortresses are an Achilles' heel. Getting the engine to understand many kinds of fortresses without negative impact on the Elo is difficult, or in other words, both things may be inconsistent. I know Chiron is one of the best in fortresses, You should give him a try to check whether his assessment is more accurate.

On the other hand, I very much doubt that a decent engine allow to be checkmated with Black here. I speculate that the creators of the puzzle would have used some poor chess app that did commit suicide, in order to be able to say that it is possible to be checkmated due to a computer blunder. If that is not true, then it's yellow journalism. To speak about the limitations of computers intelligence to play chess would require to be "more specific", because in practical chess, the limitations of the human intelligence are considerably disproportionate compared to the first ones mentioned.
gambitg1 gambitg1 4/25/2017 03:12
Artificial intelligence is not same as specific rules. Of-course you can write if-then-else code for this position. That is not artificial intelligence.
You and I did not see this position before but our brain circuitry can more objectively evaluate it.
The mathematician's point is how much work has to be done before AI can be comparable to natural intelligence.