Bilbao Man vs Machine – a resume

Man vs Machine World Team Championship in Bilbao, October 6 – 9, 2004 This encounter between chess playing entities, biological and electronic, took place in the city of Bilbao in the Basque region of Spain. It pitted three strong grandmasters against three top programs. The humans were Veselin Topalov (Bulgaria, Elo 2757, world number five); Ruslan Ponomariov (Ukraine, 2710, No. 13); and Sergey Karjakin (Ukraine, 2576, who at 12 was the youngest GM in the history of the game). The computers were: Hydra, a hardware machine running on a 16-processer array that is located in Abu Dhabi, UAE; Deep Junior, the reigning computer chess world champion, playing on a remote 4 x 2.8 GHz Xeon machine located at Intel UK (Swindon); and Fritz 8, running on a Centrino 1.7 GHz notebook in the tournament hall. The organisers felt that this would provide interesting comparative data, for amateurs and press, about program performances on three different platforms.

Final standing of the Bilbao Man vs Machine

• All the games for instant replay
• All the games in machine-readable PGN

Drama in Bilbao

David Levy

Just as I was writing my previous article for ChessBase.com, Kramnik and Leko were putting on a great show of fireworks in Brissago (game 8). It was as though these players had read my thoughts and decided to introduce an element of excitement into their World Championship. Alas by game 9 and 10 it was back to “business as usual”, with more boring draws.

The last two days of the event were held in the savings bank Bilbao Bizkaia Kutxa (BBK)

In Bilbao, however, the 1st Man vs Machine World Team Championship continued to provide high drama until the very last move of the event. When the final day’s play began the computer team led the Grandmasters by 6 points to 3, and no-one expected anything other than another comfortable round for the programs. After an hour or so, however, it appeared as though mankind might be striking back with a vengeance. Karjakin had demonstrated a significantly deeper knowledge than Fritz of a very sharp line of the Najdorf Variation, and appeared to have a winning advantage when he played 18 c3. Ponomariov steadily built up an ideal-looking position on the Black side of a Scheveningen, with the two bishops and excellent-looking prospects for the endgame. And Topalov was outplaying Junior, picking up a pawn before too long. Was this going to be a 3-0 whitewash for the Grandmasters?

Sergey Karjakin (right) and Ruslan Ponomariov, who both lost unluckily in round four

Not at all. First it was Karjakin’s turn to discover that having a winning advantage is not sufficient against a very strong program. One must also be able to convert such an advantage into the full point! Programs in general are often amazingly resilient and on this day Fritz was no exception. It fought and fought, compelling the 14-year-old Grandmaster to try to prove that Black’s king was fatally exposed or that White could transpose to an endgame in which his two connected passed pawns on the queen side would force a promotion. Once or twice Karjakin faltered, finding perhaps the second best move when only the best would do, and that was enough to allow Fritz’ king to escape. Suddenly it was White whose king was seen to be fatally exposed and Fritz quickly and with almost contemptuous ease turned the tables.

Ponomariov vs Hydra (operated by Chrilly Donninger) in round four

Meanwhile Ponomariov’s position also started to go downhill rapidly. I did not have the opportunity to discuss the game with him but I suspect that, in one line of his analysis, he had overlooked a possibility for White based on Rxh7+, that appeared to lead to at least a draw by bringing the white queen and the other rook to the h-file. (Programs do not make such oversights.) By the time he realised his error, Ponomariov was in serious trouble, from which he was unable to extricate himself, and the end was horrible to behold. This was Hydra’s second slice of good fortune in the event, having been lost against Topalov for much of the game in round 2.

Veselin Topalov (right) came very close to victory against Junior, operated by Amir Ban, in round four

That left Topalov, who entered the endgame a pawn up but was unable to retain control over the position. Junior infiltrated with a rook and even created a menacing looking passed d-pawn before the Grandmaster decided to content himself with a draw by repetition. What had looked, at one time, to be a 3-0 prospect for the Grandmasters, had turned into another bad day for humankind, a dramatic 2½ - ½ disaster.

What have we learned from this crushing defeat of the Grandmaster team? I do not believe that the 8½ - 3½ total score is a true reflection of the relative strengths of the two teams. Instead I feel that some of this disparity is due to insufficient preparation on the part of the Grandmasters. Before the start of the event I had expected that all of the GMs would have analysed two of the programs in great depth (Hydra is not available for purchase so no advance preparation was possible for Chrilly Donninger’s monster). But whatever the level of preparation of team GM it did not show itself to good effect in most of the games, although Topalov appeared to have a much better understanding of how computers play chess than did either of his team-mates.

Veselin Topalov losing his only game (to Fritz, operated by Frans Morsch) in round three

Above all, we learned in Bilbao how difficult it is for a GM to play the best moves often enough to win against a top program. When two GMs play each other, each can expect his opponent to make some inaccuracies during a game, so that one mistake will often be balanced by a subsequent mistake from the opponent. Programs are not like that. The best programs play at a consistently high level, so much so that, after a program leaves its openings book, only very rarely will it make a move that gives much away. A human player, even though he might be able to play 2750 level chess for 95% of his moves, is somewhat more fragile, and for the remaining 5% of the time he will often play the second best or third best move when only the best will do. It is moments like that that frequently determine the outcome of a GM vs computer game – give a beast a single chance and it will pounce.

A crowd of between 10 and 40 spectators watched the games

Even making allowances for inadequate preparation on the part of some of the human players, it is clear that the computer team was significantly stronger than the GMs. This first Man vs Machine Team Championship has indicated a level at which we can witness a highly dramatic contest, but with the average strength of the GMs at this level the contest will inevitably be one-sided. For the next event in this series, the average strength of the human team will need to be raised.

The progress of the games on a big plasma screen. In the bottom right panel is Leontxo Garcia doing commentary for a TV production on the event.

What do the results in Bilbao tell us about the relative strengths of the leading Chess programs? Of the individual results of the computer team, those of Hydra and Fritz stand out. Hydra’s recent crushing victory over Shredder and its score of 3½ out of 4 in Bilbao raises the question of what would happen in a match between Hydra and Junior. Even with Junior on its best form, which was certainly not the case in Bilbao, I suspect that the sheer computing power of Hydra would give it the better chances.

Fritz also made the magnificent score of 3½, including very convincing wins against Karjakin in round one and Topaolv in round three, and it did so running on a laptop! As for Junior, I was reminded of something that Gligoric had said to me many years ago. “When one plays a lot one must expect some better than normal results and some worse than normal results.” This is certainly true of computer programs because a program is a “living” entity, in the sense that it is continually developing. It is not always possible for a programming team to test their baby thoroughly, every time they make “improvements” to the software. Sometimes it happens that an apparent improvement gives rise to an unfortunate side-effect, one that only appears when the program is tested in play against a very strong opponent and in particular types of position. Junior is great program and was simply off form in Bilbao.

The public, with David Levy and TD Jaap van den Herik on the right

The Bilbao organizers have inaugurated an extremely interesting event. The idea for this format was proposed by the International Computer Games Association (ICGA) and appears to have been a success. We can look forward for the 2nd World Computer Team Championship in the certain knowledge that most of the games will be full of drama.

Links

• Official web site
• Previous ChessBase reports
• ChessBase Video report 1