TCEC: Superfinal Houdini vs Komodo

Thoresen Chess Engines Competition

Season 10

After the retroactive disqualification of Rybka by the ICGA in 2011, computer chess fans have more and more focused on the TCEC tournament (1), which many see as real Computer Chess World Championship. After all, Rybka and other strong programs which did not take part in the ICGA tournaments were allowed to start in TCEC tournaments.

The prestige of this event is even higher because all participating programs run on identical hardware and because after a number of qualifiers the two strongest programs meet in a so-called "superfinal". Here, each program plays the same opening positions twice, once with white, once with black. This helps to level the field.

Superfinal: Houdini vs Komodo

The finalists are Houdini and Komodo which play 100 games against each other. The programmers were allowed to hand in new, improved program versions for the final which began on Monday, November 20th, 2017. It looks like Houdini may be running away with it, as after 16 games, the program leads with 4 wins and no losses. The table below will remain current. Remember this is a 100 game match!

Current standings

Live game

Fritz 16 - He just wants to play!

Fritz 16 is looking forward to playing with you, and you're certain to have a great deal of fun with him too. Tense games and even well-fought victories await you with "Easy play" and "Assisted analysis" modes.

All Superfinal games

How they got here

The 10th edition — "Season 10" — started October 14th, 2017. In stage 1, 24 programs battled it out and Komodo won.

Stage 1

Click or tap on the table to expand

All Stage 1 games

Komodo Chess 11

The multiple computer chess world champion comes in a new and yet more powerful version. Thanks to co-author US Grandmaster Larry Kaufman, Komodo is the strategist among the top chess programs!

Stage 2

In stage 2 the eight best programs played against each other and this turned an exciting fight between Houdini, Komodo, and Stockfish, the winner of last year's edition. In the end Houdini and Komodo finished first with 18½/28 while Stockfish narrowly failed to qualify for the superfinal.

Standings

Click or tap on the table to expand

All Stage 2 games

Houdini 6 Standard

Houdini 6 continues where its predecessor left off, and adds solid 60 Elo points to this formidable engine, once again making Houdini the strongest chess program currently available on the market.

Orion chess computer | Photo: Dezidor (Own work) [Public domain], via Wikimedia Commons

"20 ply after 30 million years"

In 1980 computer chess was still in its early stages of development and experts estimated that a computer would need 30 million years to reach a search depth of 20 ply. Of course they were wrong.

In the last ten years computer chess has changed dramatically. More and more and stonger and stronger programs astonish chess fans, help to analyse games and change the way players train and prepare openings. Last week Fritz 16 was released, the latest version of the classic among the chess programs. In May and in October 2017 ChessBase released Komodo 11 and Houdini 6, two of the strongest commercial chess programs. Let's have a look at the most important developments in computer chess from 2007 to 2017:

2007: Zappa wins against Rybka

In their match in Mexico City 2007 Vice World Champion Zappa defeated World Champion Rybka 5½ : 4½ in a dramatic event full of exciting games while Vishy Anand won the FIDE World Championship tournament that was played parallel to the computer match in the same hotel.

But who would have the World Championship tournament if Zappa and Rybka had played in it? A pointless question — one might just as well speculate how the fastest 100-meters runners would fare against a Formula 1 racing car. And neither Zappa nor Rybka had played faultless chess. In fact, the match revealed a number of weaknesses of the programs and thus considerably contributed to their improvement, particularly in regard to Rybka. At the end of July 2008 a markedly improved version of Rybka (Rybka 3) was released and again clearly led the Elo-rankings.

2008 - 2010: Rybka on top again

While Rybka developer Vasik Rajlich made a successful new start with Rybka 3, Anthony Cozzie, the developer of Zappa, finished his new Zappa version ("Zappa Mexico II") but then quit programming chess computers at the end of January 2008 and did not take part in the Computer World Chess Championship 2008 in Beijing, where Rybka won with 8.0/9, ahead of Hiarcs 7.0/9 and Junior 6.0/9. Rybka also won in Pamplona 2009. He finished with 8.0/9, ahead of Deep Sjeng, Shredder and Junior, who all had 6½/9.

In Kanazawa 2010 Rybka won his fourth and last title, again with a score of 8.0/9. Shared second place went to Rondo and Thinker, who both finished with 6½/9. Stefan Meyer-Kahlen's Shredder finished fourth with 6.0/9, Johannes Zwanzger's Jonny became fifth with 5½/9. Interesting is that "Rondo", a version of Zappa, programmed by Zach Wegner, took part in the tournament. Anthony Cozzie had given Zappa's source code to Wegner but unfortunately the work on Zappa was not continued - at least, I did not find a version of Rondo for sale or download. The game between Rybka and Rondo ended in a draw. (a)

The surprising belated disqualification of Rybka and its consequences

To the surprise and dismay of many fans in 2011 the ICGA (International Computer Games Association) belatedly revoked Rybka from all its World Championship titles, because allegedly the source code of Crafty and Fruit had been used and thus the program would no longer meet the requirements of originality demanded by the ICGA. But here the opinions of the experts are divided. The probably best known computer expert who argued against the belated disqualification of Rybka is Dr. Sören Riis (b), and he is supported by the Rebel programmer Ed Schröder. But how did all this happen?

Rybka 4

For years Rybka dominated the computer chess scene. For years, no other program was able to achieve a similar increase in playing strength. For chess analysis Rybka has long become one of the most important analysis tools of professions and amateurs alike. So that this state of affairs will continue in the future, the Rybka-team gathered around programmer Vasik Rajlich has developed a new version which is even better: “Rybka 4 plays more aggressively and more tactically”.

From Fruit to Rybka

Here we have to go back a couple of years, to 2005, to be precise, when the French programmer Fabien Letouzey published his program Fruit 2.1 as open source under the GPL (General Public License) (c) and finished second behind Zappa at the Computer World Chess Championship. The International Master Vasik Rajlich studied, as he freely admitted in an interview as early as December 2005, (d) the Fruit source code "forwards and backwards", and incorporated a lot of it into his own chess program Rybka.  (“I went through the Fruit 2.1 source code forwards and backwards and took many things.”) With remarkable results. In the computer chess magazine "Selected Search", from February/March 2011, the Canadian programmer Peter Skinner indicated that soon after the publication of Fruit's source code Rybka's playing strength increased dramatically: previously Rybka had a playing strength of about 1800 Elo but 16 months later Rybka's playing strength had increased by about 1200 ELO-points. (e)

It is also known that a lot of chess programmers follow the open source program Crafty (f) to implement some basic concepts such as e.g. the display of the board and the pieces instead of "reinventing the wheel" time and again. Judging this morally and legally is another question. At any rate, the Crafty code can hardly be hold responsible for the fact that Vasik Rajlich's program was playing so much stronger because the Crafty versions at that time were several hundred Elo-points weaker than Rybka. Some examples: Crafty 23.4 and Rybka 4, both released in 2010, were no less than 280 Elo-points behind Rybka in the CCRL rating-lists. Even the most recent Crafty version (Crafty 25.2 of 2016) still has about 120 Elo-points less than Rybka 4.

But neither was the enhanced Fruit version 2.3.1 from August 2007 as good as the Rybka version 2.2 which had been published about nine months earlier: 2887 ELO for Rybka 2.2 against 2785 ELO for Fruit 2.3.1, that is about 100 Elo-points more for Rybka. (To avoid comparing apples with oranges I used the 32-bit versions on 1 processor for the comparison because unlike Rybka 2.2. Fruit 2.3.1 does not run on multiple processors. With 64-bit and 4CPU Rybka 2.2 reaches 2978 Elo-points.)

Critics of the Rybka-disqualification also pointed out that the interview in which Vasik Rajlich admitted that he had taken a lot from Fruit had already been published when Rybka was allowed to play in the World Championships from 2006 to 2010. Consequently, the program should have been disqualified earlier and not years later. After Rybka's disqualification the programs that finished second in the World Championships were declared as winners, namely Zappa (2007), HIARCS (2008), Shredder, Deep Sjeng, Junior (2009) and Rondo and Thinker (2010).

Every computer chess friend is asked to decide for himself whether the disqualification was justified or not. But one thing is certain: Vasik Rajlich did groundbreaking work for computer chess by continuing the work of Fabien Letouzey, while optimsing it and enriching it with ideas of his own. This made Rybka not only faster but also better than its competitors, positionally and (particularly after version 3) tactically.

From Rybka to IPPOLIT

Ironically, in 2009 unknown programmers did with Rybka what V. Rajlich had done with Fruit, though they probably acquired Rybka's source by disassembling. These unknown programmers very carefully studied the structure of Rybka 3 and other programs such as Fruit, Strelka etc. carefully, and with all these ingredients they developed a new improved chess program which they published in 2009 under the name of "IPPOLIT" as open source.

Only a clone of Rybka 3?

Vasik Rajlich accused the programmers of having published a "clone" of Rybka 3 but a look at the lively discussions in the internet forums (g) reveals that a lot of people disagree with this claim. Why should the programmers take the trouble to produce a Rybka-clone and later strip it of its ability to run on multiprocessors and to take away the multi-PV-ability as well (after which only one variation can be displayed when analysing positions)? IPPOLIT testers also noted that the new program played the endgame significantly better than Rybka 3. Clarifying the issue was impossible because V. Rajlich did not disclose the source code of Rybka 3 or at least more than small parts of it.

A lot of "new" engines

The publication of the IPPOLIT source code suddenly gave all programmers insight into the structure of a chess programs with a playing strength of about 3100 Elo (measured by the CCRL standard 40/40).

Peter Skinner:

"The publication of this code did not only change the landscape of computer chess but also the hierarchy on which many rating systems are based. Cloning this code has occured so often that downloading and using these programs is not accepted by users but is also welcomed as a new evolution in computer chess." (e)

In fact, one can assume that many (if not most) modern chess programs profited from it. But how to judge all this legally or morally is not our topic here. But fact is that computer chess has made enormous progress since IPPOLIT 0.080a was available as open source.

Because IPPOLIT was boycotted by several computer ranking lists — due to the cloning accusations — the newer versions appeared under new names as "RobboLito", "Igorrit" and "IvanHoe". Step-by-step missing functions such as the ability to run on multiprocessors, Multi-PV etc. were implemented and with the so-called "Robbobases" the programmers also developed endgame tables of their own with up to six pieces.

Other leading chess programs such as Kranium and Sentinel tried to combine the best from the four IPPOLIT-variations. This led to "Firebird" and "Fire" (until version 3.0). On the "Chess Logik" site it is claimed that Robert Houdart based his early versions of Houdini on the source code of RobboLito. (h) José Maria Velasco openly admits that his top program "Bouquet" is based on IPPOLIT. The same is the case with "ippos" by Roberto Munter. Further developments of IvanHoe are amongst others "PanChess", "Deep Saros" and "Vitruvius". Igorrit developed into "Elektro", etc., etc. (i)

The concepts and ideas that had been published with the source code of the IPPOLIT programs were sometimes simply copied but often also improved and developed further. This led to another massive push in the development of computer chess.

Houdini — the new number 1

I can imagine that the officials of the ICGA misjudged the long-term consequences of their disqualification of Rybka (j). After all, what is a Computer Chess World Championship without the best program in the world? And since the disqualification of Rybka in 2011 the interest of the computer chess friends has more and more shifted to the TCEC tournament (k), which since then has been seen by many as the real Computer Chess World Championship. Here Rybka was still allowed to participate and other strong programs which were missing at the ICGA tournaments were also allowed to play.

The value of this event is further increased by the fact that all programs run on identical hardware and that after several qualifiers the two strongest programs face each other in a so-called "superfinal". Here, the two programs play the same opening positions twice, one time with White, the other time with Black. This obviously helps to level the field.

Houdini 6 Pro

Houdini 6 continues where its predecessor left off, and adds solid 60 Elo points to this formidable engine, once again making Houdini the strongest chess program currently available on the market.

A brief history of TCEC

"Season 1", which was played from December 2011 to February 2011 was won by Houdini 1.5a after beating Rybka 4 in the final. As mentioned above, Robert Houdart, the Belgian developer of Houdini, is said to have based his program on the IPPOLIT version RobboLito (h). Even if this is true, he managed to improve this program and to make it faster. And in the endgame, where IPPOLIT is already very strong, additional knowledge was implemented, e.g. about fortress positions which a lot of the other programs still misevaluate. Until the release of version 5, Robert Houdart's homepage also mentioned that a lot of ideas and techniques were taken from the open source programs IPPOLIT and Stockfish. (l) This clearly shows the positive impact of open source programs on the development of playing strength in computer chess.

"Season 2" (February — April 2011) was again won by Houdini 1.5a, this time ahead of Rybka 4.1.

("Season 3" was not played to an end, and there was no TCEC winner in 2012.)

"Season 4" (January — May 2013) went to Houdini 3 after a win against Stockfish 250413 in the final.

Komodo and Stockfish number 1 and 2

"Season 5" was won by Komodo 1142 ahead of Stockfish 191113. With this victory the program that had been originally developed by Don Dailey and Larry Kaufman (at first it was called "Doch" but later was renamed "Komodo") broke into the world's absolute top. Unfortunately Don Dailey died at the age of 57, on November 22, 2013, shortly before Komodo won the tournament. In October 2013 he had brought Mark Lefler into the team who since then has further developed Komodo together with Larry Kaufman. (m)

"Season 6" (February - May 2014) was the first time Stockfish 1705114 could win, after beating Komodo 7 in the superfinal. Stockfish is based on the open source program Glaurung 2.1 of the Norwegian Tord Romstad. Apart from him Marco Costalba, Joona Kiiski, Gary Linscott and a lot of other programmers from various countries successfully work on the further development of the program. (n)

"Season 7" (September - December 2014) was won by Komodo 1333 ahead of Stockfish 141214.

"Season 8" (August to November 2015) was won by Komodo 9.3 ahead of Stockfish 021115.

"Season 9" (May to December 2016) ended in favour of Stockfish 8 who defeated Houdini 5 in a gripping superfinal of 100 games.

"Season 10" has been played since October 14, 2017. In the initial stage, 24 programs started.

Stage 2 in which the eight best programs played turned into an exciting battle between Houdini, Komodo and reigning champion Stockfish. In the end Houdini and Komodo won with 18½/28 each while Stockfish (18.0/28) narrowly missed the qualification for the superfinal. Behind Stockfish were Fire (15.0/28), Chiron (11½/28), Ginkgo (10½/28) and Andscacs and Boot (each with 10.0/28).

Thus Houdini and Komodo meet in the 100-game superfinal.

One sign of the tecnical progress is the fact that Houdini and Komodo will run on 44 processors whereas Zappa and Rybka in 2007 had to content themselves with "only" 8.

Fritz 16 - He just wants to play!

Fritz 16 is looking forward to playing with you, and you're certain to have a great deal of fun with him too. Tense games and even well-fought victories await you with "Easy play" and "Assisted analysis" modes.

---

An amusing prediction from 1980

In the book "Schach dem Computer" I found an interesting table about "Computational effort for analysing positions" (o). In the chapter "Wie ein Computer Schach spielen lernt" ("How a computer learns to play chess") author Frieder Schwenkel gives the following calculations for calculation depths of up to 20 ply:

With 0.5 seconds for one ply, 3 seconds for 2 and 20 seconds for 3 ply the start is rather harmless. For 5 ply he assumes 15 minutes, for 7 ply 10 hours but for 10 ply already the quarter of a year.

To calculate 15 ply deep 2500 years (!) are assumed and for 20 ply 30 million (!) years.

He sums up: "Even the fastest computers of the far-away future will have to capitulate at a depth of 20 to 25 ply."

Calculating 20 to 25 ply deep are easy for modern chess programs. It is amazing how fast 30 million years pass today!

An increase of more than 400 Elo in only ten years

In the Zappa-Rybka match in September 2007 the two strongest chess programs of that time played against each other. Ten years later Komodo, Stockfish and Houdini lead the ranking lists, programs that also succeed at the TCEC. In the revised CCRL-Elo-list (r) of September 9, 2017 (time limit: 40 moves in minutes) Komodo 11.2 (64-bit, 4CPU) was first with a rating of 3440 Elo-points. Second was Stockfish 8 (3423) and third was Houdini 5.01 (3411).

The last Zappa version (Zappa Mexico II) of January 2008 had 3017 Elo-points and was still on rank 26. On 4CPU (Quadcore = 4 processors) it is about 10 Elo-points stronger than the Zappa version that won 5½-4½ against Rybky in 2007.

Thus, the difference between Zappa Mexico from 2007 and Komodo 11.2 from 2017 is no less than 433 Elo-points. This shows that the top engines have dramatically increased their playing strength in the last 10 years!

More Elo = better for analysing games?

But I would like to correct this a bit because I do have the impression that some programmers want first of all to make a good impression in computer-vs-computer duels, and thus their programs have a somewhat limited way to search for the best move. These programs reach impressive search depths but occasionally miss brillant moves. One example is the following position from the game Jeroen Piket – Ilya Smirin (Interzonal Tournament Biel 1993), after move 30 by Black: 

Play out the moves on the live diagram!

Piket won with the queen sacrifice 31.Qxd7! Rxd7 32.Rxd7 Rd8 33.Re7 and the threats of Rc1-c7 and Nf3xe5 decide the game in White's favour. (The game ended after 33...Be2 34. Nxe5 Qa2 35. Rcc7 Rg8 36. Ne8 1-0.

On my Acer Aspire V5 (Dualcore with 1.6 GHz), Crafty 23.4 (released in November 2010) needed only 16 seconds to find 1.Qxd7! while the top program SugaR XPro 1.2 that has 550 more Elo-points (number 1 on the CCRL-Elo-list from October 2017), after more than 20 minutes still shows the clearly weaker move 1.Qa4.

SugaR is a "Stockfish", modified by Marco Zerbinati, and because it reached a higher rating than Stockfish 8 it represented the Stockfish family for some weeks on the CCRL “Pure List” 40/40. Meanwhile, this role has been taken by the program asmFish which is also derived from Stockfish.

Vasik Rajlich now develops Fritz for ChessBase

In 2015 Vasik Rajlich started to further develop Fritz. Fritz 15 was the first Fritz version developed by him. The program which was originally developed by Frans Morsch and Mathias Feist appeared first in 1991 and is one of the most successful chess programs of all times. I particularly remember the blitz tournament in the "German Museum" in Munich in 1994 in which Fritz 3 started in high class field and finally shared first place with World Champion Kasparov against which it then lost in the tie-break match.

In 1995 Fritz became Computer World Champion in Hong Kong. Matches against GM Robert Hübner (2001), World Champion Vladimir Kramnik (2002) and former World Champion Garry Kasparov (2003) were all drawn. In 2006 the multiprocessor version Deep Fritz played a 6-game match against World Champion Vladimir Kramnik in Bonn and won 4:2. The last versions developed by Frans Morsch (Fritz 13 and Deep Fritz 13) appeared 2011 and 2012. Gyula Horvath was responsible for Fritz 14. It will be interesting to see how the new version Fritz 16 will fare.

In the interview from December 2005 Vasik Rajlich said: "The main goal with Rybka is to provide a useful analysis tool for our customers." The definitely succeeded here with Rybka and with Fritz.

Fritz 16 is available in the ChessBase shop as are top engines such as Houdini, Komodo and Rybka. The interface allows to add and to comfortably use free UCI programs such as e.g. Stockfish 8 or Bouquet 1.8.

Sources and notes:

• (a) See www.game-ai-forum.org/icga-tournaments/competition.php?id=1
• (b) www.top-5000.nl/riis.pdf
• (c) chessprogramming.wikispaces.com/Fruit
• (d) web.archive.org/web/20090222085125/http://www.superchessengine.com/vasik_rajlich.htm
• (e) Computer chess magazine "Selected Search 152", published by Eric Hallsworth, February-March 2011, p. 17 - 19
• (f) chessprogramming.wikispaces.com/Crafty
• (g) en.wikipedia.org/wiki/IPPOLIT
• chessprogramming.wikispaces.com/Ippolit
• One example for the controversial Rybka - IPPOLIT discussion. In this post about the cloning accusations against the IPPOLIT program one participant in the discussion compares this with the witch hunts in medieval times. Scrolling down the page reveals how heated the discussion was.
• (h) chesslogik.com under "RobboLito"
• (i) www.chesslogik.com/
• chessprogramming.wikispaces.com/Fire
• chessvibes.com/?q=reports/chess-engine-controversy/
• chessprogramming.wikispaces.com/Bouquet
• chessprogramming.wikispaces.com/Vitruvius
• chessprogramming.wikispaces.com/Roberto+Munter
• chessengines.blogspot.de/2016/11/ippos-127-uci-chess-engine-new-version.html
• (j) Only 4 (!) programs took part in the ICGA "Computer Chess World Championship" 2017, see icga.leidenuniv.nl/?page_id=1990 and chessprogramming.wikispaces.com/WCCC+2017
• It is a funny fact that Jonny ran on 2400 (!) processors, Komodo on 60, Shredder and Chiron only on 32!
• (k) TCEC stands for "Thoresen Chess Engines Competition" or, lately, "Top Chess Engine Championship", see: en.wikipedia.org/wiki/Top_Chess_Engine_Championship
• (l) "Without many ideas and techniques from the open source chess engines Ippolit and Stockfish, Houdini would not nearly be as strong as it is now." web.archive.org/web/20161025012104/http://cruxis.com/chess/houdini.htm
• (m) chessprogramming.wikispaces.com/Komodo
• (n) chessprogramming.wikispaces.com/Stockfish
• (o) "Schach dem Computer" by Hans-Peter Ketterling, Frieder Schwenkel and Ossi Weiner (Munich 1980)
• (p) The most reliable is the "Pure List" because here the fact that newer versions of a program performance above average against the weaker versions of the same program is corrected by allowing only the strongest version to play. This Elo-list is continuously updated and this leads to permanent small changes and gradually new programs or program versions are included into the list. Here's the link to the current list.

Translation from German: Johannes Fischer

Links

• TCEC stands for "Thoresen Chess Engines Competition" or, recently, for  "Top Chess Engine Championship" (Wikipedia)
• Official site
• The TCEC "archive" provides an overview of previous TCEC tournaments and allows you to download the games