10 years Houdini

by Stephan Oliver Platz
6/30/2020 – Chess engines are programs with character and each engine has its own way of playing. That is why it makes sense to analyze with different engines and that is how grandmasters get the maximum out of the engines. Houdini is an engine with a marked character and on the occasion of its 10th birthday Stephan-Oliver Platz takes a look at its development.
ChessBase's rights to use the Stockfish software are under the terms of the GPL-3.0 licence. ChessBase has agreed with the Stockfish team to cease distributing the product for a period ending on 7 November 2023.

Winning starts with what you know
The new version 18 offers completely new possibilities for chess training and analysis: playing style analysis, search for strategic themes, access to 6 billion Lichess games, player preparation by matching Lichess games, download Chess.com games with built-in API, built-in cloud engine and much more.

10 years Houdini

"Welcome to the Houdini Chess Engine home page." You could read this sentence for the first time in May 2010, when the programmer Robert Houdart offered the first version of his new chess program for free download. At that time nobody could have guessed that only seven months later Houdini would replace Rybka as No. 1 in the world and dominate computer chess for several years. How strong was Houdini 1 and what were its special qualities? How was the program improved and what makes it especially valuable for chess analysis? On the occasion of the tenth anniversary I would like to answer these questions and of course also look at Houdini's performance in computer tournaments.

Houdini, the escape artist

In the middle of May 2010 the Belgian Robert Houdart had completed the first version of his chess program. Now he had to find a name for it. He decided to call it Houdini", certainly with good reason. First of all you'll notice that the first four letters correspond to the last name of the programmer. Why did it eventually become Houdini"? On his homepage Robert Houdart explained why: The name 'Houdini' was chosen because of the engine's tenacity in difficult positions and its ability to defend stubbornly and escape with a draw – sometimes by the narrowest of margins. At the same time Houdini will deny its opponents the same escape routes when it has the better position." (a)

Harry Houdini was the stage name of an American escape artist and magician of Hungarian origin. His real name was Erik Weisz and he lived from 1874 to 1926. When he was four years old, his family left Austria-Hungary and emigrated to the USA. In the Wikipedia article dedicated to him, we read the following about his escape tricks: New about Houdini's show was that he was actually able to escape from every sort of shackles provided to him and was tied up naked under test conditions by experienced police officers for advertising purposes." He was even able to escape from the handcuffs put on him by the police. Little by little, his tricks took on more and more grotesque forms: He even freed himself under water and later also from strait jackets hung upside down from skyscrapers". (b)

These are quite remarkable abilities which, what about chess is concerned, you can very well find in the engine named after Houdini. This is not a surprise, because without inventive defensive faculties no chess player can make it to the top of the world. The world champions Steinitz, Lasker, Capablanca, Botvinnik, Smyslov, Petrosian and Karpov, among others, were considered to be particularly tough defenders.

How strong was Houdini 1?

Since I could not find Houdini 1 in any rating list, I conducted a small test match over 20 games between the first Houdini version from May 2010 and the chess program Critter 0.90 by the Slovakian programmer Richard Vida which was published half a year later. I used the CCRL standard time control of 4 minutes for 40 moves (64-bit, 1CPU) which was usual until 2019. The final result was 5-3 with 12 draws in favour of Houdini 1. Since Critter 0.90 64-bit has an ELO rating of 3049 in the CCRL 40/4 list, the corresponding ELO rating of Houdini 1 is somewhere around 3083. This is in the range of Rybka 3's rating of 3078 ELO. Of course, significantly more test games should have been played, but a few ELO up or down are not important here. I just wanted to check if the first version of Houdini already had a remarkable strength. The small test match against Critter confirmed this assumption.

Playing like Morphy or Anderssen

I was astonished to discover that even the first version of Houdini had extraordinary skills and a style of play reminiscent of the best times of Anderssen and Morphy. Let's take a look at the seventh match game between Houdini and Critter. After 29 moves, Houdini was faced with a threatening mate which could only be parried by sacrificing the queen. But in just five moves the great escape artist not only wriggled out of all difficulties, but even turned the tables. Some more spectacular sacrifices followed and Critter resigned after 55 moves had been played:

 
New ...
Open...
Share...
Layout...
Flip Board
Settings
MoveNResultEloPlayers
Position not in LiveBook
1.Nc3 c5 2.e4 d6 3.Nf3 Nf6 4.d4 cxd4 5.Nxd4 a6 6.Be3 e5 7.Nb3 Be6 8.f3 Be7 9.Qd2 Nbd7 10.g4! This sharp line of the Sicilian usually leads to thrilling games. h6 11.0-0-0 b5 12.a3 The theory is in favour of 12.Kb1 or 12.h4 12...Qc7 13.h4 Nb6 14.Bxb6 Qxb6 15.Kb1 Rb8 16.Be2 Rc8 16...b4 looks more logical, otherwise the rook had better moved to c8 at once. 17.Nc1 Qb7 18.N1a2 Rc5 19.Rdg1 Bf8 19...g6!? 20.g5 hxg5 21.hxg5 Rxh1 22.Rxh1 Nh5= 20.g5 Nh5 21.Nb4 Nf4 22.Nd3 Nxd3 23.Bxd3 b4 24.axb4 Qxb4 25.Bxa6 Qb6 26.Be2 Ra5 27.b3 Qb4 28.Qd3 28.Rd1 might be better. 28...Be7? Missing a good opportunity. 28...Qa3 is parried by 29.Na4 Be7 30.Qc3!= Rxa4? 31.Bb5+! but 28...d5! was promising. The plan is Bf8-c5-d4. After 29.Nxd5 29.exd5? Bf5! 30.Ne4 30.Qxf5 Qxc3-+ 30...Bxe4 31.fxe4 Bc5-+ 29.Nb5? d4! 30.Kb2 Bd7-+ 29...Bxd5 30.exd5 Bc5 White must get rid of the dangerous black bishop by giving up the exchange 31.Rg4 Bd4 32.Rxd4 , etc. 29.gxh6! A strong move looking like a blunder. Qa3 How shall White prevent the threatening mate? Will the escape artist Houdini find a way out? 30.Qb5+! This queen sacrifice is pretty and necessary, too. By hxg7 White will get enough counter play later on. 30.Na4? is no longer playable on account of 0-0! 31.Rxg7+ or 31.hxg7 Rb8 threatening R or Qxa4! 31...Kh8 32.Qc3 Rxa4!-+ 30...Bd7 After 30...Rxb5 31.Bxb5+ Kd8 31...Kf8? 32.hxg7+± 32.hxg7 Rg8 33.h5 Bxb3! 34.h6! 34.cxb3? Qxb3+ 35.Kc1 Qxc3+ 36.Kd1 Qb3+ 34...Kc7 35.Be8! the game is even, e. g. Bc4 35...Rxe8? 36.Nb5+ 36.h7 Rxe8 37.g8Q Qb4+ 38.Kc1! Qxc3 39.Qxe8 Qa1+ 40.Kd2 Qd4+ and Black has a perpetual check. 31.Qxa5! Instead of this second queen sacrifice White might have tried 31.Qb8+ Bd8 32.Na4 , but after g6! Black seems to be a little better, e. g. 33.c3 Bxa4 34.bxa4 Qxa4 34...Rxa4? 35.Bb5+ 35.Kc1 Rxh6 36.Kd2!∞ 36.Qxd6? Qa1+ 37.Kc2 Ra2+ 38.Kd3 Qb2-+ 31...Qxa5 32.hxg7 Rg8 33.h5! Who is playing with the white pieces, Houdini or the ingenious Paul Morphy? Now Nc3 is en prise. Instead of sacrificing the knight White might have tried 33.Kb2 d5! 34.Ra1! 34.Nxd5? Ba3+ 35.Kb1 Be6 36.Nf6+ Kd8 37.Rd1+ 37.Nxg8? Qc3 37...Kc7 38.Rd3 Ra8! 39.g8Q Bf8-+ 34...Qc5 35.Rhg1∞ 33...Qxc3 34.h6 Within only five moves Houdini has completely turned the tables. White's two united passed pawns look devastating. How shall Black prevent 35.h7? Bf5! Critter has found the only saving move. The bishop is lost, but the black King can move to d7 preparing Ra8 or Rb8. 35.exf5 After 35.Bb5+ Kd8 36.exf5 Kc7! 37.h7 Ra8 38.Ba4 Rxa4! 39.bxa4 Qb4+ 40.Kc1 Qf4+ White cannot avoid the perpetual: 41.Kd1 Qd4+ 42.Ke2 Qc4+! 43.Kf2 Qd4+ 44.Kg3 Qf4+ 45.Kh3 Qxf3+ 46.Kh2 Qf2+ 47.Rg2 Qh4+ 48.Kg1 Qe1+ , etc. 35...Kd7 36.Rg4! After 36.h7 Black once again has a perpetual check: Ra8! 37.Bb5+ Kc7 38.Ba4 Rxa4! 39.bxa4 Qb4+ , etc. 36...Bf6? This is a decisive mistake. Critter could have saved the game by playing 36...Rb8! 37.g8Q Rxb3+! 38.cxb3 Qxb3+ 39.Kc1 Qc3+ 40.Kd1 Qa1+! 41.Kc2 41.Kd2 Qa5+ 41...Qa2+! 42.Kd3 Qa3+ 43.Kd2 Qa5+! 44.Ke3 Qc5+ 45.Kd3 Qa3+ , and again White cannot get out of the perpetual check. 37.Bb5+! Ke7 38.h7 Ra8 At first sight it looks as if Critter had successfully defended the position. Black is threatening mate and White is forced to play either Ba4 or Ra4. 39.g8N+! An important intermediate move and the only one to win the game. After 39.Ba4 Black could have saved the game by two pretty sacrifices: Bxg7! 40.Rxg7 Rxa4! 41.bxa4 Qb4+ 42.Kc1 Qf4+ 43.Kb2 43.Kd1? Qxf3+ 43...Qb4+ , and again White cannot avoid a perpetual check White cannot win by 39.Ra4 either, e. g. Rxa4 40.g8N+ Kf8 41.bxa4 Bh8 42.f6 d5!∞ 39...Kd8 39...Kf8 40.Ba4 Bh8 41.f6!+- 40.Ba4 Bh8 41.Nh6! Qxf3 42.Rhg1 Ke7 43.Rc4 Qb7 43...e4? 44.Rc7+! Kf6 44...Kd8? 45.Rd7+ 45.Ng4+ Kxf5 or 45...Kg5 46.Ne5+ 46.Rxf7+ winning the queen. 44.Ng8+ Kd8 45.f6! 45.Bc6? is no good because of Qa7! 46.Bxa8 Qxg1+ 45...d5 46.Ne7!? Once again prefering a spectacular sacrifice instead of playing the solid move 46.Rd1+- 46...dxc4 46...Bxf6 leads to a pretty variation: 47.Nxd5! Qxd5 48.Rg8+ Ke7 49.Rc7+ Ke6 49...Kd6? 50.Rd7+ 50.Rc6+ Qxc6! 50...Kf5? 51.Rxf6+! Kxf6 52.h8Q+ 50...Ke7? 51.Rxa8+- 51.Bxc6 Ra6 52.Be4+- 47.Rd1+ Kc7 48.Rd7+ Kb6 49.Rxb7+ Kxb7 50.Bc6+ Kb8 51.Bxa8 Kxa8 52.Ng8 White is only one pawn up, but the endgame is hopeless for Black. c3 After 52...cxb3 53.cxb3 Kb7 54.Kc2 Kb6 55.Kd3 Kc5 56.Ke4 Kd6 57.b4+- is winning. 53.b4 e4 53...Kb7 54.Ka2 e4 55.Kb3+- 54.Kc1! Kb7 55.Kd1 Kc6 and Black resigned at the same time. The following moves will show you why: 56.Ke2 Kb5 57.Ke3 Kxb4 58.Nh6! Bxf6 59.Nxf7 Ka3 60.Kxe4 Kb2 61.Kd3 Bg7 62.h8Q Bxh8 63.Nxh8 Ka3 64.Kxc3 with an easy win even without the white knight. 1–0
  • Start an analysis engine:
  • Try maximizing the board:
  • Use the four cursor keys to replay the game. Make moves to analyse yourself.
  • Press Ctrl-B to rotate the board.
  • Drag the split bars between window panes.
  • Download&Clip PGN/GIF/FEN/QR Codes. Share the game.
  • Games viewed here will automatically be stored in your cloud clipboard (if you are logged in). Use the cloud clipboard also in ChessBase.
  • Create an account to access the games cloud.
WhiteEloWBlackEloBResYearECOEventRnd
Houdini 1 64-bit3083Critter 0.90 64-bit30491–02020B90Test match 40/47

The secret of Houdini's strength

Regarding the origins of the program, it is rumored that Robert Houdart took the free public domain engine RobboLito as a basis. He himself stated on his homepage that he used many ideas from the free open source programs IPPOLIT/RobboLito, Stockfish and Crafty. (c) This probably explains the very high playing strength right from the beginning. In fact Robert Houdart did much more than using ideas of others. He added a lot of improvements to his new brainchild as you can easily see for yourself with the help of a little test:

(White to move)

 
New ...
Open...
Share...
Layout...
Flip Board
Settings
MoveNResultEloPlayers
Replay and check the LiveBook here
1.Be1 Kb5 2.Bf2 Kc6 3.Kg5 Kd7 4.Kxg6 Kc8! 5.Kxf5 White can cut off the black King from reaching b8 by playing 5.Bg3 , but he cannot achieve more than a draw: f4! 6.Bxf4 Kd7 7.Kf5 Kc8 8.Ke6 Kd8 9.Kd6 Kc8! 10.Ke6 Kd8= 5...Kb8 6.Ke6 Ka8 7.Kd7 Kb8 8.Bg3+ Ka8 Attacking the black pawn would result in a stalemate. Therefore White cannot win.
  • Start an analysis engine:
  • Try maximizing the board:
  • Use the four cursor keys to replay the game. Make moves to analyse yourself.
  • Press Ctrl-B to rotate the board.
  • Drag the split bars between window panes.
  • Download&Clip PGN/GIF/FEN/QR Codes. Share the game.
  • Games viewed here will automatically be stored in your cloud clipboard (if you are logged in). Use the cloud clipboard also in ChessBase.
  • Create an account to access the games cloud.
WhiteEloWBlackEloBResYearECOEventRnd
White-Black- Test position, White to move

This position was given by correspondence chess grandmaster Arno Nickel. (d) RobboLito 0.085g3 will show you a score of + 4.26 in favour of White (38 plies search depth), while Houdini 1 immediately recognizes that this position is balanced (+ 0.03). This is quite correct, because Black need not care about his pawns on the King's side at all. By simply moving his King via b5, c6 and d7 to c8, he can easily prevent White from winning the game. Try it out and you'll see that White will never get beyond a stalemate. This example proves that even the very first version of Houdini from May 2010 was able to master this difficult endgame without using any tablebases.

Computer scientist and chess player

Robert Houdart was born in 1968. He is a computer scientist and professional software developer with a FIDE ELO rating of 2101. (e) These are good qualifications to optimize a chess program more and more. The constant ongoing development of Houdini 1 soon paid off: Houdini 1.03a, released two months later in July 2010, had an ELO rating of 3116 ELO  in the CCRL 40/4 list (64-bit, 1CPU). The last free Houdini version 1.5a from January 2011 gained 54 more ELO points and thus reached 3170. For comparison: The strongest Rybka version 4.1 had an ELO rating of 3108, while the commercial Houdini versions 2, 3 and 4 climbed up to 3202, 3217 and 3240 ELO. In the three and a half years between May 2010 (Houdini 1) and November 2013 (Houdini 4) Robert Houdart was thus able to improve Houdini from about 3083 ELO to 3240 ELO (64-bit, 1CPU). This is a plus of almost 160 ELO. Running on four processors (4CPU) Houdini 4 even reached 3332 ELO. (f)

Many additional options

Besides an increase in playing strength, Robert Houdart added several additional features to his program. With Houdini 1 you could already set the number of threads, the so-called split depth (search depth, from which the calculations run over two or more processors), ponder (calculating while the opponent has to move) and how much hash size the engine may use. In the following versions more options were added gradually. Since Houdini 1.03 the engine can display several variations at the same time when analyzing positions (Multi-PV). With version 1.5 endgame tablebases were included. Robert Houdart also added a so-called contempt factor to his program to avoid unnecessary draws against weaker opponents. In spite of a slightly worse position Houdini from now on played on against weaker opponents, which naturally lead to a lower draw rate. Contempt is deactivated by default during analysis. This is important to obtain an objective evaluation of the position. In the realm of computer chess all of these improvements pushed the program to the top of the world in just seven months.

Houdini, the new number one

Since 2011 the TCEC tournaments are considered the real kingmakers in computer chess. (g) As a consequence of the controversial disqualification of the hitherto strongest chess program Rybka the ICGA (International Computer Games Association) had practically devaluated their own competitions, since a tournament without the best available chess programs can hardly be sold to the public as the real computer chess world championship. This reminds a little bit of the situation of the FIDE world champions in comparison with the classical" world champions Kasparov and Kramnik during the years 1993 to 2006. Of course the human world champion title is considered much more important by the public than the world champion title of any chess program. Anyway, in the TCEC computer tournaments Houdini dominated over all of the world's strongest chess engines from the very beginning.

The TCEC tournaments are very reliable, because each chess program runs on  identical hardware. After some preliminary rounds the two best engines face each other in a so called superfinal where each opening is played twice with colours reversed thus levelling the chances.

Season 1" was held from December 2010 to February 2011. In the superfinal Houdini 1.5a won clearly against Rybka 4 (+ 12, - 5, = 23).

Season 2" was played from February to April 2011. In the superfinal Rybka 4.1 performed slightly better than its predecessor. The final result was +9, -5, =26 in favour of Houdini 1.5a.

Season 3 was not finished. This is why there was no TCEC champion in 2012.

Season 4 took place from January to May 2013. In the superfinal Houdini 3 won against Stockfish 250413 (+6, -4, =38).

The long time waiting for Houdini 5

You can already see from the narrow victory over Stockfish in May 2013 that the competitors gradually managed to get closer to Houdini. In fact Komodo and Stockfish dominated during the next few TCEC tournaments. Komodo won seasons 5, 7 and 8, while Stockfish was succesful in season 6. One reason for this is that Robert Houdart had not published a new version since the release of Houdini 4 in November 2013. Not until three years later, in November 2016, Houdini 5 was released and made it straight to the superfinal of season 9, but lost against Stockfish 8 with +8, -17, =75. In season 10 which was played from October to December 2017, the new Houdini 6 again made it to the superfinal and won convincingly against Komodo (+15, -9, =76).

The final version of Houdini

Houdini 6, the TCEC champion of season 10 and runner-up of season 11, is about 200 ELO stronger than Houdini 4. This is a level of play that should leave nothing to be desired for a long time to come. Will there still be a Houdini 7 one day? On my request Robert Houdart informed me that he has been working on completely different projects since October 2017 and is currently not involved in chess programming. So the current Houdini version will probably remain the last one for a long time.

Houdini's greatest strength

Houdini's greatest strength was and is the correct evaluation of positions which is decisively important not only for practical play, but especially for analysis. In an article The brave new world of chess engines correspondence chess grandmaster Arno Nickel wrote about Robert Houdart and his chess program Houdini:

He is particularly proud of his evaluation function, which indeed seems to be the most moderate - and often perhaps even the most realistic. Houdini also has a very intelligent and efficient search function that allows the engine to find strong positional continuations more often and faster than other programs". (h)

For good reason Houdini is very popular especially among correspondence chess players who often have to analyse complicated positions very thoroughly. Indeed, no analyst can do without a correct and objective evaluation. Therefore Houdini is and remains an indispensable analysis tool today.

One can only congratulate the programmer Robert Houdart on 10 years of Houdini and hope that one day he will continue the development and surprise the chess world with Houdini 7. It would also be desirable if he would release some of the older Houdini versions for download on his homepage. At the moment only Houdini 1.5a can be found there.

References:
(a) The wayback machine archived the homepage where Houdini 1 was presented to the public:

https://web.archive.org/web/20100519120411/http://www.cruxis.com/chess/houdini.htm

(b) https://de.wikipedia.org/wiki/Harry_Houdini

(c) Without many ideas from the excellent open source chess engines Ippolito/Robbolito, Stockfish and Crafty (in that order), Houdini would not nearly be as strong as it is now."

https://web.archive.org/web/20100519120411/http://www.cruxis.com/chess/houdini.htm

(d) Arno Nickel, Die schöne neue Welt der Schachengines", published in Schach", edition from March 2012, p. 45

(e) http://ratings.fide.com/card.phtml?event=202185      

https://www.chessprogramming.org/Robert_Houdart

(f) In the 40/40 rankings, the earlier Houdini version 1.03a was not tested, but in 40/4 it was, so the ELO ratings in this article refer to a time control of 4 minutes for 40 moves (repeating) and to the 64-bit versions using 1 CPU. With 4 processors (4CPU) the ELO ratings are correspondingly higher. With Houdini 6, for example, this would mean 3517 ELO (4CPU) compared to 3444 or 3445 (1CPU). Unfortunately, the proven 40/4 time control is no longer used by the CCRL testers for several months. Instead they now play blitz games with 2 minutes for the whole game + 1 second increment on faster hardware. The last archived 40/4 rankings I could find are from October and November 2019 respectively:

https://web.archive.org/web/20191029004707/https://ccrl.chessdom.com/ccrl/404/rating_list_all.html

https://web.archive.org/web/20191121115328/https://ccrl.chessdom.com/ccrl/404/cgi/compare_engines.cgi?class=Single-CPU+engines&print=Rating+list&print=Results+table&print=LOS+table&table_size=12&cross_tables_for_best_versions_only=1

(g) TCEC stands for Thoresen Chess Engines Competition" or more recently Top Chess Engine Championship"

https://en.wikipedia.org/wiki/Top_Chess_Engine_Championship

(h) Arno Nickel, Die schöne neue Welt der Schachengines", published in Schach", edition from February 2012, p. 61

Houdini 6 continues where its predecessor left off, and adds solid 60 Elo points to this formidable engine, once again making Houdini the strongest chess program currently available on the market.

Or, to compare:

The most popular chess program offers you everything you will need as a dedicated chess enthusiast, with innovative training methods for amateurs and professionals alike.

and here's the new version.

Last year Komodo won the world championship title on two occasions and can call itself "2019 World Computer Chess Champion" and "2019 World Chess Software Champion". And the current Komodo 14 has been clearly improved over its predecessor!


Stephan is a passionate collector of chess books and for years he has been successfully playing as an amateur for his German club. The former musician and comedian works as a freelance journalist and author in Berlin and in the Franconian village Hiltpoltstein.

Discuss

Rules for reader comments

 
 

Not registered yet? Register

We use cookies and comparable technologies to provide certain functions, to improve the user experience and to offer interest-oriented content. Depending on their intended use, analysis cookies and marketing cookies may be used in addition to technically required cookies. Here you can make detailed settings or revoke your consent (if necessary partially) with effect for the future. Further information can be found in our data protection declaration.