Interview with Robert Houdart, author of Houdini

My main academic background actually lies in structural mechanics and heat transfer. I graduated from university in 1991 and then went on to work for Tractebel Energy Engineering, in the design of the Belgian and Swedish Nuclear Power Plants. For eight years my job was to design nuclear piping systems that were resistant to earthquake and airplane crashes. For that we developed some new computational methods with fancy names like "Twin Mode Rotation" and "Adaptive Modal Summation".

After that period I moved on to a completely different sector, business software known as "Customer Relationship Management". Since 1999 I’ve developed two CRM applications "eLink" and "Efficy" that are used today by some 25,000 people, mostly in Europe. While I've never had any formal programming education, I have always been programming, since I was 13 or 14 years old. It's in my blood and I've always given a "programming" touch to whatever professional activity I was occupied in.

Since the middle of the 1980s I've more or less constantly been writing chess related software as a hobby – be it chess engines or a program to print chess diagrams on a matrix printer for the magazine of the Leuven chess club. I've written several private engines, mostly not too strong and not very complete.

The work on Houdini started in 2009. At the time I was waiting for the mirror of a big telescope project I was engaged in. The mirror was significantly delayed, and I was in a way looking for an outlet for the creative energy that was building up. I started with this idea to build the best chess engine that I could – and I was helped a lot by the open culture that has come with the Internet. You know, two decades ago you had to invent every part of a chess engine from zero (and I've done my fair share of that), but today we're in a situation where techniques, ideas and examples are readily available on the Internet. You can call it a coming of age of the computer chess scene – as an engine author you're no longer obliged to sit in your corner reinventing the wheel. The computer chess Wikipedia, some strong open source engines, and discussions on Internet forums about chess programming techniques and ideas make the design and development of a strong engine a lot easier than, say, twenty years ago.

Two key concepts: good evaluation and even better selectivity. It's self-evident that good evaluation of a position is the key for a good chess engine. Houdini is probably the best engine to evaluate piece mobility and space control on the board. It has a very balanced evaluation in all phases of the game. I've always tried to link Houdini's evaluation to a probability of winning the game. For example, when Houdini 3 shows a +1.00 evaluation in the middle game it has an 80% chance to win the game against an equally strong opponent at blitz time controls. I believe this is a very useful aspect of the engine.

Selectivity is another key feature in Houdini. Just like a human player, an engine doesn't look at all the moves to the same depth. Potentially good moves are examined exhaustively, whereas apparently weaker moves are only given a quick, shallow look. Some moves are examined 40 or 50 plies deep, other only five. Houdini has a good ability to identify which moves in the position have some potential. It's similar to the instinct and experience of a strong human chess player – looking at just a handful of moves in a position, discarding nearly instantaneously and without thinking the 30 other moves.

This ability – or “instinct” if you like – has been significantly refined and optimized in each version. Compared to Houdini 1.03, Houdini 1.5a added more breadth to the search, identifying more potentially interesting moves. Houdini 2.0 added an improved tactical layer to the breadth of Houdini 1.5a, converting more of the potential of the "interesting" moves. Houdini 3 now brings additional depth to this whole search tree. This also explains why Houdini is already very strong at blitz time controls: its "instinct" makes it pick better moves, even when the time is very limited.

As I mentioned earlier, the Internet community is a great source of inspiration and the information that is now available in seconds would have taken ages to collect twenty years ago. Other than the Computer Chess Wiki, which is an awesome resource for any aspiring chess engine developer, I must credit the Stockfish open source engine, which was the inspiration for the multi-threaded implementation of Houdini, and the IPPOLIT open source engine that provided a whole array of search and evaluation techniques. The development effort is done entirely by myself, but I'm supported by people from around the world that send ideas for improvement, very often positions in which Houdini doesn't perform well. Some fans even have donated hardware for engine testing. It's amazing how supportive the community has been over the past two years.

For the development it helps that I’ve been a decent chess player myself (around 2250 during the 1990s). It assists in assessing weaknesses and to device improved evaluation terms to correct them.

Houdini builds on the Rybka legacy in the sense that it provides accurate and useful assessments in nearly every chess position that occurs in games. It’s great for opening preparation and post-mortem analysis. It will show you precisely (and without mercy) what went well and what went wrong in your games – from opening to end game.

Houdini is very good in defending difficult positions (even against other top engines), and its tactical prowess is literally super-human. I’ve always tried to keep the style of Houdini quite aggressive – it consistently has the lowest draw rate of all the engines in the rating lists. Against 3000+ opponents Houdini has a lower draw rate that what you can observe in the average human GM tournament.

The most impressive performance I’ve seen from Houdini was in the TCEC match against Rybka about 18 months ago. It was short after the release of Houdini 1.5a, and Martin Thoresen organized this fun match – kind of non-official World Champion – on some very strong computer hardware using classical time control. The games could be followed live on the Internet, and so I was watching how this game developed – Houdini sacrificed a pawn, two pawn, three pawns in a queen-less middle game, to end up winning the game in convincing fashion. During the game I wasn’t sure at all that what we were seeing was a brilliant game – and not some obscure bugs I’d left in the engine… I don’t think any other engine could have played this game the way Houdini did. The blend of aggressiveness with superb tactical and positional evaluation makes this probably the best engine game ever – especially against a former five time Computer Chess World Champion. There’s this very interesting video that has been published about the game, which expresses very well the awe and wonder of the performance.

Here is the game to replay on our JavaScript board:

I love chess and programming, so what better way to express this than in creating and improving a chess engine that is used by thousands of chess fans over the world. After the very demanding release of Houdini 3 I do hope to take some time off for the family and to go out observing stars and planets with the big telescope. But early 2013 I’ll start again working for Houdini 4.

From the initial public version of May 2010 I've now been able to add about 150 Elo strength improvement. That's over 50 points per year, a rate any human GM would be delighted with. It's gratifying to look back at this evolution, as every Elo point that is won represents a lot of creativity and hard work. I certainly did not expect in 2009 that Houdini would be dominating the chess engine scene for several years, but now that we’re at this point I’m happy to continue the dream.

Copyright Houdart/ChessBase
Pictures of Robert Houdart by Jan Lagrain of Schaakfabriek

An unbeatable combination: the world's strongest chess engine
running on the world's finest chess interface

Give in to the magic of this program! Only two years ago the Houdini chess engine stormed to the top of the ranking lists, and since then has been the uncontested number one chess engine in the world. The secret of its success: Houdini introduces pure magic into the game of chess! The engine of Belgian programmer Robert Houdart finds tricks in places where the other engines can see nothing.

The new version, Houdini 3, goes even further, providing the chess world with yet another increase in playing strength: at least 50 Elo points, thanks to a host of improvements in its search algorithms. They manifest themselves in different ways, in various phases of the game. In the opening Houdini 3 demonstrates an even more subtle understanding of space and activity; in the middlegame the program spots quicker than before when pieces are in danger of being dominated; and in the endgame the right evaluations and solutions to problems are found much sooner thanks to a faster, deeper search. During the course of this new development and the fine tuning of the engine Houdini played, believe it or not, ten million test games!

Houdini 3 is supplied with the latest Deep Fritz 13 chess interface and thus puts at your disposal all the training and analysis functions of the world's premium chess program. Included in the package are a one-year classic membership to the chess server Playchess.com, online access to the world’s largest analysis database “Let’s Check”* and the use of the ChessBase Engine Cloud.

Houdini 3 includes:

• The Houdini 3 engine
• The DeepFritz 13 user interface in Windows Office 2010 standard
• The DeepFritz 13 database management system
• Classic membership of Playchess.com for twelve months
• The ChessBase Engine Cloud
• The ChessBase "Let’s Check" function (until 31.12.2015)
• A database with over 1.5 million games

Houdini 3 Standard multiprocessor version
Supports up to six cores and four GB of hash + 12 months Playchess.com (classic)
ISBN 978-3-86681-336-6 – EAN 4027975007229. Price: 79.90€

Houdini 3 Pro multiprocessor version
Supports up to 32 cores and 256 GB of hash + 12 months Playchess.com (classic)
ISBN 978-3-86681-337-3 – EAN 402797500723-6. Price: 99.90€

System requirements: Minimum: Pentium III 1 GHz, 1 GB RAM, Windows Vista, XP (Service Pack 3), DirectX9 graphics card with 256 MB RAM, DVD-ROM drive, Windows Media Player 9 and Internet access to activate the program, Playchess.com, Let’s Check, Engine Cloud and updates. Recommended: PC Intel Core i7, 2.8 GHz, 4 GB RAM, Windows 7 (64 bit) or Windows 8 (64 bit), DirectX10 graphics card (or compatible) with 512 MB RAM or more, 100% DirectX10 compatible sound card, Windows Media Player 11, DVD ROM drive and Internet access to activate the program, Playchess.com, Let’s Check, Engine Cloud and updates.

Houdinin 3 will be available next week – watch out for further announcements