Houdini wins TCEC Superfinal

Houdini 53-47 Komodo

It's a Computer Chess World Championship, in all but name. Two preliminary tournaments (stage 1 and stage 2) establish the current crean of the crop in computer chess, and the top two then play an epic 100-game final match. This final is played with special rules to level the field as far as possible.

It seems as if Stockfish, Komodo and Houdini are currently playing with almost equal strength. But this time it was the Belgian program Houdini (named after the American stage magician and illusionist) and the US program Komodo that made it to the finals. Houdini needed no "great escape", it won decisively.

During the match Robert Houdart, Mark Leffler and Larry Kaufman, the developers of Houdini and Komodo gave an interview in which they talked about the current situation of computer chess, developing methods and speculated about the future of computer chess. 

A short time after this interview they were caught up by the developments of the AlphaZero project. However, this coup by Google still leaves questions open. We reproduce the interview excerpts courtesy Chessdom.

Slaying the dragon

The superfinal of the TCEC season 10 ended on Thursday, December 7th, 2017, with a win by Houdini. The program which was developed by Robert Houdart from Belgium defeated Komodo 15-9, 76 games ended in a draw. The hardware on which the programs played was stronger than in any previous TCEC tournament 2 x Intel Xeon E5 2699 v4 @ 2.8 GHz with 64 GB RAM and 44 processors. Both programs ran on 44 processors and often reached search depths of 40 ply!

The time-limit was 2 hours for the whole game + 15 seconds/move. The opening book for the superfinal was by Jeroen Noomen, who also worked on the opening book of the former number one, Rybka. Every opening variation was played twice, once with White and once with Black to level the field. However, in the first two games the programs had to "think" on their own from the start — in these games no opening line was given. (a)

After a rather unspectacular draw in the first game, the second game developed into a crucial encounter: In the MacCutcheon-Variation of the French Defense with the pawn sacrifice 8.Bc1!? Houdini played with White and lost a second pawn on move 15. And though Komodo could consolidate the position and managed to bring its king to safety on the queenside the program found no way to shake off White's initiative. After lengthy maneuvering the game was almost drawn twice because no pawn had moved and no piece was taken for 50 moves. But then, on move 147, Komodo sacrificed a pawn on the kingside to prolong the game. Still, the remaining extra pawn did not suffice to win: draw after 185 moves! A frustating game for the spectators, but the engines of course don't care!

But after that things went splendidly for Houdini, almost as if this missed chance did have a "psychological" impact. After losses in the fourth, sixth, twelfth and fourteenth games Komodo, developed by Don Dailey (+ 2013), Larry Kaufman and Mark Lefler, quickly trailed 0-4, and in the further course of the match the top program was unable to reduce this gap to less than three points. Consequently, after 100 games Houdini was well ahead and won the match.

Relatively low drawing rate

Of the 100 games, 24 had a decisive result — a drawing rate of 76 percent. A surprisingly low number if you consider the strength of the two programs and probably also due Jeroen Noomen who selected the openings. All in all, a successful computer event with interesting games, live-transmission and lively discussions in the chat.

Houdini 6 Pro

Houdini 6 continues where its predecessor left off, and adds solid 60 Elo points to this formidable engine, once again making Houdini the strongest chess program currently available on the market.

Komodo Chess 11

The multiple computer chess world champion comes in a new and yet more powerful version. Thanks to co-author US Grandmaster Larry Kaufman, Komodo is the strategist among the top chess programs!

On November 21, 2017, during the sixth game of the superfinal, Nelson Hernandez (TCEC) conducted an interesting interview with the programmers (via skype). (c) They talked about a number of interesting topics:

Why Stockfish did not make it to the final

Larry Kaufman (Komodo) gave one possible reason why Stockfish, winner of the TCEC 9 the year before, did not make it to the superfinal:

"One thing I suspect as a reason that Stockfish didn’t make the final was because both Komodo and Houdini have a fairly strong form of contempt that helps us beat up on the weaker engines, and Stockfish doesn’t do that, and I imagine that cost them at least a half-point they needed."

Contempt is a setting that leads a chess program to avoid a draw even though it has the worse position — of course, within previously defined limits. This is useful when playing against weaker opponents but might couse unnecessary defeats when playing against opponents of equal or higher strength. On December 5th, the Stockfish developers reacted and changed the contempt settings for Stockfish. In tests against Stockfish 7 this brought the program additional 33 Elo-points. (c)

Larry Kaufman drew interesting comparisons between Komodo and Stockfish:

"I can tell you that my belief is that Komodo is better in most things than Stockfish. But there is something holding us back that has to do with search depth. We’ve been trying to figure it out for years, I don’t know what it is, but there is some reason we are not able to get the same search depth as Stockfish even if we tried to copy all their algorithms. We’ve tried experiments where we’ve tried to make Komodo act like Stockfish but it doesn’t work, and I don’t know why, but I feel that if we ever figure that out we’ll just be clearly #1. But almost every time we tried any idea from Stockfish in Komodo, nine times out of ten it makes Komodo weaker. I feel we’re doing most things better, but obviously not everything, or else we’d be stronger."

Houdini's greatest strength

About the strengths of his program Robert Houdart said:

"I think the feature in which Houdini is better than both Komodo and Stockfish, is mobility. It is piece mobility. Might be surprising, but I think the piece mobility is something which is different in Houdini than in other programs. We have some special handling of it and it results in a bit more aggressive chess actually, because you can see that when you look at the programs playing. You see that Houdini is a slightly more aggressive, tends to keep queens on the board, for instance, and I didn’t program anything particular for that, but it’s just the result of the rest of the program. It happens that because of that special mobility we can create a bit more…contempt-like…if I can again use that word, contempt-like behavior without actually having contempt. Behavior that favors keeping queens on the board, favors keeping pieces on the board, and favors aggressive play that tries to win the game. So that, I think, is one of the bigger differences between Houdini and the other programs."

Houston we have a problem

With a six point lead around the mid-way point of the match, the Komodo team noticed "a compiler bug" — a glitch in their program, and requested to update it to fix the problem, but the request was denied under tournament rules. They published a statement from the Komodo team:

We accept full responsibility for not discovering the problem before the start of the final. Although the bug has probably cost us some points it probably does not fully explain the current five point score deficit.

In short, for the program to be updated during the competition, the opponent would have to agree. Robert Houdart raised three sensible objections, including:

Houdini has been playing the whole TCEC Season 10 with the release version of Houdini 6 (in essence a version which is now 2½ months old). This is a deliberate choice; it assures that TCEC receives a high-quality, well tested build. With that in mind, it doesn’t make sense to allow other engines the luxury of correcting non-critical bugs in the middle of a TCEC stage.

Do chess computers change modern opening theory?

About the opening play of modern chess programs Robert Houdart said:

"That is one of the surprising things about the modern top engines, that they can play really good openings right from the start. You can start from the starting position and get a really good variation of a really good opening played by the engines. It’s not like 20 years ago where engines would play really weird openings and play really weird developing moves. That is no longer the case."

This indicates that opening theory in the years to come might be more influenced by computer games. Here it might be interesting to take a closer look at game 14 of the superfinal.

Links

• TCEC: Superfinal Houdini vs Komodo

Sources:

(a) tcec.chessdom.com

(b) Interview with Robert Houdart, Mark Lefler, and GM Larry Kaufman (Chessdom)

(c) abrok.eu/stockfish