Fair play in engine chess

by ChessBase
12/30/2016 – About a month ago Houdini 5 appeared. But how strong is the new engine by Robert Houdart? Is it even stronger than Komodo 10? A lot of users want to find out and use the Fritz 15 graphical user interface to let engines play a match against each other. We show you what to keep in mind to give both engines a fair chance in such a match.

ChessBase 17 - Mega package - Edition 2024 ChessBase 17 - Mega package - Edition 2024

It is the program of choice for anyone who loves the game and wants to know more about it. Start your personal success story with ChessBase and enjoy the game even more.

More...

Recently we came across a tweet by a Houdini 5 user who apparently was unhappy about the performance of his new engine: Houdini 5 had lost a test match against Komodo 10 with the clear result of 1-9 and faring much worse than the old Houdini 3 of the user:

 

How is that possible? What do these statistics say? And what do you have to keep in mind when letting engines play a match?

ChessBase Wiki

Do you have questions about Fritz or your ChessBase program? What are hash tables and how do you set them? How do I use the update service for the ChessBase Mega Database? How to activate or deactivate your Fritz trainers? Where do I find a manual for the ChessBase accounts? If you have a question about our products, check the new ChessBase Wiki! Here you do find answers, instructions and solutions. Have a look at the ChessBase Wiki 

First of all: the result of the engine match mentioned above might be surprisingly clear but ten games are simply not enough to make any reliable statements about the playing strength of the engines. Ten games are statistically completely irrelevant. Apart from that is is plausible that mistakes were made during the configuration of the engine which slowed down Houdini 5 completely or at least gave Houdini 5 a serious disadvantage. In the following we would like to show you the most important possible sources of error:

1. The Houdini 5 Engine has not been activated yet

The online-activation is the basic requirement to let the engine calculate at full speed. If you start the engine without activation a warning appears, however if you skip the activation, Houdini 5 still starts - but with drastically reduced computing power. It is indeed quite possible that the engine was not activated in the case of the user mentioned above and that this caused the weak performance of Houdini 5.

If you get the following note when starting the program

you should immediately activate the engine with your program code!

Moreover, there are a number of additional settings you should keep in mind when using engines or letting engines play matches.

2. Different opening books or "Learn" activated

The settings dialogue for an engine match asks for several entries. First, you have to select the two engines at "White" and "Black" with the "Define" button:

After having selected the program from the list of engines you go to "Advanced".

Here you e.g. select the opening book which the engine should use at the beginning of the game and you set the "learning value". It is important that both engines use the same settings - otherwise one of the engines will, for example, play without opening book.

3. The allocated hash tables are too big for the memory of the computer

The playing strength of the programs depends to a large extent on the size of the hash tables. These are tables in the central memory in which positions the engine has calculated are buffered. Before the program starts to calculate a variation tree it checks the hash tables to see whether this position is known and whether it might resort to these previous calculations which saves a considerable amount of time. You also define the size of the hash tables under "Advanced":

Most programs have a greater playing strength if their hash tables are larger. Therefore you should always set the hash tables to the possible maximum. The dialogue "load engine" offers a recommended value based on the available RAM of your system.

And here you have to be careful! If you extend this value it might happen that Windows has to "swap" for a long time before the memory you want is available. During "swapping" the hard drive runs and the engine can only calculate at a crawl. During the setting of an engine match you should therefore keep in mind to allocate both engines hash tables of the same size and you should take care that the sum of both hash tables is below the shown maximum value.

4. Too many threads allocated

For optimizing the computing power of the engine the number of threads is crucial. Call up "engine parameter" at the "extended" settings:

For an engine match you have to take care not to allocate more threads than the system has available. One example: on a quad system you cannot let both engine calculate with "Permanant Brain" and 4 threads each. Here a maximum of 2 threads with "Permanent Brain" or 4 threads without are possible. As in both cases all eight threads are busy the programmers recommend to leave one thread for the GUI and the system. And fairness requires - similar to the hash tables - to allocate both engines the same number of threads!

Moreover, there are other constellations that might make the engine perform worse. The tablebases might, for example, slow down an engine if not enough memory is available. The ChessBase Wiki has a lot of tips and tricks to make your engine perform better and it has answers to the most often asked questions about ChessBase and Fritz!

 

Reports about chess: tournaments, championships, portraits, interviews, World Championships, product launches and more.

Discuss

Rules for reader comments

 
 

Not registered yet? Register

moryaccool moryaccool 6/6/2018 07:37
How to avoid time loss in engine match played on komodo 12 GUI even though after having significant increment time control ? it does gets over most of the time. I want the autoflag feature to be off and avoid time loss. Please help it.
erikbergren erikbergren 1/3/2017 07:17
If Stockfish had won 9 out of the first 10 games at the TCEC this year, it would have seemed quite significant statistically to the programmers watching, I would guess none of them would say " Ten games are statistically completely irrelevant" as this article states. It is *possible* that the other engine could catch up as more games are played, but that does not mean a 9-1 lead at the start is "irrelevant".

I doubt that "programmers recommend to leave one thread for the GUI and the system" as the author states, because on every computer system I've tried (2 to 8 cores) the nodes per second rate increases by nearly the core count, so leaving one core for the system would drastically reduce the calculation speed if there are few cores in the system.
I might agree if maybe the author(or "programmers") meant that the two engine's play would be more *fair*
(but less strong) with one core left dedicated to the operating system, I guess so the system is not forced to steal unevenly from one of the engine's resources, I don't know enough to be sure about that, but TCEC assigned all of the system's 22 cores to the programs and none to the system and they thought play would be fair.

It was interesting to hear from this article that Houdini had to be activated to play strongly.
billvan61 billvan61 1/2/2017 02:23
God bless Chessbase and the coverage it provides on this site, but it is not a charity. Nor does it pretend to be. It has its interests to serve, and being impartial when it comes to the playing strengths of computer engines is not one of them. This fact doesn't make it diabolical. It just means that we must keep this in mind. A look at the webanalytics for Chessbase (try Alexa.com) reveals further links between profit and content. It explains, for instance, for the many articles about Indian chess, chessplayers, and tournaments.

psilocybin psilocybin 1/1/2017 04:04
Who is the author of this article?
jsaldea12 jsaldea12 1/1/2017 03:49
CORRECTION: MAKER, NOT MASKER.
jsaldea12 jsaldea12 1/1/2017 03:48
CONGRATS TO MASKER OF kOMODO, THE WORLD NO ONE CHESS ENGINE NO HUMAN CAN DEFEAT IN MATCH...
kf2wins kf2wins 1/1/2017 02:59
haha this is funny, why avoid stockfish in this match. i guess stockfish is unbeatable and will destroy these weaker engines. i guess a free engine is hard to talk about, unless there is a way to make money out of it.
klklkl klklkl 12/31/2016 10:21
Ludicrous article to steer away from all reference to STOCKFISH. Apparently an engine is only worth mentioning if Chessbase can make money selling it.
jajalamapratapri jajalamapratapri 12/31/2016 08:32
Chessbase promotes Houdini 5 because it sells Houdini 5 which is hard to do when Stockfish 8 is stronger and is free. Still H5 won some games at the TCEC final match so it is better in some positions. Also the "tactical mode" of H5 is very cool.
Denix Denix 12/31/2016 08:03
Happy New Year!
Timothy Chow Timothy Chow 12/31/2016 05:27
It is misleading to say that "ten games are statistically completely irrelevant." A score of 9-1 after ten games has a very small probability of occurring just by chance. Suppose, for example, that two players have a 50% chance of drawing any particular game and that each player has a 25% chance of winning any particular game. Then the probability of an outcome as lopsided as 9-1 or 1-9 is only about 0.0004.
cythlord cythlord 12/31/2016 01:31
I don't understand why chessbase promotes Houdini 5 so much when it was recently smashed by Stockfish 8 in TCEC season 9.
rkjulian rkjulian 12/31/2016 02:55
What we really want is an engine that plays like a person. Computer vs. Computer comparisons don't help any more than an F1 car comparison helps a marathon runner.
vinniethepooh vinniethepooh 12/30/2016 06:01
Houdini 5 is strong agreed, mabye a bit more than Komodo 10, but my Houdini 3 is even more so! It often finds out combinations in 4-5 seconds which my Komodo 10 takes around a minute! Really confusing, please answer. (I am not talking about engine match, I am talking about engine strength)
Mr TambourineMan Mr TambourineMan 12/30/2016 05:35
If you want the best analysis you should give the engines plenty of time. When I have done so in the past I always got the same result: Komodo. Likewise, if you want to run with a super short reflection time: Houdini. If you want to keep on with chess in the usual way and not as a scientist or rocket scientists, and if you want to keep your money in the bank account, use: Stockfish. But the most fun playing style and the best level may be however, when playing against the old table chess computers! Try a Saitek, Novag, Mephisto or whatever...
CostaMaison3 CostaMaison3 12/30/2016 03:12
Did you hear about Stockfish? It is considered the strongest engine.
1