Fair play in engine chess

Recently we came across a tweet by a Houdini 5 user who apparently was unhappy about the performance of his new engine: Houdini 5 had lost a test match against Komodo 10 with the clear result of 1-9 and faring much worse than the old Houdini 3 of the user:

Does anyone have a comment?@ChessBase @NewInChess @chessdom pic.twitter.com/xuGX6WT5bX

— Sprarty (@spartacuswines) December 11, 2016

How is that possible? What do these statistics say? And what do you have to keep in mind when letting engines play a match?

ChessBase Wiki Do you have questions about Fritz or your ChessBase program? What are hash tables and how do you set them? How do I use the update service for the ChessBase Mega Database? How to activate or deactivate your Fritz trainers? Where do I find a manual for the ChessBase accounts? If you have a question about our products, check the new ChessBase Wiki! Here you do find answers, instructions and solutions. Have a look at the ChessBase Wiki

First of all: the result of the engine match mentioned above might be surprisingly clear but ten games are simply not enough to make any reliable statements about the playing strength of the engines. Ten games are statistically completely irrelevant. Apart from that is is plausible that mistakes were made during the configuration of the engine which slowed down Houdini 5 completely or at least gave Houdini 5 a serious disadvantage. In the following we would like to show you the most important possible sources of error:

1. The Houdini 5 Engine has not been activated yet

The online-activation is the basic requirement to let the engine calculate at full speed. If you start the engine without activation a warning appears, however if you skip the activation, Houdini 5 still starts - but with drastically reduced computing power. It is indeed quite possible that the engine was not activated in the case of the user mentioned above and that this caused the weak performance of Houdini 5.

If you get the following note when starting the program

you should immediately activate the engine with your program code!

Moreover, there are a number of additional settings you should keep in mind when using engines or letting engines play matches.

2. Different opening books or "Learn" activated

The settings dialogue for an engine match asks for several entries. First, you have to select the two engines at "White" and "Black" with the "Define" button:

After having selected the program from the list of engines you go to "Advanced".

Here you e.g. select the opening book which the engine should use at the beginning of the game and you set the "learning value". It is important that both engines use the same settings - otherwise one of the engines will, for example, play without opening book.

3. The allocated hash tables are too big for the memory of the computer

The playing strength of the programs depends to a large extent on the size of the hash tables. These are tables in the central memory in which positions the engine has calculated are buffered. Before the program starts to calculate a variation tree it checks the hash tables to see whether this position is known and whether it might resort to these previous calculations which saves a considerable amount of time. You also define the size of the hash tables under "Advanced":

Most programs have a greater playing strength if their hash tables are larger. Therefore you should always set the hash tables to the possible maximum. The dialogue "load engine" offers a recommended value based on the available RAM of your system.

And here you have to be careful! If you extend this value it might happen that Windows has to "swap" for a long time before the memory you want is available. During "swapping" the hard drive runs and the engine can only calculate at a crawl. During the setting of an engine match you should therefore keep in mind to allocate both engines hash tables of the same size and you should take care that the sum of both hash tables is below the shown maximum value.

4. Too many threads allocated

For optimizing the computing power of the engine the number of threads is crucial. Call up "engine parameter" at the "extended" settings:

For an engine match you have to take care not to allocate more threads than the system has available. One example: on a quad system you cannot let both engine calculate with "Permanant Brain" and 4 threads each. Here a maximum of 2 threads with "Permanent Brain" or 4 threads without are possible. As in both cases all eight threads are busy the programmers recommend to leave one thread for the GUI and the system. And fairness requires - similar to the hash tables - to allocate both engines the same number of threads!

Moreover, there are other constellations that might make the engine perform worse. The tablebases might, for example, slow down an engine if not enough memory is available. The ChessBase Wiki has a lot of tips and tricks to make your engine perform better and it has answers to the most often asked questions about ChessBase and Fritz!