Optimizing Fat Fritz, the top rated engine in the world

by Albert Silver
11/25/2019 – A new edition of the Computer Chess Ratings List, published on November 23, ranks Fat Fritz as number one. Now the engine is available to consumers through the new Fritz 17 release. A recurring question by users new to Fat Fritz and Leela, and even by veterans, is: How do I modify the settings in the engine and how do I obtain the best performance with both a standard GPU, as well as a top-end system sporting two powerful graphics cards? Read on to find out!

Fritz 17 - The giant PC chess program, now with Fat Fritz Fritz 17 - The giant PC chess program, now with Fat Fritz

The most popular chess program offers you everything you will need as a dedicated chess enthusiast, with innovative training methods for amateurs and professionals alike.

More...

A new paradigm

CCRL, the Computer Chess Ratings List, is the one of the oldest and longest running chess engine ratings lists in activity, going strong for 15 years now. You can find full tests of engines going back to Chess Tiger 2004! Following the times, they also now include testing with graphics cards to test neural networks, which includes not just Leela and Fat Fritz, but also Allie, and Stoofvlees. But it's Fat Fritz, running with an RTX2080 GPU that tops the current (November 23rd) list. While we don't believe that engine vs engine competition is the best reason to use our new flagship engine — its strengths as an effective analysis partner go beyond mere rating — it's still a valuable independent metric.

CCRL standings

November 23rd results matrix, top 12 engines (best versions only)

The website is very sophisticated with an impressive range of filters, details and statistics. By default, it will display only the best versions of any specific engine, but if you click on the Complete List link, it will show every version and hardware setup used. 

A breakdown of the individual match results can be found at the bottom of the list, or clicking on each entry. All games are available for download.

Getting the most performance from your engine

One of the most fascinating aspects of the neural networks is their reliance (for now) on a graphics processor (GPU) to achieve best results. For some, this is considered 'unfair' when comparing to a conventional chess engine such as Stockfish, Komodo, and so many more, since it gets a Central Processing Unit (CPU) and a GPU. However, this is a very misleading way of describing what goes on. The reason is two-fold:

  1. The GPU isn’t used to actually do any calculations. While the search is conducted on the CPU just like any engine, the huge weights file, containing millions of values which make up its understanding of chess, is read by the GPU for each node to give its evaluation. Even with a top-of-the-line GPU, only two cores are used to run the search. More will actually hurt speed, since the bottleneck is really how fast the GPU can read the neural network file, and not the CPU’s search calculations.
  2. The classic engine can certainly use two CPU cores, or four, or 32, or 128. Every added core just increases the speed of its calculations. Fat Fritz and Leela gain nothing from mountains of additional CPU cores.

As a result, saying the neural network has an advantage thanks to the GPU is incorrect since it cannot benefit from many CPU cores as conventional engines can. It is simply different.

Understanding that a conventional engine is different from a neural network is fine, but how can one compare them in a balanced situation? Sadly the answer is not straightforward.

How to compare: The AlphaZero ratio

When DeepMind published their paper on AlphaZero and the results against Stockfish 8, they gave a crucial piece of information that, at the time, probably seemed a curiosity at best: the ratio. All the results they published were based on two things: the time control, and above all the overall speed advantage in nodes per second that Stockfish had: 900 times more nodes per second (on average). 

Source: DeepMind

Why was this important? Because when neural networks came to the PC as a practical reality to reproduce AlphaZero, the only way to compare results was to set up a match that used the same conditions (around 63,000 vs 58,100,000). This means that if Fat Fritz or Leela is running at 10,000 nodes per second, Stockfish should be running around 9,000,000 nodes per second. Tip the balance too much one way or the other and you will then be clearly favouring one side over the other. 

The Diesel Dragster

One curiosity that was also shared by DeepMind in their first pre-paper, was how AlphaZero relied on a minimum depth to really show its strength.

Performance of AlphaZero and Stockfish, plotted against time per move. (source: DeepMind)

In a nutshell it shows that at very shallow depths, even AlphaZero lost and lost badly to Stockfish, but right around 30 thousand nodes per move it broke even, and only pulled ahead beyond that. Please note that was nodes per move, not per second. For AlphaZero, on their impressive hardware, that meant about half a second of analysis, but on a slower GPU that might mean more time was needed. Fat Fritz, for all its wonderful creativity has shown itself to illustrate this in the spades. GM Moradiabadi has noted how principled Fat Fritz’s play was, meaning that if the position called for material sacrifices for activity, it would not hesitate, even if this led to sharp double-edged positions. It is perhaps for this reason that results in very shallow games (or weak hardware) can lead to disappointing scorelines, as the complications do need a certain amount of calculations to be resolved. 

However, if you do take these factors into consideration, you can get superlative results. Here is a match played in 100 games against Stockfish Dev (November 6 build) using the TCEC 16 openings, in 10-minute games with a 5-second increment.

The hardware was such that Stockfish ran at around 27 million nodes per second (based on start position) on 32 threads, while Fat Fritz (a newer build) was running at 30 thousand nodes per second, for a perfect 900 to 1 ratio as explained above. This also means that for each move, Fat Fritz was getting a minimum of 400 thousand nodes. Remember that AlphaZero was run in far longer games reaching easily 30 million nodes per move, with Stockfish getting 900 times that.

 

So why ‘Diesel Dragster’? The idea was to convey a racer that may be slow to get to speed, but that has a fantastic top speed once it gets there.

Optimal configurations

If you look at the internal settings of both Fat Fritz and Leela, you will notice many values that are completely different, such as the CPUCT, CPUCT factor, and so on. These values are the result of a deeply automated process known as CLOP, which helps determine the best performing values for an engine. These will vary from engine to engine, or in this case from neural network to neural network, so do not assume that one set of best values will work on another. They might easily cripple the other neural network instead of help it

Still, as a small secret shared now with the readers: the tuning process was extended to a full 170 hours for Fat Fritz, and its new optimal values are slightly different than the ones that came with the first release. While they will be included in the next engine update, feel free to use them now:

Just right-click on the engine pane, and select Properties to open the UCI options

Change the cpuct to 3.56 (instead of 3.67), the cpuctfactor to 2.74 (instead of 2.54), and the Policy Temperature to 1.84 (instead of 1.87). You can see these values and where they are above.

Nvidia 16xx video cards

If you own a machine that has one of the newer Nvidia mid-range cards such as the GTX1650, GTX 1650ti, GTX1660 or GTX1660ti, you can enjoy a nice speed boost by changing the UCI option called the Backend to cudnn-fp16. Be sure to leave the number of threads to 2 though. More than that will hurt performance.

Multiple GPUs

One scenario that was not covered at all in the installation process is the matter of multiple GPUs. There is no question you can get maximum results with more than one GPU, and contrary to games in the past which required a special connection linking the two, all you need is to have them installed, and make a few changes in the UCI options. 

The exact changes are:

Backend — Multiplexing
Backend Options — (backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)
Threads — 4

Conclusion

Hopefully you will now find yourself armed with the means to get the most out of your Fat Fritz or Leela, and what to expect.

For more, check out all stories on Fat Fritz and the new Fritz 17.


Fritz 17 - The giant PC chess program, now with Fat Fritz

The most popular chess program offers you everything you will need as a dedicated chess enthusiast, with innovative training methods for amateurs and professionals alike.

More...




Born in the US, he grew up in Paris, France, where he completed his Baccalaureat, and after college moved to Rio de Janeiro, Brazil. He had a peak rating of 2240 FIDE, and was a key designer of Chess Assistant 6. In 2010 he joined the ChessBase family as an editor and writer at ChessBase News. He is also a passionate photographer with work appearing in numerous publications.
Discussion and Feedback Join the public discussion or submit your feedback to the editors


Discuss

Rules for reader comments

 
 

Not registered yet? Register

Eclipse2008 Eclipse2008 12/2/2019 05:56
I conducted a 14 game match between SF dev 29112019 (the latest one) and Fat fritz with a TC of 10min+3sec using perfect book.ctg (upto 4 moves) with a GTX 1050ti GPU. The score at the end was 7.5 -6.5 in favor of Fat fritz. It didn't lose a single game. BTW, SF was running with 6 cpus (6threads). Conclusion as far as me...FF plays really strong chess with good opening and middle game understanding. It equalizes with black from a standard opening line quite quickly. Its only problem is that it wastes lot of time converting a won game (positions with plus 3 or 4) and sometimes even squanders in time pressure.
Karbuncle Karbuncle 11/29/2019 06:52
Fat Fritz got uprooted on that blitz chess leaderboard by Leelenstein (another Leela chess Zero clone):

http://ccrl.chessdom.com/ccrl/404/

In fact, the top 5 engines are 4 Leela clones and Stockfish. That being said, I don't put much stock in blitz/bullet engine metrics. Longer time controls are much more fascinating to see where an engine's strength and weaknesses come into play, especially considering most people want the strongest longplay engine for deep analysis.
Reinderman Reinderman 11/28/2019 08:16
Just found https://de.chessbase.com/post/fat-fritz-welche-grafikkarte-braucht-man: the tables suggests that RTX 2070 gives most nodes for money.
Reinderman Reinderman 11/28/2019 06:26
Ben769 says in a reply to https://www.chess.com/blog/HanSchut/when-to-use-leelachesszero-and-when-stockfish-for-analysis "Note that if you want to go out and buy a GPU, make sure to buy any RTX gpu, which range from the $350 RTX 2060 to the $1200 RTX 2080 Ti, over any other GPU (unless you want to shell out for the premium V100 at $8000), and then choose the --backend=cudnn-fp16 option. This way, you can get a 10x speedup over a similarly priced GPU, and 100x over a CPU. "

This article says "If you own a machine that has one of the newer Nvidia mid-range cards such as the GTX1650, GTX 1650ti, GTX1660 or GTX1660ti, you can enjoy a nice speed boost by changing the UCI option called the Backend to cudnn-fp16."

Does RTX or GTX matter?

Any specific recommendation for a 300-500 euro GPU to work with Fat Fritz?
SvenRichter SvenRichter 11/28/2019 05:20
@Albert Silver: I made some comments on this German post...maybe you can help me
Stonker Stonker 11/28/2019 01:12
@sovaz1997
Wow. That's quite a statement. As an objective, yet admittedly past enthusiastic fan of various Chess base products and services (as well as user and admirer of Stockfish and Komodo and Houdini), I excitedly look forward to you presenting your evidence. Precise figures (89%) need some pretty precise evidence...unless you're just having a sarcastic bash at partisan fanboys/girls of course.
sovaz1997 sovaz1997 11/28/2019 12:15
I noticed that ChessBase has 89% of the articles about chess engines consist of lies. Conscience does not torment?
Masquer Masquer 11/28/2019 03:21
@ Albert
You're saying the GPU does no calculations. What then is performing the MCTS (search) calculations?
Rusty Razor Rusty Razor 11/27/2019 12:06
Should SmartPruningFac be set to 0 (As in the image), or 1.33000 (as default), or some other value? Thanks.
OlivierEvan OlivierEvan 11/27/2019 07:10
@Albert
Peut être quelques info sur la gestion du temps, fréquemment FF perd au temps. Cela vaut autant en local sur une même machine que le jeu en ligne.
En blitz avec ou sans incrément, en semi rapid 12+2'' 16 minutes.

Olivier
Graham Banks Graham Banks 11/27/2019 12:43
CCRL only benchmark their computers to that old Athlon. Nobody actually uses one for testing. The testers mainly use modern Intel and AMD computers these days.
Albert Silver Albert Silver 11/26/2019 05:41
@OlivierEvan - C'est lié a l'optimisation que j'ai fait. Tout les parametres sont liés, donc je ne conseil pas de le changer.

@Eclipse2008 - There will be an update with a stronger version of the weight file. As to a new version of Leela, you need to go to the official site, download the one you want, place it in the folder where lc0 is located, and in the UCI parameters, enter its name in the Weights space.
Eclipse2008 Eclipse2008 11/26/2019 12:35
If there is a significant improvement in Fat fritz weight file, will there be an upgrade to a newer version of Fritz 17 program? Or is the updgrade limited to changes in parameters of the engine? Also how to change a new version of weight file for a Lc0 engine in Fritz 17?
OlivierEvan OlivierEvan 11/26/2019 12:07
@Albert Cool l’article, merci.
Une petite question technique, pourquoi FF utilise pour FpuStrategy comme valeur Absolute et Lc0 comme valeur Réduction ?

Cela a t-il un réel impact sur la qualité d’analyse et de jeu pour FF.
Encore merci pour l’article.

Olivier
Albert Silver Albert Silver 11/26/2019 06:41
@andras62 - Glad to hear it!

@cc2150 - The correct value is 3.56.

@PurpDriv2 - I think there is a bit of confusion. In terms of functionality, the neural network engines are engines just like any other. You turn one on in Fritz or Chessbase and it will output the analysis with evaluation and mainline. The only critical difference is that they require a good graphics card (really for reading the large files, not imagery) to get the best speed. That is all. There is no 'learning thing'.

@chessfriend2019 - Yes, you can perfectly well use both GPUs, different, and get the joined performance of both. Just follow the configuration recommendation in the article for multiple GPUs. My personal system has one 2080 and one 2080ti, which are different GPUs, and they work great together.
Masquer Masquer 11/26/2019 02:04
@PurpDriv2 We'd still need to know if the NN engines are stronger than the A-B ones such as Stockfish and for that we need a rating list. Of course, the testing conditions, such as what hardware is used, matter a lot as well and we should be aware of that.
chessfriend2019 chessfriend2019 11/26/2019 12:13
Hello, I can put the 2 cards an rtx 2070 and an rtx 2060, in pci ex 3 and another in pci ex 2, is it because as I have those 2 slots and one is going to have low performance or isn't it? or it suits me that the 2 cards are the same even if you lose performance.
my motherboard is B350 TOMAHAWK Ryzen,Power supply 850w 80 plus.
PurpDriv2 PurpDriv2 11/25/2019 11:53
These Neural network engines should have their own separate rating list. These shouldn't be mixed up with real engines that can play chess on the go without this learning thing.
It confuses many people.
cc2150 cc2150 11/25/2019 11:05
The Picture shows CPuct value of 3.54, but the description underneath says "Change the cpuct to 3.56". Which value is correct?

Thanks
andras62 andras62 11/25/2019 09:24
Thank you. Great improvement for my gtx 1660!
Albert Silver Albert Silver 11/25/2019 06:44
@metaphysician - To my knowledge, demux was made to help with performance on 3 GPUs or more. I have not heard anyone report better results on two GPUs, but by all means try it and compare! It won't break it, it just may run slower, in which case you revert back.

@andras62 - You need to look in the 'cuda' version of Fat Fritz in the engines list, not the OpenCL.

@tom_70 - Lol. I actually have a functioning 1996 laptop with Windows 98 installed. I have ChessBase 6.0 installed and Chess Assistant 7.1 (with a beta of the engine Ferret). The engines even on it are far beyond 2000 Elo it should be noted. The screen has a massive 800x600 maximum resolution. :-)
Albert Silver Albert Silver 11/25/2019 06:36
@dumkof - The configurations were modernized to allow people to run the ew engines such as Fat Fritz and Leela, which require a graphics card to run well. Not for imagery, but speed of calculations. If you have no interest in them, then you can replace the graphics card with any you choose.

@keshava - any rating, top or other, will by the nature of evolution and time, be temporary. Still, we don't stop celebrating a record in a race just because tomorrow someone might break it, do we?

@illuzhin - The top of the list is misleading, and the tester who runs the RTX2080, test on a normal, modern CPU. I don't even know if it is possible to build a machine with such an old Athlon. :-)
tom_70 tom_70 11/25/2019 04:38
I still use my old trusted Radio Shack 2150L from 1995. With an ELO of about 2000, It is more chess computer than I will ever need and I can actually understand the moves. I suppose there are a few demigods out there who need a 3600 program to practice with.
Metaphysician Metaphysician 11/25/2019 01:32
Thank you for this, Albert.
My machine has two 2080ti graphics cards. Recently someone who knows about these things suggested that I use “demux” rather than multiplexing. What does “demux” mean, and what do you think about the advice?
Also, when installing Fritz 17 on a Win 64 machine, which .exe program(s) do I install?
Scorpion29 Scorpion29 11/25/2019 01:10
Tom Box Albert himself can't run a 40-40...this is the closest one can simulate for slow controls. I believe that Fat Fritz is indeed stronger...but only just.
Tom Box Tom Box 11/25/2019 01:08
Why didn't you say that this was only in the 40-moves-in-4 minutes category? The ChessBase articles on Fat Frtiz always seem like advertising more than objective discussions.
illuzhin illuzhin 11/25/2019 12:54
The others are running on a 15 year old Athlon 64 X2 4600+ (2.4 GHz). An RTX2080 is a monster in comparison.
AMCA AMCA 11/25/2019 11:50
Chess is a game to be enjoyed, not exhausted.
Keshava Keshava 11/25/2019 11:14
Does a 2 point rating difference mean 'stronger' or 'eqivalent'?
Would this difference in Elo predict a different result in a 100 game match than if the ratings were tied?
CCRL has these engines playing hundreds of games; isn't it entirely possible that engines this close in rating can temporarily spike over each other without justifying a verdict on their strength relative to each other?
Also, this rating is on the 40/4 list but I do not see Fat Fritz on the 40/40 list; does it's relative strength hold up at slower time controls?
1