Workshop: with hash tables size matters

by ChessBase
11/22/2003 – Time and technology march onward. The newer machines with faster processors and higher amounts of RAM have made the old formula for setting hash tables obsolete. In this week's ChessBase Workshop by Steve Lopez you can get the latest information on setting the proper hash table size for the Fritz family of playing programs. More...

ChessBase 17 - Mega package - Edition 2024 ChessBase 17 - Mega package - Edition 2024

It is the program of choice for anyone who loves the game and wants to know more about it. Start your personal success story with ChessBase and enjoy the game even more.



by Steve Lopez

There's been a bit of controversy over the last couple of years regarding proper hash table sizing in the Fritz family of playing programs. The trouble began with my February 24, 2002 column on "Setting hash tables". In it I advocated the use of a formula (dating back to Fritz5) to determine proper hash table size. Immediately, numerous chess message boards erupted with posts claiming that information was incorrect. My only defense at that point was to point out that the original information came to me from the programming staff and had never been superceded (at least not to my knowledge).

Several weeks ago, the ChessBase programming staff obliged me with an e-mail describing the new procedure for determining the proper size for hash tables. In this week's ChessBase Workshop, I'm going to pass that information along to you.

If you'll refer back to the aforementioned article (or have read it previously), you'll know that hash tables are also called transposition tables. They're a method of storage in which a chessplaying program keeps track of positions it's already analyzed as part of its current search. It stores positions and their evaluations in these tables, and if it later encounters a position it's already analyzed/evaluated (because that position has been reached by a different move order) it will use the previous evaluation instead of analyzing the position from scratch all over again. The program simply "recalls" the previous analysis and moves on to analyze another position in its search. Hash tables are thus a way to speed up a chess program's searches.

The program stores these positions/tables in temporary (i.e. RAM) memory. Way back in the days of the dinosaurs, when 32 Mb or 64 Mb RAM made your computer the envy of the Western world, it was possible (even likely) to set the size of a program's hash tables to a value higher than was really necessary. This could cause your hard drive to start ratcheting, as your computer was trying to free up RAM by storing information in temporary "swap files" on the drive (and, here again, this too was explained in the Feb. 24, 2002 column). So the programmers devised a handy-dandy formula for setting hash table size, based on the average number of seconds allocated for each computer move as well as on the processor speed of the machine the program was running on.

Off the shelf computers have come a long way since then. These days it's commonplace to see relatively inexpensive machines with 1 GHz+ processors and 1 Gb RAM included as standard features. This in turn has made the old hash table formula (as described in the 2/24/02 article) obsolete. So what's the new formula? In general, it's this:

Set your hash table size as large as you can without causing any undue hard drive activity.

Or, in plain English:

Set the hash table size as large as you can without making your hard drive bang/ratchet away each time it's the program's turn to move.

This is ridiculously simple in practice. The first time you run Fritz or another of our chessplaying programs, hit F3 to open the engine selection dialogue. In that dialogue you'll see the box for setting the hash table size; you'll notice that a value is already set (and the amount will usually be somewhere between 50% and 70% of the total RAM on your machine). You can simply use this default value if you like. But if you want to try increasing the RAM, bump the number up a bit higher and have your program analyze a few chess positions (by hitting ALT-F2 for Infinite Analysis mode, or by selecting it from the Game menu). If your hard drive light isn't coming on (with the accompanying rattling sound) for an extended period, you can try bumping up the value yet again and checking the hard drive's activity after starting "Infinite analysis" again. Repeat the process until you get a good bit of hard drive "rattling"; then you'll know the hash table value is set too high and that you should back it off a bit.

However, nothing in life is ever simple and there are some exceptional cases. So here we go:

1) Note that some engines (particularly older ones) might not allow you to use a high RAM value. Some engines from "back in the day" were programmed to use some (lesser) maximum amount of RAM (such as 128 Mb or 256 Mb). You can try setting these engines to use higher hash table values, but when you check the hash table size again (by hitting F3 for the engine selection pane) you'll see that the value has dropped back to some lower limit. Don't let that spook you. Just use the lower limit -- it's an indication that the engine isn't programmed to use an astronomical amount of RAM for its hash tables.

2) You'll still want to set lower values for blitz games compared to values for games with longer time controls. Even if you have a greasy-fast processor, you won't want to set your hash tables above 64 Mb or 128 Mb for a 5-minute blitz game (and still less if you have, say, a Pentium II or III processor -- something like 32 Mb RAM might be more appropriate).

Time and technology march ever onward and, in this case, they've made some older information obsolete. So you can safely ditch your calculator (at least as far as hash table size determination is concerned) and just crank the tables up to as large as you want them -- provided that they don't start your hard drive a'banging.

Until next week, have fun!

© 2003, Steven A. Lopez. All rights reserved.

Reports about chess: tournaments, championships, portraits, interviews, World Championships, product launches and more.


Rules for reader comments


Not registered yet? Register