Over the last 20 years, and even more so the last decade, we have seen technology advancing by leaps and bounds in ways that would shock someone from only 1990. Hardware has advanced so much that our smartphones of today (the first iPhone dates back only seven years) are far more computer than phone, but software has also advanced hand-in-hand with this process. The strength of chess engines has also taken off, but it is sometimes hard for the layman to differentiate how much of the progress is due to hardware and how much to software. The answer is both of course.
Still, it was a bit of a shock to listen to a conversation between two strong players, one of whom is a grandmaster, claim that engines had really not progressed at all in the past ten years, and that the Elo leaps the programmers were taking credit for, were entirely due to the faster hardware. Needless to say, for someone who knows several of these programmers, and the enormous time and effort expended, to see them classified as con artists did not sit well with me.
I realized the statements were not meant maliciously, and were sincere in their utter ignorance. After some thought I realized the vast majority of users are probably unclear on just how much progress has been made in pure programming terms. There are of course ratings lists of engines, but the numbers are so outrageously high over human ratings that it becomes hard to fit them into any normal perspective. Consider one of the top ratings lists over the past years: CCRL (Computer Chess Ratings List).
Komodo 8 tops the ratings lists for computer chess
At the top on a single core is Komodo 8 with a mere 3232 Elo. For most of us, the near 2900 rating of Magnus Carlsen is already approaching science fiction, so where does 3232 fit in the picture? That is today though. The best engine 10 years ago was in fact Shredder 8, but since the list does not go back that far, we will make do with Shredder 9, rated…. 2725.
Although a top engine in its day, the lists suggests that in pure software there is a 500 Elo gap
That suggests that software has advanced over 500 Elo in ten years, without factoring in the faster computers themselves. Can this be true?
I decided to run a small experiment. Since Komodo 8 exists on both the desktop as well as the Android smartphone, I decided to play a small match between it on a smartphone, facing Shredder on a modern, top of the line quad-core i7 processor. I ran a small test to compare the speeds of the platforms, by running Komodo on the start position and seeing how long it took to reach 20 plies. The result was that it ran about 50 times slower on a smartphone. Not 50%, no, 50 times.
To put that in perspective, if Komodo were racing at the speed of Usain Bolt at his fastest (roughly 45km/h) then Shredder would be racing in a Concorde at 2250 km/h. Therefore if Komodo’s edge over the best of yesteryear was in fact only 150-200 Elo, then the fantastic hardware advantage by Shredder should allow it to crush it. Although I actually own Shredder 8, and could dig it up, it is unable to make use of more than one core (2004 after all), so instead I used Deep Shredder 10, from 2006, which is not only much stronger but able to make full use of the quad-core i7.
A view of Shedder 10 using an i7, facing Komodo 8 running on a smartphone
Three positions were chosen to be played with reverse colors, each engine playing it both as white and black, at a time control of fifteen minutes with a ten second increment. Why only six games? Quite simply because I had to do this by hand, and moreover, was forced to manually tell Komodo when to play. The reason is that it was not designed to play against an opponent on a smartphone, just as an analysis engine, so I would enter Shredder’s move, let Komodo analyze, and then tell it to play its move.
I took the opportunity of recording the sixth and last game in such a way that readers might
follow the analysis of Shredder 10, but also Komodo 8 on the smartphone at the same time.
The final result after six games was a merciless crush by Komodo 8 with four wins and two draw, winning by 5-1. Had the match been closer, I might have considered extending it to see how it progressed, but 5-1 is pretty clear. As a reference, the performance over the six games was a 280 Elo lead, therefore if you accept that Deep Shredder 10 on an i7 is at least 2800 Elo (it was rated 2770 on a slower single-core system), then Komodo 8 is well over 3000 Elo on an Android smartphone. For the record, the smartphone in question was an LG Optimus G Pro from 2013.
If anyone had any doubt as to the genuine progress of chess engines, and the merit of buying new versions, they can lay these concerns to rest. Chess engines have not only progressed over the last ten years, but there is reason to believe they have outpaced the hardware even. If you want the latest and greatest, Komodo 8 is the king of the hill.
