Winning starts with what you know
The new version 18 offers completely new possibilities for chess training and analysis: playing style analysis, search for strategic themes, access to 6 billion Lichess games, player preparation by matching Lichess games, download Chess.com games with built-in API, built-in cloud engine and much more.
Several weeks ago I wrote an article focusing on draw-related trends in top-level chess, wondering whether the trend was really as disturbing as some people seem to think. My conclusion was that although draw rates have risen in the past 20 years, the rate of short draws was actually going down, and so perhaps there was no need for radical measures. In the meantime, I have performed a lot more analysis, and have also followed some lines of investigation suggested to me by grandmasters John Nunn and Bartlomiej Macieja, both of whom have been extremely helpful in explaining a grandmaster's perspective on this topic.
First of all, let's quickly review the basics. Much of the analysis that follows will be focusing just on the games played between top-20 opponents since 1980. Here is the overall trend of drawing rates within these "elite" games across the years since 1980:
You can see that about 65% of games played between top-20 opponents were drawn during the early 1980's, and this dropped to barely 50% in the early 1990's, and then jumped back up to 60% around 2000, and currently is pretty steady at about 62%. Please note that the X's on the graph show the actual data points, whereas the solid line shows the overall trend – you will see this design in several more graphs below.
However, what about short draws versus long draws? As I said in the previous article, I think that to a chess fan it is the shorter draws that are more embarrassing, and a long draw can often be quite exciting and educational. So, do we see this same "down, then up, then steady" trend, if we try to count up just the short draws played between top-20 opponents?
In fact, here we see a quite encouraging trend – in the early 1980's, almost 30% of elite games were draws lasting fewer than 25 moves, whereas these days it is barely 10% of such games! And we are not seeing any sort of upward trend since 1993; if anything, the rate of short draws is slowly decreasing over time. This may well be an effect of Bilbao scoring (3/1/0 for a win/draw/loss rather than the traditional 1/0.5/0), or Sofia rules (no draw offers allowed before a certain point in the game). Surely there are many factors contributing to this one trend, but this high-level analysis says to me that there is no immediate need for further anti-drawing measures to be taken.
If you consider the implications of the previous two graphs, you may have realized that the incidence of "long draws" at the top must be increasing, and this indeed is true. Currently about 25% of games played between top-20 players, is a "long draw", meaning it lasts past Move 40, and in fact these long draws are happening more frequently than at any point since 1980:
However, this is not the final word on the subject. When I asked John Nunn and Bartlomiej Macieja, they both seemed to feel that measuring the length of a game is not the best way to differentiate between "good" draws and "bad" draws. For instance, Bartlomiej Macieja pointed out that there are long games where a grandmaster observer might look at the final position and wonder why a draw was agreed in such a live position, where all three results are still possible. Conversely, some short games are obvious draws where only one result is possible between grandmasters. Certainly one way we could investigate the frequencies of these types of draws would be to start from each position where the draw was agreed, play many computer vs. computer games, and see how frequently a draw actually occurs in those games. This would definitely help tell us which games really were dead draws, and which still had life left in them. Of course, it would be quite an analytical effort and so it is not something I have tackled yet!
John Nunn also had some very interesting insights:
If two players go through 25 moves of theory, play one or two new moves then agree a draw, that's not very exciting, but a game which takes an original course from an early stage might be quite interesting even if it is relatively short. One could imagine an indicator such as number of moves which have never been played before, although it would be hard to implement (some games would score zero!). Unfortunately, I think that the effect of extensive computer use is that many players avoid sharp lines because they fear running into a computer-generated innovation. Hence you see many Petroffs and Berlin Defences. There are only a few players who will risk the King's Indian or Najdorf at a high level.
This indicator (the depth at which the novelty occurred) is not readily available in the large game databases, but I decided it would be worthwhile to generate it for purposes of analysis, and so I processed all master games from the latest ChessBase Mega Database (along with online updates bringing the data up to mid-September 2011) and determined a "novelty depth" for each game. John also had made the following useful suggestion:
Often you find that a 'novelty' arising in a game between strong players had in fact been played before in an obscure game, often without the novelty player really being aware of the point of the novelty. With so many games being played around the world, it's not surprising that such moves can be played by chance. Therefore you may get better results if you define a novelty as a move which had not been played before by a (say) 2200+ player.
Along those lines suggested by John, I defined a novelty as a move played by a "master" that leads to a position never before reached by a master, where a master is defined as someone having a 2200+ rating. Prior to 1980, and certainly prior to 1970, FIDE ratings are not necessarily available, and so I made the simplifying assumption that all recorded players were masters prior to 1980. Thus all games prior to 1980 in the Mega Database could contain a novelty, as long as they lasted long enough to reach a unique position. One reason this analytical task is very complicated, of course, is that we have to consider transpositions; if one game goes 6.cxd5 exd5 7.exd5 Nxd5, and the other games goes 6.exd5 exd5 7.cxd5 Nxd5, they still lead to the same position despite having different move sequences. So we have to track each position in the game, rather than just looking for completely unique move sequences. But I handled all of that, and found some interesting things!
First of all, let's look at the big picture. Setting aside the draw question for the moment, how fast is "chess theory" advancing, and have computers accelerated this advance? By looking at the average depth of novelties played by masters, across the years, we can start to answer this question. The following graph indicates the average depth of novelty played by masters each year, from 1850 to the present:
Please remember that the "depth" refers to plies, not move numbers. So for instance, you see that from 1880 to 1890 the average novelty depth was pretty steady at 13. That means the 13th ply of the game, which is White's Move 7, and thus we learn that during the 1880's the typical game between masters resulted in a novelty around White's seventh move. As another example, in 2000 the average novelty was at ply 21 (White's Move 11), and then ten years later, in 2010 the average novelty had finally reached ply 22 (Black's Move 11). So at the current rate, novelties advance by one ply every ten years! And I don't see any great evidence of different trends in the past decade; it looks remarkably similar to the progress in the 1960's and 1970's.
It is interesting to notice how the depth of novelties was actually declining around 1920, which would tend to mean that people were finding new ideas (at an earlier point in the game) in previously unpopular openings. Also of great interest is the big increase from 1978 through 1983. In just five years, the average depth of novelties increased by 1.5 ply, something that normally takes fifteen years rather than five. I can't give a general explanation for this, and it does seem to be a widespread effect across many openings, but the two biggest individual factors appear to be the deep advances in theory of the symmetrical English and exchange Grünfeld, both of which greatly increased in popularity within elite play between 1978 and 1983.
We can also restrict our view to just the novelties played during the elite games (between top-20 opponents), which is perhaps a bit restrictive but still interesting:
As you might expect, the top-20 players know their opening theory better than most! And so the games between top-20 players tend to reach their novelty at a later point in the game, which is why the average for all 2200+ players is a "novelty depth" of 22 ply right now, versus 26 ply in just the elite games. You will notice, I am sure, that the average depth has plummeted in the games during 2011. That is a small sample size, not even 200 games, but is still quite intriguing. The largest factors contributing to this drop during the year 2011 are the virtual disappearance in these elite games of the C42 Petroff's Defense, which tends to have its novelties around ply 30 (Move 15), and of the B33 Sicilian Sveshnikov, which tends to have its novelties around ply 37 (Move 19!), as well as the C89 Ruy Lopez Marshall, which tends to have its novelties around ply 39 (Move 20!). In addition, we are seeing more A04 Reti Openings and C65 Ruy Lopez Berlin Defenses, both of which have their average novelty around ply 20 (Move 10). So on average we are seeing the depth of novelties in these top-20 games declining, something that has not happened to nearly this extent in any of the years since 1980 that I have considered.
Anyway, knowing the point at which a novelty occurred, as well as the point at which the draw was agreed, tells us how many "new moves" were played, the indicator that John Nunn suggested. This lets us look at the same sorts of graphs again, where we are looking at the rates of "bad" draws and "good" draws, but with an indicator that is more meaningful from a player's perspective rather than a casual fan's perspective. First, let's look at how many games among top-20 players are agreed drawn within 30 ply (15 full moves) of the first novel position:
Once again, it is very encouraging to see this trend. Fewer games than ever before are being given up as drawn within 30 ply (15 full moves) of the novel position. In fact, 2010 was evidently the first year in decades where more than half of all draws between top-20 players lasted 30 or more new moves (15 or more full moves after the novelty). So whether it is Bilbao, Sofia, young players, computers, or whatever, the frequency of these "less interesting" games is only half of what it was 30 years ago, and comparable to the period in the early 1990's where draw rates in general were so low.
This brings us to the other side of the coin. We are indeed seeing more and more "good draws", which by this definition are draws having a lot of new moves in them. So if we look at how many elite games include at least 50 new moves (25 or more full moves), the rate is currently at 28%-30% and probably will keep creeping upward, if the previous six or seven years are any indication. This rate was quite steady at 22% to 24% for a very long time, but has clearly increased in recent years:
To me the key question is whether or not this is a disturbing trend. It is a quite reasonable position to say this trend is bad for chess because draws look "boring" for casual fans or sponsors who want to see decisive games. But it is also a quite reasonable position to say this is good for chess, that players at the top are not abandoning the games to a draw when there is still life in the position, and it is simply in the nature of the game that top players will be evenly matched and often unable to overcome the famous "margin of the draw".
I am in the latter camp. I feel that draws are a fundamental feature of chess, and they are incredibly useful to making the rating system work well, and tournament pairings work well, and any measures that try to minimize all draws (rather than just minimizing draws in positions that still have a lot of life in them) would be a step in the wrong direction.
One last point – I know that last time I promised to reveal many interesting insights about the drawishness of top players, and of different openings, and I have indeed discovered many things. But for now I will have to postpone those revelations for a later day, and just leave you with a quick quiz to puzzle over:
Over the past five years, which top-20 player was least likely to have a "bad draw" in games against other top-20 opponents (meaning the game was agreed drawn within 30 ply of the novel position)? There is a clear winner in this category…
Over the past five years, which top-20 player played the deepest novelties on average against other top-20 opponents (looking only at move depth)? His novelties averaged almost 30 ply (Move 15), a full ply ahead of anyone else.
Conversely, which two top-20 players deviated from theory earliest (on average) in their games against other top-20 opponents during the past five years? Their novelties averaged less than 24 ply.
In these elite games between top-20 opponents since 2006, which player was the most likely to get their novelty in first? He played a novelty in a remarkable 65% of such games, far ahead of the rest of the crowd.
In these elite games, which player was least likely to play a novelty, instead sticking to known theory until the bitter end? He played a novelty in only 41% of such games between top-20 opponents since 2006.
What is the only ECO code with nine or more "elite" games since 1980 where the draw rate was 100% in those games?
What is the only ECO code whose average novelty depth during 2010 was at move 21 (ply 41) or later? In fact it is more than two full ply ahead of the next-deepest ECO code. This includes all novelties played by 2200+ players, not just the top-20.
As a hint, in the questions about individual players, I am only considering the players with 60+ elite games since 2006 (where an elite game is one between top-20 players). Thus the possible answers are: M. Adams, V. Anand, L. Aronian, M. Carlsen, B. Gelfand, A. Grischuk, V. Ivanchuk, D. Jakovenko, G. Kamsky, S. Karjakin, V. Kramnik, P. Leko, S. Mamedyarov, A. Morozevich, R. Ponomariov, T. Radjabov, A. Shirov, P. Svidler, V. Topalov, and W. Yue. Good luck!
Who is teaching what to whom? Jeff Sonas (left) discussing his draw statistics
with Unix inventor Ken Thompson during a visit in Sea Ranch, California, in
September this year (see pictorial of last
year's visit)
There will be a prize for the best answer submitted before the end of October. You should reply using the feedback link in the left navigation on this page. The subject line must be "Sonas Quiz". Please note that multiple submissions will be ignored. In case of more than one correct result the winner will be picked at random.
The prize is a bulletin of one of the events used in the Sonas survey: the 2008 GM tournament in Biel/Bienne. The bulletin has been signed by all of the participants – including a 17-year-old who finished equal second/third, something he seldom does these days.
 |
Sonas: what exactly is the problem? 06.09.2011 – In recent weeks two strong GMs have sent us their solutions to the problem of draws in chess (have rapid game deciders on the same day, they propose). This has caused quite a stir in the chess world. We asked chess statistician Jeff Sonas to scientifically analyse the problem – it's got to be good to know exactly what we are talking about. Here are his remarkable findings. |
 |
Kasimdzhanov: Open letter to FIDE – with a proposal 21.07.2011 – Uzbek-born grandmaster and former FIDE knockout world champion Rustam Kasimdzhanov, now a permanent second of World Champion Vishy Anand, is a profound thinker, and not just in chess openings. He has now written an open letter to FIDE, describing the current unsatisfactory situation in top professional chess and proposing a startling solution. |
 |
Kasimdzhanov's proposal – our readers react 03.08.2011 – Two weeks ago former FIDE world champion Rustam Kasimdzhanov made a startling proposal, designed to lift chess out of what he perceives to be a crisis: eliminate draws by playing rapid and then blitz games if there is no decisive result. Naturally our readers commented on this idea with vigor – and many with interesting counter-proposals. Here is a (very large) selection of letters. |
 |
Shipov: Chess as a sport and the three-point system 07.08.2011 – Sergei Shipov is a Russian grandmaster (peak Elo 2662), trainer of many top players and talented juniors, journalist and author. His web site Crestbook is well appreciated by Russian fans, as is his live commentary in top events. Sergei sent us the following article, in which he weighs in on our recent debate, initiated by GM Rustam Kasimdzhanov, on draws in chess. |
 |
The Kasimdzhanov/Shipov draw proposals – readers' feedback 19.08.2011 – Former FIDE World Champion Rustam Kasimdzhanov recently proposed that there were too many draws in chess, and that eliminating them from top-level chess would greatly enhance the popularity of the game. After him GM Sergei Shipov made a similar proposal, involving tie-break rapid or blitz games to be played immediately after draws. Here a selection of reactions from our readers. |
