Precision count on Playchess

by Frederic Friedel
7/20/2017 – Some of you may not have noticed: in our live broadcast of international tournaments, when computer analysis is added to the games, there are "precision" statistics given at the end of the notes. They give the percentage of relevant (non-forced) moves that are identical with the ones a top chess engine would play. In top events we find correlations between 15% and 97%. You can compare the level of precision of different players, and also look for suspicious values. Here's how, and why you should care.

ChessBase Account Premium annual subscription ChessBase Account Premium annual subscription

At the airport, in the hotel or at home on your couch: with the new ChessBase you always have access to the whole ChessBase world: the new ChessBase video library, tactics server, opening training App, the live database with eight million games, Let’s Check and web access to playchess.com

More...

Live.ChessBase.com

As you probably know by now: our Live Broadcast screen provides the moves of games as they unfold in different parts of the world. You can see the events that are currently running in the right navigation on the news page. Clicking on any one of the tournaments — the ones with the radio tower icon are currently being broadcast — will take you to a game board were the moves appear.

You can also click on "Live Tournaments" for an overview:

On this screen you will usually find cross tables or current standings. Clicking on an event, on the front page or in the overview, will load the tournament.

Clicking on a game will transfer it to the main board. While they are being played — and certainly afterwards — the most important games of the round are analysed and annotated by a very powerful computer, in real-time. The machine analysis includes didactic opening notes, tactical analysis, threats, better lines — all given in natural language.

At the bottom of the notation window on the right there is an evaluation bar which indicates how the game went for the two players. You can simply click on the bar to jump to the position where things came to a head. There are also buttons for editing, if you want to (delete, promote, cut lines, unannotate, undo, redo), save and play out the position against Fritz or tweet it to others. Hovering the mouse over any button will show you its function. The Live Book section shows you the moves that have been played in the current position, by what level of players and with what results. The Live Book is the most comprehensive and up-to-date openings key in the world.

You can maximize the replayer, auto-play, flip the board and save all games in the bar below the board. You can access the live games at any time during the round. To the right of our news page you have the live tournament navigation.

Precision measurement

There is one feature that many have not noticed, and some not fully understood: the "precision" data given at the end of annotated games. It tells us what percentage of moves are ones the computer would also play. Mind you, this is not mechanically counted: the program only considers move that were played after the opening phase, and more importantly: only non-obvious move. For instance necessary recaptures are not part of the precision count, only moves where there is a genuine non-forced choice.

Let us look at some precision measurements from recent events:

Round 1 of FIDE Geneva Grand Prix 2017: both players showing a high level of precision

In this Geneva game you can see who was playing precisely and who was not

The computer approves: the highest level of precision we saw in this event

Very high precision from the World Champion in round 3 of the recent blitz in Paris

Round four blitz: Magnus won in 24 moves, after very imprecise play by Mamedyarov

In the round five Blitz Carlsen was ground down and lost on time

In the same round Wesley So was defeated after considerable imprecision

Let us take a look at some recent tournaments. Here are some tables that give the percentage of (unforced) moves that match the ones played by the stongest chess engines. In some cases, where the game was a short, uninteresting draw, the system did not undertake machine analysis, and in such cases there is no precision data. We have listed the values for the top players in the final standings, two very strong players (Aronian, Gelfand) who did not do so well, and the players at the bottom of the table.

FIDE Geneva Grand Prix 2017

Player/rd
R1
R2
R3
R4
R5
R6
R7
R8
R9
Aver.
Radjabov
73%
86%
59%
85%
-
74%
97%
-
73%
80.0%
Grischuk
79%
81%
-
83%
-
48%
76%
64%
71%
71.7%
Nepomniachtchi
45%
-
47%
57%
77%
68%
54%
79%
74%
62.6%
Mamedyarov
70%
71%
38%
-
-
53%
67%
65%
-
60.7%
Giri
28%
60%
70%
70%
61%
-
-
75%
69%
72.2%
Svidler
71%
71%
-
78%
93%
69%
55%
59%
85%
72.6%
Gelfand
78%
-
-
76%
-
68%
60%
49%
72%
67.1%
Aronian
55%
62%
68%
72%
80%
31%
-
25%
84%
59.6%
Rapport
70%
51%
63%
32%
61%
34%
71%
32%
-
51.75%
Hou Yifan
31%
58%
77%
32%
40%
69%
79%
48%
58%
54.7%
Salem
52%
43%
59%
81%
54%
42%
42%
56%
31%
51.1%

After nine rounds the final standings were as follows:

Rk
SNo
  
Name
FED
Rtg
Pts
1 12 GM Radjabov Teimour AZE 2724 6,0
2 4 GM Grischuk Alexander RUS 2761 5,5
  6 GM Nepomniachtchi Ian RUS 2742 5,5
4 2 GM Mamedyarov Shakhriyar AZE 2800 5,0
  3 GM Giri Anish NED 2775 5,0
  5 GM Svidler Peter RUS 2749 5,0
  8 GM Harikrishna Pentala IND 2737 5,0
  9 GM Adams Michael ENG 2736 5,0
  10 GM Li Chao B CHN 2735 5,0
  17 GM Riazantsev Alexander RUS 2654 5,0
11 1 GM Aronian Levon ARM 2809 4,5
  7 GM Eljanov Pavel UKR 2739 4,5
  11 GM Gelfand Boris ISR 2728 4,5
  14 GM Jakovenko Dmitry RUS 2703 4,5
15 13 GM Inarkiev Ernesto RUS 2707 4,0
16 15 GM Rapport Richard HUN 2694 2,5
  16 GM Hou Yifan CHN 2666 2,5
18 18 GM Salem A.R. Saleh UAE 2638 2,0

Let us turn to the evaluation of precision in blitz games:

Next Move GCT 2017-Blitz, Leuven (1st half)

Player/% R1 R2 R3 R4 R5 R6 R7 R8 R9 Aver. Pts
Carlsen 56 57 75 63 - 63 35 57 74 60.0% 7.5
Giri 25 72 39 44 38 82 66 38 58 51.3% 5.5
Vachier-l. 44 88 50 46 30 39 79 62 37 52.8% 4.5
Kramnik 38 59 48 37 62 64 44 58 54 51.6% 6.0
Aronian 41 62 42 65 79 27 39 23 46 47.1% 4.5

Above are the precision values for the top players in the first half of the Grand Chess Tour Blitz section in Leuven. In the first round Carlsen (56%) defeated Anand (41%). In round two Aronian (62%) beat Ivanchuk (23%), in round three Carlsen (75%) had black against Vachier Lagrave (50%) and won in 60 moves. In the same round Ivanchuk (62%) had a great advantage over Giri (39%) but lost on time. In round four Carlsen (63%) beat Ivanchuk (28%) in just 22 moves, and in round six Carlsen (63%) beat Aronian (27%) in 60 moves. In round seven Carlsen (35%) lost to Giri (66%), and in round eight Aronian (23%) lost to Anand (84%).

For the second half of the Blitz we show you individual games that stood out, with the precision data and the result for each of them.

Next Move GCT 2017-Blitz, Leuven (2nd half)

White Black Rnd wP bP
Res
Carsen Anand R10 74% 69%
draw
Aronian Vachier R10 38% 47%
0-1
Nepomniachtchi Ivanchuk R10 28% 62%
0-1
Giri Kramnik R10 68% 62%
draw
Jobava Wesley So R10 18% 54%
0-1
Kramnik Carlsen R11 19% 68%
0-1
Vachier-L. Giri R11 69% 76%
draw
Anand Wesley So R11 81% 39%
1-0
Ivanchuk Carlsen R12 19% 68%
0-1
Carlsen Vachier R13 70% 24%
1-0
Wesley So Kramnik R13 70% 89%
draw
Aronian Nepomniachtchi R13 29% 30%
1-0
Giri Ivanchuk R13 50% 67%
draw
Jobava Anand R13 63% 42%
draw

Conclusion: Precise, but human

You can spend hours (as we have just done) following the precision evaluations, looking for trends and patterns. I would like to mention that the system is being used to identify suspicious performances — in some proven cases of cheating in the past our internal checks have revealed that the accused players were consistantly scoring above 90% in their games. The above values, however, strongly suggest that there is no cheating going on, at least not at this level of tournament play. Everything looks completely plausible: players vacillate between very high scores in one game and very low precision values in subsequent games.

We hope you enjoy this feature, and as always welcome your feedback, questions, and suggestions on how to improve it and make it more useful.

ChessBase Account Premium annual subscription

At the airport, in the hotel or at home on your couch: with the new ChessBase you always have access to the whole ChessBase world: the new ChessBase video library, tactics server, opening training App, the live database with eight million games, Let’s Check and web access to playchess.com

Editor-in-Chief emeritus of the ChessBase News page. Studied Philosophy and Linguistics at the University of Hamburg and Oxford, graduating with a thesis on speech act theory and moral language. He started a university career but switched to science journalism, producing documentaries for German TV. In 1986 he co-founded ChessBase.

Discuss

Rules for reader comments

 
 

Not registered yet? Register

Resistance Resistance 8/6/2017 11:57
There are some issues to deal with. First, the 'Precision' label for this function is rather misleading. The concept of precision as such does not mean much unless you use it in connection to something else (--typically, to that which you consider as the best or most desirable: economy, beauty, creativity, weirdness, simplicity, extravagance, etc, etc, etc--). In this case, you're establishing a contrast to that which the computer thinks about the different positions arising in a game (--shall we say economy, then?--), i.e., you are saying that precision is the similarity between the moves of the players and the suggestions provided by the computer: the closer to the computer, the more 'precise' the move. Therefore, an appropriate name for this new function would have to address this contrast (--names like 'Computer Precision', or 'Computer Parity', might be more appropriate for instances of this sort--).

Also, as other commentators have already suggested, this kind of evaluation will depend, (2) on the particular engine used for contrast, (3) on the depth of the analysis, and (4) on the particular suggestions you consider from the whole set of possibilities the computer might generate (1st, 2nd, 3rd, 4th alternative, etc, etc). That is, you might get the impression that a cheater wasn't actually cheating: if you don't find the right engine, if you don't reach the corresponding depth, or if you don't take into account the proper engine suggestions (1st, 2nd, 3rd alternative, etc, etc.)

(5) Now, that the moves in a game show a high level of computer precision from one of the players, it doesn't necessarily mean that the guy was cheating. It does make him look suspicious, yes, but it isn't necessarily a proof that the guy was cheating (what if the guy was having a great day, as other commentator has already suggested?). On the other hand, if the guy is repeatedly showing high levels of computer precision throughout his games, then it makes sense to think of cheating.

Overall, I think this (computer) precision index is a very good idea. For it will certainly discourage cheaters from cheating on other players (--no question about that--), despite the system not being 'perfect' yet. That people can see that your games reek of computer cheating is incentive enough to persuade yourself to stop doing it, in case you were...
RayLopez RayLopez 7/23/2017 07:11
Article: "The above values, however, strongly suggest that there is no cheating going on, at least not at this level of tournament play. Everything looks completely plausible: players vacillate between very high scores in one game and very low precision values in subsequent games." - oh, great, so now if a patzer plays the game of their life and scores 90% precision in a complex tactical game, they will be accused of cheating? (just kidding, but probably true; however, if the patzer repeats this for many games, the cheating accusation is either just, or the patzer is really a budding master)
genem genem 7/20/2017 10:34
I wish Fritz and Komodo would offer to automatically mark (with the Informant 'square' symbol) every move made when the move was the only good move available.
The square would be uninformative for routine completions of exchanges, but it would also identify other subtle moves which were the best defense etc.
erikbergren erikbergren 7/20/2017 08:11
If I did this I would release a paper describing the algorithm for rejecting the "forced moves" and for marking the moves that "match computers". Those tend to be opposites: the number of moves OK to different computers and different depths increases as the move is less forced, creating a paradox in your stated voting process, the barrier of "acceptable" spawns bifurication. I expect about 4 moves out of 40 can hold an even game even in midgame positions that look "unclear", a player must not be penalized for chosing the wrong of those 4. Also there will be a lot of obvious noise in you measurements: the attacking player will get higher ratings because it is less easy in chess to find saving defending moves than to finish a thematic attack accurately.
Often play against imperfect opponents involves making not the best (and perhaps more difficult to complete) plan but a plan which your opponent is predicted to have trouble dealing with. Books in the 1990's used to say that players with experience against computers had about a 100 ELO advantage playing with that bias than playing the "best moves" they saw. But your algorithm would report the opposite: that they were playing at a worse level when beating those 1990's computers with "trick" moves devised for their particular opponent. A similar error would occur for your algorithm for some human versus human play. We could imagine that, as chess becomes more and more perfectly played with society's tools and experience with it, kibitz criticism can be more precise. But to believe that, after the Opening and before the Endgame, such rhetorical schemes can be a better judge than the win of the game, and the ability of the winning player in his future games, is a gamble which ignores some aspects of the game still used for winning.
horius horius 7/20/2017 08:08
"They give the percentage of relevant (non-forced) moves that are identical with the ones a top chess engine would play"

Is it only the top move? or the average of the best 1, 2 or 3 moves? which would make more sense
Boye Boye 7/20/2017 08:03
Interesting. Is this feature available in CB14/F15? According to the online manual you have to do a "Let's Check" analysis, however I do not find the percetages after analysis finished. It would be interesting to go over some classical (old!) games with this feature.
AgainAgain AgainAgain 7/20/2017 07:43
Will this be available in the Live Database? Or to somehow check own game?
GalacticKing GalacticKing 7/20/2017 06:25
This is the coolest new feature ever! It will be fun to see how many games are won with less precision by boonswaggling the opponent!
Leavenfish Leavenfish 7/20/2017 05:00
Precision counts...this should be available for any member to run his own games on.
chessmatt chessmatt 7/20/2017 04:38
Another relevant question is depth of evaluation. What depth is used in the analysis?
Timothy Chow Timothy Chow 7/20/2017 02:04
How sensitive is the precision figure to the choice of engine? Do the engines rate each other as having over 99% precision?
1