Let's Check: the elite are better than you know

by Albert Silver
10/17/2022 – There are several ways to check a player's performance using an engine. One is to simply ask an engine to analyze every move and highlight every disagreement however small. Another is to use the tool in ChessBase and Fritz known as Let's Check. Here are the results from the recent Sinquefield Cup including a 100% match and the curiously high results by...

Fritz 18 Fritz 18

"Evolving Genius": learn to attack and play brilliancies. Fritz offers you everything you will need as a dedicated chess enthusiast.

More...

There are three main ways today to evaluate a player using an engine. This is not to be confused with annotating a game.

The most sophisticated one is used by analysts such as Dr. Ken Regan, who compile the average error rates of players per level. For example, I might lose an average 0.15 pawns per move compared to the engine's best, while a top GM might lose only 0.02. His system will go deeper than this, but that is still the foundation on which it lies and it will be better at catching 'smart cheaters' than a more basic system such as below.

The simplest is just to analyze a game with an engine and ask it to highlight every move it disagrees with, however small the difference. Obviously the risk is that in some positions, there might be three roughly equal moves that three engines play slightly differently.

Imagine you are analyzing with only Stockfish, and it says that five moves out of ten are not a match. This might overlook that two of the moves that don't match its choices, are chosen by another top engine such as Komodo Dragon 3. In other words, only five match Stockfish, but seven in all match top engine choices. That is the underlying point of Let's Check. When you analyze a game with it, it will not only tell you what a variety of engines thought of each move, it will give you a summary called Engine Correlation at the top, showing the percentage of times a player's moves matched the top choice of an engine.

However, unlike a plain engine comparison, it won't compare with just one top engine move, it will compare with several, and if the move matches any of those engines, then it is a match for Engine Correlation. 

Komodo Dragon 3

The new Komodo Dragon 3 engine has gained 100 Elo points in playing strength over its predecessor when using a processor core in blitz. That's a huge improvement for a program that already reached at an Elo level of over 3500!

Sinquefield Cup

Recently there were several claims about high Engine Correlation matches between Hans Niemann's games and the Let's Check choices, so out of curiosity I ran a complete Let's Check on all the games in the recent Sinquefield Cup and I must say the results were unexpected.

The first result to come out was that one player did actually obtain a 100% match. This was not the result of some ultra-short draw, since Let's Check will ignore theory moves, and games with too few moves played. I.e. a game that was 28 moves long but had 20 moves of theory will not be eligible for an Engine Correlation result. Who is this engine matching wonder? Wesley So.

In his game against Ian Nepomniachtchi, the American player achieved a 100% Engine Correlation score. However, he was not the star performer overall in terms of such measurements, since it was his only game over 80%. No, one player managed to score three times in excess of 90% engine correlation. Aha! I hear you cry out. We have him! So who is this chess engine-like god?

Levon Aronian had several of the highest quality games according to Let's Check

Meet Levon Aronian, late-bloomer extraordinaire, who had an engine correlation of 92% against Caruana (who himself has a 96% correlation in that same game) over 45 moves, 91% against Wesley So in 43 moves, and 91% against Magnus Carlsen in 36 moves. Plus two more games with over 80%.

He was not quite alone though, and none other than Ian Nepomniachtchi had two as well, plus several over 80%, showing the quality of play that led him to win the Candidates this year. Note that he had an average 78% engine correlation for the entire Candidates, 11% more than second-best Caruana.

The burning question on your mind, dear reader, is what about Hans? In terms of engine correlation, Hans was the worst. His best game, with an 88% match over 55 moves, was in round seven against Maxime Vachier-Lagrave. In his game against Carlsen it was a modest 68%, but of course Magnus was playing dreadful that day, and had only 37%. 

The mythical 100%

So how rare is 100% after all? It is rare but not as rare as you might think. I ran some random checks through games in 1999-2000 as I was curious about Kasparov and Kramnik. All in all I had some 150 eligible games, maybe less, yet it turned up a higher-than-expected number of perfect matches.

For example, the rapid games Amber tournament had several 100% perfect games, including Jeroen Piket in one, and Kramnik in another. And against Topalov no less... Memories of Toiletgate. There were also two(!) by Kasparov in Bosnia in 1999, another in Bosnia in 2000, one more by Kramnik in the World Knockout event against Korchnoi over 41 moves and later one by Michael Adams against Vlad in that same event.

However, there is a caveat that must be mentioned when using such tools. It is eminently possible to game the system to show a 100% match where it normally might not. You see, when doing a Let's Check analysis within Fritz, you have the option of providing your own engine, and then telling it to only use it for moves that did not match engine choices. In other words, you are trying to find an engine it will match. And if it does.... the engine correlation will improve.

 

Originally, this game was only a 90% match, with no engine choosing Garry Kasparov's 16.cxd5 for example. After trying several, I found an engine that chose it, and entered it as another Let's Check choice. Now the tally reads:

So yes, the results can absolutely be manipulated by the unscrupulous. A telltale sign might be in the engines listed. If a new game shows Stockfish 14+, Komodo 12+ and so on, it should be fine, but if you see some very old engines or odd names for that same new game, be on your guard, as they may have been used only to get an extra match. 

Mega Database 2022

The ChessBase Mega Database 2022 is the premiere chess database with over 9.2 million games from 1560 to 2021 in high quality.

Regardless, here is the signature win by Kasparov with notes from Mega Database:

 

Conclusion

Does this in any way invalidate the use of a tool such as Let's Check? Of course not, but as all such tools, they must be used with good sense and judgement. The fact that modern elite players can rattle off multiple games with such extraordinarily high engine matches is a testament to the increasing overall quality of the chess players, since the engines they are matching today, are also hundreds of Elo stronger than engines of a decade ago. These players are also studying and learning from the engines, and that increase in pure ability is a consequence of it.

 


Born in the US, he grew up in Paris, France, where he completed his Baccalaureat, and after college moved to Rio de Janeiro, Brazil. He had a peak rating of 2240 FIDE, and was a key designer of Chess Assistant 6. In 2010 he joined the ChessBase family as an editor and writer at ChessBase News. He is also a passionate photographer with work appearing in numerous publications, and the content creator of the YouTube channel, Chess & Tech.

Discuss

Rules for reader comments

 
 

Not registered yet? Register

lajosarpad lajosarpad 10/18/2022 05:16
@arzi "Still, that doesn´t prove anything."

Yes, this is correct. I was just quoting the allegation, without agreeing with it. Carlsen strongly suggested that Niemann has cheated against him at Saint Luis by saying that he accomplished what only the super elit is able to accomplish (which is Niemann is not part of), while seemingly not even concentrating on the game. These two thoughts, paired with "cheated more and more recently" clarifies to me that Carlsen really think Niemann cheated against him.

"Niemann is accused here that his game is too precise, like Stockfish or some other programs. How then can you distinguish a cheat if it is only used a few times? If, on the other hand, Niemann's game delivery is not accurate, like Stockfish's, then Niemann will be accused of cheating, which he does only a couple of times in the game, of course. One way or another, the end result is always the same, Niemann cheats. Doesn't this seem a bit silly?"

Agreed. If the claim is that Niemann is only occasionally cheating, then pointing out that his moves were in the top three choices of Stockfish in his game with Carlsen suggests that he is a very strong player who can find computer moves OTB. If the claim is that Niemann is cheating whenever he has the chance, then in his "flawless" games we should find no errors. But we find mistakes even in his game against Carlsen and we find precise moves on his part quite frequently.
Science22 Science22 10/18/2022 03:00
@Jacob Woge : Hejsa, thank you for the wise evaluation. What a joy after the Erasmus Montanus storm here. You are absolutely right, more evaluations is needed. There is so much work to be done, and I am confident FIDE has experts to entangled everything. I can send you a lot of statistical material later directly to the institute, it is all to much here.

I can not go into detail with the statistics here. The methods and models. But there is a follow up from after 2020 that ad to the deviation between live transmisison / not live transmission.

To those of my friends who have written to me that I should not place any kind of personal information on an internet site with access for apparatchiks and chicken thieves : I listen to your advices and send a direct invitation to Hans Niemann instead in cooperation with chess friends . Of course he should have a chance to proof his strenght under circumstances that makes us confident he does not receive help during the game.
arzi arzi 10/18/2022 02:49
To Jacob woge, US Chess Championship 2022, two games left.

Jeffrey Xiong (2690) 6 points, 1 win, 10 draws (2697).
Hans Niemann (2699) 5,5 points, 3 wins, 3 lost games, 5 draws (2674).

Does it mean that Niemann has cheated in three games and not-cheated in 8 games? What about Jeffrey Xiong? Does his current game result relate to his playing style? What about Niemann's result? Is it just because he is cheating?

What do you think, Science22?
Jacob woge Jacob woge 10/18/2022 02:31
The way I would look at a compilation as presented by Science22, thank you very much

https://twitter.com/atl_kings/status/1568656197812891653

is in a simple 2x2 matrix. 1½ year of 2019/20 tournament activity, BC=broadcast

BC no BC
win rating 9 0
lose rating 0 9

(I regard the twitter link table as a) complete for period covered, and b) trustworthy. I don't know for sure. )

This shows that you can use existence/absence of BC (assumed to be live throughout) of a random tournament in the set to predict if said player had a rating gain, or a loss. The predictive skill is near perfect (100%). Near, because of one tournament left out in this table, due to no rating change.

You would expect an even distribution over the four boxes, or even slightly skewed across the other diagonal, if strong tournaments are more likely to be broadcast. In no way should BC or not impact anyone's performance.

I am not one to say if this deviation has enough significance to constitute proof of supposed foul play OtB, somehow connected to live BC during that period. What one can say, is that the table fails miserably in proving the opposite, i.e. absolving. That theory is not supported.

This simple approach does not involve any chess playing programme comparison. There are a ton of ways to do that, and some results are, as demonstrated below, difficult to interpret. The underlying hypothesis is also unclear: cheating all the time? some of the time? at critical junctures? against specific opponents? The answer to that question influences exactly which statistic is important.

This leaves all that aside and focuses the assumed purpose, which is not high quality of the games by some measure, but enhancing the sportive result.

Plus, we may begin to form a not too outlandish theory about method.
arzi arzi 10/18/2022 01:57
lajosarpad:"The thing is that if a GM receives outside help only once in a game, then there will be no reason for a suspicion based on the moves. That is why any (!) analysis of cheatings at games looks at correlations between the player's move and the engine's move."

A very good point. How can we then define the quantity and quality of the cheat if the cheat is only used a couple of times in the game? Niemann is accused here that his game is too precise, like Stockfish or some other programs. How then can you distinguish a cheat if it is only used a few times? If, on the other hand, Niemann's game delivery is not accurate, like Stockfish's, then Niemann will be accused of cheating, which he does only a couple of times in the game, of course. One way or another, the end result is always the same, Niemann cheats. Doesn't this seem a bit silly?
with_cheats_you_lose with_cheats_you_lose 10/18/2022 01:26
@lajosarpad, even if there are, say, 5 forced moves in the game Carlsen - Niemann, the engine/game correlation values for Niemann and Carlsen are still higher than the ones presented by Albert Silver from the Let's Check tool analysis. (33+9-5)/(57-5-5) = 79%; (22+3-5)/(57-5-5) = 43%.

I do not have access to Let's Check analysis, so cannot substantiate the author's values, maybe someone else can.

And engine accuracy is not the same as engine correlation, please read this for more information: https://lichess.org/page/accuracy
Science22 Science22 10/18/2022 01:09
Sane readers , go to : https://twitter.com/atl_kings/status/1568656197812891653 . Niemann 2019- 2020 :


SCF Performance rating with no live games : ELO 2404
USCF Performance rating with live games : ELO 2610
USCF Rating change with no live games : ELO - 112
USCF Rating change with live games : ELO +131

Keep the facts on top. Keep the hate out. Look at reality. Ignore abuse.
Science22 Science22 10/18/2022 01:01
@Johannes Fischer : This happens with 100 % acurracy. . Lajosarpad, arzi, mc1483 moves in and provoke an otherwise nice debate with personal insults. If you talk back they get furious. They work like a wolf pack. I am truly sorry for chessbase.com but you should ban them. Then you would have 1o.000 members more. One for each comments they have abused.

Detection og signals : It is easy to check for signals towards a player without disturbing the player. It can be done from long distances. However, explainig anything to the wolf pack is in vain.
arzi arzi 10/18/2022 12:48
science22"As Hans Niemann has family in Denmark, I would like to make him a similar offer. Around his 20th birthday on June 20, 2003 I would like to arrange an unofficial world championship over 12 games between him and Magnus Carlsen in Denmark with a cash prize of 1 million."

Do you have a time machine of your own? Can you calculate or use a pocket calculator? Does it mean that on next year, 2023, Niemann is 40?
arzi arzi 10/18/2022 12:43
science22"Lajosarpad, arzi, mc1483 ... read the wise words of Johannes Fischer. We are all tired of using time on nothing."

That was the best thing you have ever said so far. Btw, what color is your tongue or should I use the word. "we"?
arzi arzi 10/18/2022 12:28
Lajosarpad:"But Carlsen said that Niemann did not even concentrate during the game and Carlsen also said that Niemann has beaten him like only a few players are able.."

Still, that doesn´t prove anything. You know as a mathematician that there is no 100% certain of anything. There are always a margin of error included. The world is quantized. Maybe Niemann was the first youth in our world who could do that. We cannot prove otherwise. Instinct is not enough to prove anything except maybe in the movies.
lajosarpad lajosarpad 10/18/2022 12:25
@with_cheats_you_lose I think the difference you describe between Stockfish's result and Silver's result is mainly due to forced moves. I would be interested to see how many forced moves were ignored, which were considered to be forced moves and which moves were matching an engine's choice according to mister Silver's analysis. I suppose that information would explain your doubts, but I cannot be sure about this until Albert Silver publishes more details about his findings, especially of the Carlsen-Niemann game.

@Arzi "This mean that Carlsen was even MORE childish in his action and sore loser. He was upset to LOSE a game! It doesn´t matter if the opponent has somewhere in his life been a cheater. Carlsen lost a game in a fair play. Great story! "

True, if e-mars was correct, then the above is the valid consequence. But Carlsen said that Niemann did not even concentrate during the game and Carlsen also said that Niemann has beaten him like only a few players are able. This strongly suggests that Niemann was cheating (according to Carlsen) and I treat this as a clear implicit accusation by Carlsen.
arzi arzi 10/18/2022 12:18
to Johannes Fischer, ok, will do, I LOOOVE facts.
lajosarpad lajosarpad 10/18/2022 12:16
@Jacob Woge

"The rating gain / loss correlation with live broadcast / no, or delayed broadcast should not deviate significantly from zero. If the correlation is (very) significant (and positive!) then e.g. FIDE has a strong incentive to pursue the matter. "

FIDE is already pursuing the matter, an investigation is on-going and I welcome that. If there is a tendency for some players to perform worse if a live transmission is in place, that's a strong reason to be suspicious, but not strong-enough to accuse them. Maybe abolishing live transmission and postponing it by an hour or so prevents the type of cheating we are discussing here.

"Such inspection should be kept secret, if future tournaments are to be included."

Exactly. If Carlsen strongly suspected that Niemann has cheated, then insinuating and then openly accusing him, while hysterically quitting the tournament was misbehaviour for several reasons:

1. The principle of innocent until proven guilty applies in Niemann's case, so the suspicion should have been communicated to authorities secretly and let them collect the evidence wihle behaving normally, as if nothing special would be in play
2. If Niemann has cheated, then, instead of catching him, Carlsen has made a martyr out of him
3. The larger community starts to talk about chess for the wrong reason
4. A world champion is also an ambassador of the game and needs to be extra careful about his behaviour. He has a right to be erratic and paranoid, like Fischer was, but that's certainly undesirable
lajosarpad lajosarpad 10/18/2022 12:15
@Science22

"Even book moves now become moves that cannot be classified as recommended by the computer! I think it is absurd. "

No, book moves were not included into the analysis, because there is no reason to think Niemann was cheating in the first few moves, given the fact that chess players usually memorize those moves.

"Albert Silver mention in the article the percentage 68 % using other computers to analyse Niemann, but that still leave us with 18 moves that no computer suggest at all. Which moves ? "

Good question! I would also like to know that.

"[...] because I knew as a scientist [...]"

:)

"Note to sane readers : The Bermuda Triangle where all informations is lost, mc1483,arzi and lajosarpad usually moves in and spam any discussion after some time. Just ignore them. Insults replace brain activity."

I have not insulted you, yet, you regularly insult and accuse me. I did not see many actual scientists who would engage into personal attacks like you do bravely benefitting from the anonimity of your nickname.

"You have to measure DURING the game"

That would disrupt his concentration.

@Jacob Woge "Agree, statistics on every single move cannot be the right method at this level. Because it does not fit with any plausible hypothesis of cheating method, being used by a pretty strong player. Why would he do that? "

The thing is that if a GM receives outside help only once in a game, then there will be no reason for a suspicion based on the moves. That is why any (!) analysis of cheatings at games looks at correlations between the player's move and the engine's move.
lajosarpad lajosarpad 10/18/2022 12:10
@Science22

"In the above, 54 of Niemann's 57 moves is within the first 3 choices of Stockfisch. This gives 94.7% accuracy."

First, the second suggestion of an engine can be much worse than the first, in the case of 29... Nc4, which, according to the video was the third possibility, the video claimed that it ruined the win.

Second, you assess the second and third move of the engine as precise moves, but they are less precise than the first move according to the engine.

Third, you ignore forced moves.

Fourth, even your summary claims that Niemann had 3 mistakes against Carlsen.

"Because the numbers have to be manipulated, "Great moves" and all other perfectly reasonable moves are thrown out of the Silver analysis, so that only the 33 "Best moves" moves remain."

You accuse Albert Silver of behaving dishonestly and deliberately shifting the input. This is an unworthy way of arguing. If you disagree with Albert Silver, why do you accuse him? Why do you decide not to remain in the bounds of valid arguments? As a matter of fact, he explained in the video that only a subset of moves is taken into consideration, so-called forced moves are not included into the analysis, because the player was pressurized by the necessities of the position during the game to make that move. If you wonder which moves were considered to be forced move, now that's a legitimate question and I also wonder about that.
Johannes Fischer Johannes Fischer 10/18/2022 12:09
@arzi @science22
Could you be so kind to keep the discussion civil, focused on facts, and, most importantly, free from insults and insinuations? Thank you!
lajosarpad lajosarpad 10/18/2022 12:07
@joachimus the cheating scenario of receiving only minor help as a GM is something we cannot analyze upon. I understand the reason for your suspicion, but awkward interviews are more of a proof of a lack of ability to collect one's thoughts for an interview rather than a proof of cheating.

@e-mars your point is technically correct, but fundamentally wrong. Carlsen did not say that Niemann cheated against him indeed, but he said Niemann has cheated more and more recently and then he also said that Niemann was not even concentrating against him in the game. So, to be precise: Carlsen did not say that Niemann has cheated against him, but he has strongly suggested so.

@Albert Silver "Myeah... I have more than one game as a 2100 player that meets that criteria."

That's a good point.

@Science22 "@Albert Silver : You fail to understand my critic. I will have to ask you again because it is so important for the evaluation of your analysis."

If you talk like this, don't be surprised if mister Silver will not answer you.

"This means that your method of analysis must at a minimum conclude that Niemann played with 93.4% correlation to computer programs' selected moves."

Nope, the method was checking for matches between the played moves and any engine's top moves (in the case of let's check) and it excluded only moves and opening moves, second and third lines do not come into play in this analysis.
Science22 Science22 10/18/2022 11:45
@with_cheats_you_lose : Thank you very much for the video. I cant stop laughing.

The snake - oil salsman explain in a rampling style how to cheat the opponent into a lose. "The greatest chess course of all time " . Holy smoke I did not know that. You dont need to be good at chess. You only need to be good at cheating. Then you will win, no matter your actual skills. Now I get it.

When the Mozart of Chess (Magnus Carlsen) meets Antonio Salieri ( Hans Niemann) the creative genius will suffer.
with_cheats_you_lose with_cheats_you_lose 10/18/2022 11:12
Hans Niemann is indeed a chess fraudster. Have a look at a snake-oil salesman promoting his chess course: https://youtu.be/1zkWaPNF7P4

"And that's why the/this course is the greatest of all time"...
arzi arzi 10/18/2022 10:57
Jacob woge:"Niemann puts in a decent performance in this championship. Half hour broadcast delay, tons of security unforetold. After a rather weak start, he has picked up and may go on plus. Same way Carlsen used to do. No crash and burn, as some have ventured to predict. The longer the tourney, the more youth will prevail. Old folks (30) get tired these days, no adjournments."

What about performance of Levon Aronian in that same championship? Is it possible that he is also a cheater? No superstar can play so poorly, right?
arzi arzi 10/18/2022 10:51
science22:"To Jacob Woge: You should be able to understand the need for many models in order to create security. We come from the same country Denmark, are both scientist. I still play active but I have not seen you for a while."

I don´t know about you as a scientist but I could believe that as a job from Jacob.
Science22 Science22 10/18/2022 09:57
Science22 8 minutes ago
@ Jacob Woge : Hejsa,
After a terrific start at the US Open where he rejected to comment on variations, it was clear that he was heading out into the cold. The report of all his online scams hung like a heavy cloud and it was time to play it safe and lose some games. But now he is back again, and that's why I predict we'll see terrific play from Niemann in the last three rounds.

I do not agree about the security in US Open. You have to measure DURING the game, not before because he does not use electronic device. You have to measure for other types of wavelenght ( like infrared). Dont worry, it will come.
Science22 Science22 10/18/2022 09:33
@Jacob Woge : Of course the statistics exist. There are plenty of investigations, see as an example Niemann 2019- 2020: https://twitter.com/atl_kings/status/1568656197812891653

I was only trying to emphasize that it is this and other types of solid models that FIDE must investigate thoroughly.

SCF Performance rating with no live games : ELO 2404
USCF Performance rating with live games : ELO 2610
USCF Rating change with no live games : ELO - 112
USCF Rating change with live games : ELO +131

You should be able to understand the need for many models in order to create security. We come from the same country Denmark, are both scientist. I still play active but I have not seen you for a while.
Jacob woge Jacob woge 10/18/2022 09:23
“After the game, Niemann suggested Qh4 for white which loses a piece at once. He can't see anything in the analyzes that matches the playing strength during the game. He knows why, and therefore he gives no more interviews.“

He actually did, one of the later rounds of U.S. Ch. The draw with Wesley So, I think.

Whlie I agree on the dismal analysis of the Firouzja game in particular, there are other possible explanations. The guy is nineteen, has just been harrassed by the (still) reigning WC, and would have his mind in a number of other places. No juvenile that I can recall has been put thru such a wringer before.

Still, I agree the game analysis ranges from from superficial to flawed.

Niemann puts in a decent performance in this championship. Half hour broadcast delay, tons of security unforetold. After a rather weak start, he has picked up and may go on plus. Same way Carlsen used to do. No crash and burn, as some have ventured to predict. The longer the tourney, the more youth will prevail. Old folks (30) get tired these days, no adjournments.

What still really pisses opponents off is his tendency for ridicule in post-game interviews. “On every level”. Some people just are like that, no intention of being polite. The friends you make that way will be there, but they will be few.
Science22 Science22 10/18/2022 09:08
The conclusion is absolutely clear. Niemann moves according to Stockfish with 94,3 % accuracy, and Albert Silvers model is useless.

I still remember the long period with Lance Armstrong when no one was allowed to say out loud that the man was doped. He took the joy out of watching the Tour de France because I knew as a scientist that you couldn't train to get such high number of red blood cell. He cheated, and mocked us all while he cheated.

My doubt is gone, he's cheating big time. The number of observations are overwhelming. For example, Carlsen has played the opening against Niemann in a lightning game up to move 5. But in the studio after the game, Niemann said with a chuckle that he had the whole variation ready up to move 15 "because Carlsen had played it before in 2018.

He sits and lies directly to our faces, and gets furious that chess.com publishes the fraud he himself has admitted in over 100 games. It's not the fraud that makes him angry, it's the publication. He has a personality which enables him to unconscionably violate all rules for personal gain, and lie without blinking an eye.

Note to sane readers : The Bermuda Triangle where all informations is lost, mc1483,arzi and lajosarpad usually moves in and spam any discussion after some time. Just ignore them. Insults replace brain activity.
Jacob woge Jacob woge 10/18/2022 08:55
@science22

Hejsa - your former post led me to believe statistics on rating gain/loss vs. yes/no of live broadcast already existed. Your next post seems to indicate this is not the case, but FIDE is working on it.

Please, which is it? I have seen claims of guilt based on such statistics, and if it doesn’t yet exist, how can you?
arzi arzi 10/18/2022 06:12
e-mars"Someone seems to forget that Carlsen NEVER said he thought he lost his game to Niemann because of Neimann's cheating at the Saint Louis event. Carlsen was planning to withdraw from the tournament before the tournament even started, but then opted to play. He was upset to lose against a cheater REGARDLESS Niemann cheated in the actual game."

This mean that Carlsen was even MORE childish in his action and sore loser. He was upset to LOSE a game! It doesn´t matter if the opponent has somewhere in his life been a cheater. Carlsen lost a game in a fair play. Great story!
Science22 Science22 10/18/2022 02:33
Albert Silver mention in the article the percentage 68 % using other computers to analyse Niemann, but that still leave us with 18 moves that no computer suggest at all. Which moves ?

After the game, Niemann suggested Qh4 for white which loses a piece at once. He can't see anything in the analyzes that matches the playing strength during the game. He knows why, and therefore he gives no more interviews. Fraud speaks for itself.
Science22 Science22 10/18/2022 02:33
@mc1483 : I send an answer at once, but it ws not published. So her we go again. Here's what the video gives of exact percentages for Niemann with Carlsen's numbers in parentheses

1. Great move : 9 ( 3)
2. Best move : 33 (22)
3. Excellent move: 1 (12)
4. Good move : 3 (6)
5. Book moves : 5 (5)
6. Inaccuracy : 3 (5)
7. Mistakes : 3 (1)
8. Blunders : 0 (2)


In the above, 54 of Niemann's 57 moves is within the first 3 choices of Stockfisch. This gives 94.7% accuracy. Since inaccuracies can be assessed in many ways, the percentage is reduced by Stockfish to 93.4%.

As you can see in the table, Stockfish has a move that is ranked higher than "Best move". It is called "Great move" and these fantastic moves (only win) have Niemann 9 of.

Because the numbers have to be manipulated, "Great moves" and all other perfectly reasonable moves are thrown out of the Silver analysis, so that only the 33 "Best moves" moves remain. Even book moves now become moves that cannot be classified as recommended by the computer! I think it is absurd.

It is naturally to be expected that a professional fraudster at this level incorporates minor inaccuracies. But there is nothing that gives the opponent a chance to get a win. No blunders, no really serious mistakes.
Science22 Science22 10/18/2022 02:28
@Jacob Woge : The rating gain / loss correlation with live broadcast / no, or delayed broadcast should not deviate significantly from zero. If the correlation is (very) significant (and positive!) then e.g. FIDE has a strong incentive to pursue the matter. If not, then this is not it.

I completely agree with Jacob Woge. Thoroughly worked out statistical models with clear assumptions that can test for significant deviations have exactly the same value as evidence in this case as DNA has for a serious crime.

FIDE is forced to use very solid statistical models to draw a safe conclusion in the matter. They will do just that. It will be the statistics that bring down Hans Niemann and nothing else.

Although in principle you can send infrared radiation to water molecules in a person's ear and create an acoustic signal, you have to prove that Niemann actually received the signal, and you can't. If you block the signal and he plays badly you have not proven any cheat. He can always claim that he is now in an affect and cannot play good chess anymore.
with_cheats_you_lose with_cheats_you_lose 10/18/2022 01:24
I am afraid you are also wrong, the video with Stockfish's analysis does not confirm Let's Check analysis results obtained by Albert Silver for the Carlsen - Niemann game in the Sinquefield Cup. If you take a closer look at the end of video, 33 were best moves and 9 were great moves out of a total of 57 moves, not considering the 5 book moves, the engine correlation for Niemann should be: (33+9)/(57-5) = 81%. So, 81% of Niemann's moves, not considering the first 5 opening moves, were the Stockfish's top moves. Carlsen has (22+3)/(57-5) = 48% of engine correlation. These values are higher than the ones suggested by Albert Silver from Let's Check analysis: 68% for Niemann and 37% for Carlsen. Let's Check analysis results should normally be higher, and not lower, than those obtained with a single chess engine, as explained in this article. Only the author knows how he came up with those values.

I did some chess engine analysis of that game to confirm the findings, with Stockfish 15 at depth 24, and obtained similar results to the Stockfish's video analysis: https://pastebin.com/DyXTPfzi

The engine accuracy values of 93,4% for Niemann and 83,9% for Carlsen are calculated from the centipawn losses and they are not the same as engine correlation values. More information in this link: https://support.chess.com/article/1135-what-is-accuracy-in-analysis-how-is-it-measured
tauno tauno 10/18/2022 01:16
Did Hans Niemann Cheat? Reviewing the statistical evidence
by Nate Solon - Oct 04
https://zwischenzug.substack.com/p/did-hans-niemann-cheat

Note: The Chess.com Niemann Report was also published Oct 04.

The Perpetual Chess Pod
Bonus Pod - FM Nate Solon joins to Discuss Chess.Com's Report
about the Carlsen/Niemann Controversy - Oct 05
https://www.perpetualchesspod.com/new-blog/2022/10/5/bonus-pod-fm-nate-solon-joins-to-discuss-chesscoms-report-about-the-carlsenniemann-controversy
mc1483 mc1483 10/17/2022 11:23
@Science22: you're not even capable to understand the videos you yourself post here. The video with Stockfish's analysis indeed confirms Albert Silver's findings, as it clearly states that 33 times (out of 57) the best move was chosen, that is 33/57 = 57%.
The 93,4% accuracy is just a parameter that evaluates how much, in the whole game, the moves were accurate according to the engine, and has nothing to do with the Let'sCheck's analysis.
shivasundar shivasundar 10/17/2022 11:07
Here we go again... okay give me some time :-)
tauno tauno 10/17/2022 10:51
Let’s forget Let's Check. According to ChessBase you can proof cheating by using Centipawn Analysis.

ChessBase 16 Help
http://help.chessbase.com/CBase/16/Eng/index.html?centipawn_analysis.htm

“Strong GMs in long games reach CP losses between 15 and 25. Club players achieve between 30 and 60. However, the values can vary to a great extent. Even club players can sometimes absolutely manage low values such as 10.

A low value in an individual game is not proof of cheating. In quiet draws the values tend to be lower.

Reliable values can only result from larger numbers of games.”
Jacob woge Jacob woge 10/17/2022 10:46
“I think the entire issue is with the concept of cheating , that all moves have to be correlated to top engine choices.
This is the approach which I find not useful because for GMs it is enough to receive signal in key moment when evaluation changes and then follow the route to victory. That's why I find Niemann game case such controversial.”

Agree, statistics on every single move cannot be the right method at this level. Because it does not fit with any plausible hypothesis of cheating method, being used by a pretty strong player. Why would he do that?

“What has been decisive for my own assessment has been the huge difference in performance when Niemann plays in tournaments without live transmission, where he has lost ELO every time the last 3 years, and those with live transmission where he has gain rating every time. No statistical analysis can explain this as a natural deviation.”

Also agree. There is an underlying hypothesis one tries to prove, namely that transfer of illicit information is connected with the live broadcast. I would like to see the OtB statistics referred to, for such is my wont. I haven’t Noticed direct reference so far, Any clues about where to look?

The rating gain / loss correlation with live broadcast / no, or delayed broadcast should not deviate significantly from zero. If the correlation is (very) significant (and positive!) then e.g. FIDE has a strong incentive to pursue the matter. If not, then this is not it.

There are two snags. The first being, broadcast or not may itself depend on tournament strength. The second, that once a cheater becomes aware of this being investigated, the avoidance game begins. Such inspection should be kept secret, if future tournaments are to be included. Otherwise, limited to hindsight. And I guess the cat is out of the bag by now ...
Science22 Science22 10/17/2022 10:08
joachimus: You point out something very important :

"If he would have thought/analyse over the board then he wouldn't have problem with sharing his thoughts about the games and post-analysis and certainly not creating stories like "by miracle I checked this game this morning" which he repeated few times. Not mentioning his victory over Yoo in very complicated position which he just comment as wonderful game which speaks for itself. Come on really? "

Super precisely said. It is in these moments when he is most excited about himself that he forgets that the eyes of the world are upon him. He also said he checked the opening because Carlsen had played it in a game in 2018. Which does not exist. Niemann creates explanations for his choices that simply do not hold up to closer scrutiny.
Science22 Science22 10/17/2022 09:24
@Albert Silver : You fail to understand my critic. I will have to ask you again because it is so important for the evaluation of your analysis.

The computer program Stockfish says that Niemann did not make a single move in the game against Carlsen that was not in the computer's top 3 choices, and that he chose the computer's first move in 93.4% of the moves:

Stockfish : https://www.youtube.com/watch?v=BbEiW-60hf0.

This means that your method of analysis must at a minimum conclude that Niemann played with 93.4% correlation to computer programs' selected moves. There is no other logical conclusion. Yet you say he played with only 57% correlation.

Would you please list the moves out of 57 moves in the game where Niemann does not play as a computer's preferred choice?

You clarify in your assumptions that you have expanded the framework for when a player's choice matches a computer's choice, and still end up with a percentage of Niemann choices that is far below what just a single program Stockfish conclude. Where is the logic ?
Albert Silver Albert Silver 10/17/2022 09:00
"Niemann did not made a single move which was not within the first 3 choice of Stockfish"

Myeah... I have more than one game as a 2100 player that meets that criteria.