Is Hans Niemann cheating? - World renowned expert Ken Regan analyzes

by Albert Silver
9/20/2022 – It is the story that refuses to die, and with the renewed silent protest by World Champion Magnus Carlsen it is little wonder too. Could he be justified in these dramatic gestures? The world's greatest expert on cheating detection in chess, Professor Kenneth Regan, has analyzed all of Hans Niemann's games over the last two years, online and offline, and renders his verdict.

ChessBase Magazine 209 ChessBase Magazine 209

2022 Candidates Tournament with videos by Rogozenco and Ris, "Special" on Anna and Mariya Muzychuk, opening videos by King, Marin and Sokolov. 11 opening articles with new ideas for your repertoire and much more.

More...

In view of the tremendous commotion the unraveling of the story has caused, speculation has continued to be rife. There are pitched battles in the recesses of social media with some wondering why an event such as Meltwater allows a 'cheater' to play and they do not blame Magnus for his attitudes, while others bemoan the lack of data to properly render judgement. 

Amateurs have sprung up with their own detailed verdicts, providing spreadsheets comparing the broadcast and offline event results of Hans Niemann, others doing Elo performance charts with lovely bell curves, and others running his games from events such as the Capablanca Memorial (that he won) through engines to prove his guilt beyond any doubt. As one grandmaster confided in private, "The Capablanca video is pretty damning".

The problem with all of these is the lack of scientific rigor and depth to properly make such analyses valid points of reference. On the flip side, with a lack of better sources of information, anything goes. 

Who is Professor Kenneth Regan?

Ken Regan was better known as a chess player before becoming an academic, reaching the IM title as a junior, and qualifying for the US Championship even. He explains he never planned on a career in chess, despite his talent and enjoyment for the game, and eventually obtained a PhD in Mathematics from Oxford University in complexity theory. 

In 2006, at the cusp of the infamous 'toiletgate' controversy during the World Championship, he was brought in to provide expert analysis of the cheating accusations leveraged by the Topalov Team against Vladimir Kramnik. The claims were that Kramnik's moves in Game Two, the point of bitter contention, achieved an unreal 80%+ match with the best engine of the time, Rybka. Dr. Regan confirmed this claim, but debunked the cheating accusations. He demonstrated that many of the moves that matched the engine were in fact quite forced, and if you eliminated these forced moves from the analysis, then the overall number of moves that matched engine choices was quite normal. 

Dr. Ken Regan on the cover of Chess Life magazine for his pioneering work in cheating detection in chess

Showing this greater depth and rigor in his analysis immediately propelled him into the limelight as the new top expert, even if many did not understand or accept the depth of his research. He has since been used by FIDE to supplement expert analysis in cases such as Sebastien Feller, Igor Rausis, and general oversight of major events.

Regan has developed statistical methods to prove cheating in a series of scientific papers which are accessible on his homepage.

The Interview

Dr. Ken Regan was kind enough to agree to an interview in which he not only explains how his methods work, but also the detailed work he did on the Carlsen-Niemann Affair and what his conclusions were. 

This is a slightly abridged version of the full interview (see below), removing notably some of the lengthier technical explanations.

Are broadcast games more suspicious?

In the interview I queried specific points being brought up online and elsewhere such as a possible difference between games that were broadcast and games that were not.

"What I'm saying, as justifying my not needing to take the time to individually look into tournaments to see which were broadcast and which were not, is that if there is any bias in my data, then it's towards broadcast games (i.e more of it is analyzed due to availability) and yet I show something entirely normal."

ChessBase 16 - Starter Package Edition 2021

Your key to fresh ideas, precise analyses and targeted training!

On the quality of his system

"My system deliberately does not use specific chess knowledge, it's all based on the quantitative data from the computer's evaluations of the moves transplanted into a utility function, which is then fed into a predictive analytic model that really works in a broadly similar way to how economists base predictive analytical models on utility functions."

Screenshot of Dr. Regan's computer and the breakdown of the results

He also mentions that the system is also capable of highlighting players who are better than others at inducing errors from their opponents. Notables at this are Mikhail Tal and Alexey Shirov.

On the result in the Capablanca Memorial

You will find this dealt with in great detail, including a breakdown of Niemann's error rate, the error rates of his opponents, and how well his opponents played against him.

It turns out that not only did Niemann perform almost exactly as expected of him in terms of move accuracy, but his opponents are the ones who underperformed against him. The 0.152 on the right in blue is the error rate of his rivals.

Likelihood of a miracle in chess results

"Littlewood's Law says that if you observe a million happenings one of them will have a million-to-one prior probability. which is the common definition of a miracle. So if you see a million things a day, then a miracle happens every day. And in online chess a million games per day is the norm."

The verdict

Dr. Regan analyzed all of Hans Niemann's games over the last two years, including online games, such as played on Chess.com and their events, and his conclusion is there is no reason whatsoever to suspect him of cheating. The wide range of results in a bell curve, with some good and some bad, is actually a sign of a healthy distribution of results. Many of the so-called points of suspicion are in fact quite normal and suspicion is really the result of faulty analysis by zealous amateurs. Even online his play has been quite devoid of anything unusual.

Full results of his data can be found at this link

Full interview

Here are a few mainstream news reports:


Born in the US, he grew up in Paris, France, where he completed his Baccalaureat, and after college moved to Rio de Janeiro, Brazil. He had a peak rating of 2240 FIDE, and was a key designer of Chess Assistant 6. In 2010 he joined the ChessBase family as an editor and writer at ChessBase News. He is also a passionate photographer with work appearing in numerous publications, and the content creator of the YouTube channel, Chess & Tech.
Discussion and Feedback Submit your feedback to the editors


Discuss

Rules for reader comments

 
 

Not registered yet? Register

arzi arzi 9/20/2022 06:40
SuperGm79:"I completely understand why Magnus does not want to play him....I personally think Magnus has the right to forfeit the game if he does not want to play him. "

Unbelievable how such a simple thing can be very complicated for some people. If there are certain rules to follow, then you can't do things on your own. Fide has clear rules that everyone should follow, even the world champion. If the world champion doesn't want to follow them, then others don't have to either. If no one follows the rules, no organization will function anymore. If the sponsors cannot trust the players' actions, then there will be no money tournaments either. Simple.
If Carlsen suspected a scam, cheating, he had only one right way. Continue the Sinquefield Cup to the end. After this, he should have asked for an investigation into the cheating. For future tournaments, he should have demanded that Niemann not be in the same tournament as a condition of his playing. Now he let everyone down: the players, the sponsors, the public, Fide. You have to do this the right way or forget it entirely.
soulblazer soulblazer 9/20/2022 06:37
So much hate toward Magnus. The guy just quited the tournement without sharing his motives. He owes apologies to no one, except maybe Caissa for the miss beautiful games he would have played. Love for everyone from Canada.
adbennet adbennet 9/20/2022 06:35
I want to thank ChessBase for presenting this information from Dr. Regan, and just generally for taking a balanced editorial approach to this ongoing news storyline. However, missing from the article is a relevant question, who or what prompted Dr. Regan to do his analysis? Was it academic curiosity? Did FIDE ask him to do it, or did some private party commission it? Would like to see this addressed, thanks.
Jack Nayer Jack Nayer 9/20/2022 06:07
Sorry, but this is nonsense. If I play 100 games a year and I cheat in two, using a program to generate two moves in each game, there is very simply absolutely nothing statistical to analyse.
checkmate3000 checkmate3000 9/20/2022 05:53
I think Dr. Regan's system may not necessary catch a smart cheater who only cheated in significant games. For instance, analyzing 200 games of a suspected cheater will not give us any clue if the said person only "cheated" in 3-5 crucial games.

The only way to better judge this situation is if we let the suspected cheater play against the 10 top players over several in-person tournaments and have him/her searched meticulously for any potential transmitter devices.
tauno tauno 9/20/2022 05:53
The rules of chess can be a jungle and sometimes difficult to understand for the general public, so I would like to clarify a few points related to the current situation:

1. If you lose a game, you may leave the entire tournament without giving any reason, but only if it happens during the first half of the tournament (in agreement with the organizer). In that case, your result in the tournament will be cancelled. The games will not be included in the official statistics or negatively affect your ELO rating.

2. Losing a game on purpose is allowed, but only during the first half of the tournament and only once per player (taking into account financial and other consequences for the rest of the players).

3. If you are suspected of cheating, you can be banned without presenting evidence. (If there is evidence, it should be kept secret, because otherwise the other chess players can use that information to cheat.)

4. You have the right to remain silent. If you tell the truth, you can get into big trouble. Anything you say can and will be used against you in a FIDE court.
WillScarlett WillScarlett 9/20/2022 04:20
Experience and reflection has led me to the following tentative conclusions (pending irrefutable evidence to the contrary).
A short list of people to shun or at least avoid:
1.) Cheaters
2.) Lawyers
3.) Statistical experts
4.) Journalists
herralex herralex 9/20/2022 03:58
I have a question if I understood Dr.Regan correctly. What if a computer engine is running n lines ordered from best to worst and a cheater manipulates his precision score by choosing the worst line that keeps the position within a certain value (let's say 0.5 either way()? He can then choose the computer's top line only when it significantly alters the evaluation in his favor. A low precision score might also be a clue to cheating if it's too low for the player's purported strength.
Frederic Frederic 9/20/2022 03:29
@GrahamBanks: ignoring obvious moves in cheating analysis is obviously necessary. The ChessBase Weighted Error detection has done this for many years. In 1.QxQ KxQ, the black move should clearly not be evaluated ("wow, engine move, he must be cheating!"). But our software goes further: it only takes non-trivial moves into account in its checking, only moves that come up in a reasonably deep search.

I need to keep this vague and not provide any more details. I do not want to give cheaters detailed instructions on how to better avoid detection in the future.
John Maccormack John Maccormack 9/20/2022 03:29
Maybe losing that game to Niemann in St. Louis has scrambled things inside Carlsen's head, and the champ is now just spooked about Niemann beating him again.
Maybe it's a psych thing, and not a plausible suspicion of cheating.
Sometimes the changing of the generational guard can be a difficult process.
Supergm79 Supergm79 9/20/2022 03:10
Hi,
Two remarks:
- It is very difficult to track cheating behavior, esp. when someone does this occasionally. At the same time, strong GMs can indeed play some games with almost perfect chess..it is very difficult to track differences here in real play and cheated play.
- I completely understand why Magnus does not want to play him. It is very frustrating that you play someone known for cheating and sudden extreme differences in terms of quality of play. I disagree with statement like "...Magnus should prove that Hans is cheating without which he has to play and not act". My question is then: if you strongly feel that something is not correct and the other party is probably cheating but you cannot prove it, what should you do then? => I personally think Magnus has the right to forfeit the game if he does not want to play him. His only mistake he did in my view is that he posted this youtube videa on twitter of Mourinho...

ps. We should also not forget that Magnus never claimed officially that Hans is cheating. That is what the audience is making from this!! And even if this would be the case, it is difficult for Magnus to convey as he has no official proof....

Cheers,
Graham Banks Graham Banks 9/20/2022 02:46
These snippets sum it up for me:

The world's greatest expert on cheating detection in chess...

In 2006, at the cusp of the infamous 'toiletgate' controversy during the World Championship, he was brought in to provide expert analysis of the cheating accusations leveraged by the Topalov Team against Vladimir Kramnik. The claims were that Kramnik's moves in Game Two, the point of bitter contention, achieved an unreal 80%+ match with the best engine of the time, Rybka. Dr. Regan confirmed this claim, but debunked the cheating accusations. He demonstrated that many of the moves that matched the engine were in fact quite forced, and if you eliminated these forced moves from the analysis, then the overall match in moves was in fact quite normal.

He has since been used by FIDE to supplement expert analysis in cases such as Sebastien Feller, Igor Rausis, and general oversight of major events.

Regan has developed statistical methods to prove cheating in a series of scientific papers which are accessible on his homepage.
A Alekhine A Alekhine 9/20/2022 02:27
More bad behavior from Magnus. The world champion should be bigger than this display of pettiness. Whatever Niemann did as a junior, he has apologized for it. Now Niemann is playing top-level chess with the big boys, and there is not the slightest hint that Niemann is doing anything wrong. Instead it is Carlsen who is shaming himself.
nbeqo nbeqo 9/20/2022 02:06
Here is my verdict, the guy has displayed an idiotic behavior, he cheats in critical moments only, and once a cheater always a cheater! The reputation speaks for itself!
NYTed NYTed 9/20/2022 02:04
The Little Boy Who Cried Wolf
mc1483 mc1483 9/20/2022 01:53
First not willing to defend the title, now dealing with paranoia's issues. Carlsen is slowly becoming Fischer.
ChessSpawnVermont ChessSpawnVermont 9/20/2022 01:47
I see a civil court action looming against Carlsen and chess dot com for defamation of character. Should Niemann decide to go in that direction there will be any number of top tier defamation litigators willing to take Neimann's case on a contingency basis due to the very deep financial pockets of both Carlsen and chess dot com. As an attorney, I would not want to be stuck with the task of trying to defend them. Then again, with a sizable retainer up front and a good retainer replenishment clause, it could be a financial winner for their defending counsel.
arzi arzi 9/20/2022 01:33
Let's think about this in more detail for a moment. If Niemann was a fraud, why hadn't chess.com blocked Niemann from playing on their platform? Why did chess.com decide to throw Niemann out of their site when Carlsen was quitting his tournament with his cryptic sentences? Why does chess.com make their decision based on Sinquefield Cup, without having any evidence of what actually happened there? It's like persons complaining afterwards that shit is in their pants when they could have gone earlier in time to the toilet.
arzi arzi 9/20/2022 01:12
To FF: However, chess.com did not deny access to its services before the tournament, but after Carlsen's loss and quitting the tournament. Where is the logic of this policy? Business policy between chess.com and Carlsen?
Frits Fritschy Frits Fritschy 9/20/2022 01:03
As stated before, chess.com doesn't directly accuse people of cheating. They just have the policy that they deny access to their services if they have strong reasons to believe someone cheats. You must agree to that policy before entering their competitions. That is because statistical evidence alone is never enough outside private competitions, neither under FIDE rules nor in court. But their house, their rules. However, when chess.com becomes a monopolist, a judge may consider that professional players lose the possibility of making an income, and will demand juridically acceptable proof. Apart from that, chess.com has a strong commercial reason to make online chess mainstream, and may use its power to get FIDE to take over their rules.
arzi arzi 9/20/2022 12:57
AW:"It is close to impossible to detect a smart cheater, even with the best statistical methods."

How come? Can´t you check cheaters chess moves and compare them to the moves of different chess engines? Or do you mean that some unknown and brilliant GM (better than, for example, Magnus) is helping a cheater with his own new moves? A GM who purposely makes bad moves to fool the cheat inspectors.
GabrielCuri GabrielCuri 9/20/2022 12:46
These acts of hysterical divism by Carlen must be stopped now. Fide has to ban him for playing for six months, like they did with Kariakin. And Niemann should be talking with some New York lawyers, who will take from the idiot that we sadly have as world champion more money than he ever thought he could loose.
Arnold Weber Arnold Weber 9/20/2022 12:45
It is close to impossible to detect a smart cheater, even with the best statistical methods.
It is hard to cheat smart in online blitz (and rapid), because of the limited time. But over the board, I think it is simple, especially if there is a second person involved.
PhishMaster PhishMaster 9/20/2022 11:58
@sebtak, you will never be able to prove a negative, so lacking any firm results, considering Regan is the most widely-accepted expert in the field, YES, you are as wise as you are going to get.
Arminio12 Arminio12 9/20/2022 11:55
@ jenyes / sebtak
Niemann admitted having cheated twice, i.e. at 12 and at 16 years of age, which is before those "last two years" Dr. Regan has analysed. That doesn't necessarily mean that Niemann has told the whole story or that chess.com may not have further evidence. However, if they did have anything substantial to the extent they can be "legally confident" about their data, it would be nearly criminal not to take (court) action. They are not doing anything. Yes, they have closed Niemann's account (temporarily, it would seem by their own words) but without any explanation. That amounts to next to nothing, which means it looks very much like they haven't got any real proof at all. Notice that no one seems to have any proof: Dr. Regan found nothing in the last two years, the Sinquefield Cup arbiters saw no evidence whatsoever of foul play by anyone, Carlsen and chess.com pretend there's something about Niemann without even accusing him explicitly of anything, or without even, apparently, being able to come up with anything to justify their actions, and the players, both in the Sinquefield Cup and here, though they may express opinions on Niemann having cheated before, take no action of any kind to support Carlsen or take sides against Niemann. Apart from that several experts have had a look at the Carlsen-Niemann game in the Sinquefield Cup and concluded Carlsen's loss was caused by poor play on his part rather than exceptional play on the part of Niemann (he didn't play all that well anyway). Innocent till proven guilty, that is the key thing. And you can't condemn the young man here and now for what he did in the past, even if that was more than he admitted. "Once a cheat, always a cheat" is no argument. Without serious proof, Niemann walks free. That is a basic human right.
mc1483 mc1483 9/20/2022 11:50
The cheating on Chess.com occurred 3 years ago, while Regan's analysis covers the last two years. That's why he did not detect anything suspicious.
lajosarpad lajosarpad 9/20/2022 11:26
@sebtak he was caught when cheating online, so it was not so easy to cheat after all. Also, the fact that he cheated in games previously, years ago does not mean that he cheated against Carlsen. Even the justice system does not accuse a former convicted person of another crime if there is no evidence. In our case no evidence was prevented that a crime existed in the first place, let alone Niemann being its perpetrator. The current analysis is a strong argument against the cheating allegations and those who suspect Niemann of cheating and aim to destroy his career with these allegations should either present their evidence or retract the accusation and apologize.
arzi arzi 9/20/2022 11:00
Is it possible to get clear evidence from Carlsen about Niemann's cheating. Wouldn't it be fair to the sponsors, the public and perhaps also to Niemann to get the facts out without mysterious puns?

"Magnus, tell us the facts! Don't hide them anymore! If in doubt, don´t ask Mourinho for help."
jenyes jenyes 9/20/2022 10:42
Very interesting article but to imply that this is sufficient for online play is a stretch. Hans has admitted to cheating online. Chess.com caught the two occurrences that he admitted to. They have evidence of more occurrences. Keep in mind, their policy for titled players is to have the data hold up in court so they are "legally" confident. For online games, this analysis misses data that could easily change and / or override the conclusion. Chess.com could have access to video, audio, and input data. There are also methods for analyzing more data in OTB games too, though I doubt that these methods are financially practical.
arzi arzi 9/20/2022 10:39
sebtak, if Dr. Regan had proven in his studies that Niemann cheated in his games, would you be wiser after that? What is the ultimate prove to all of us that Niemann cheated in his game against Carlsen? Should Niemann's confession be forced? With electricity or working water torture? Maybe we just trust our intuition and punish Niemann so that he really learns. Sentenced to 20 years as a collector of used matches, without pay.
sebtak sebtak 9/20/2022 10:11
Surely, Chess.com banned him for cheating and Hans himself admitted that he did cheat online. So one can have reasonnable questions about the sensitivity of a cheating detection test when it can't find it when actual cheating has occured. I have many questions for another day, but for the sake of the question at hand - did Hans cheat against Carlsen OTB, we are none the wiser after reading Dr Regan's analysis.
arzi arzi 9/20/2022 09:10
"Dr. Regan analyzed all of Hans Niemann's games over the last two years, including online games, such as played on Chess.com and their events, and his conclusion is there is no reason whatsoever to suspect him of cheating."

As I suspected. The only way for Magnus now to move on with his life is to admit his mistakes, apologize for his paranoia to the public and sponsors and his bad losing game to Niemann. No one is perfect, not even Magnus.