Is Hans Niemann cheating? - World renowned expert Ken Regan analyzes

by Albert Silver
9/20/2022 – It is the story that refuses to die, and with the renewed silent protest by World Champion Magnus Carlsen it is little wonder too. Could he be justified in these dramatic gestures? The world's greatest expert on cheating detection in chess, Professor Kenneth Regan, has analyzed all of Hans Niemann's games over the last two years, online and offline, and renders his verdict.

ChessBase Magazine 209 ChessBase Magazine 209

2022 Candidates Tournament with videos by Rogozenco and Ris, "Special" on Anna and Mariya Muzychuk, opening videos by King, Marin and Sokolov. 11 opening articles with new ideas for your repertoire and much more.

More...

In view of the tremendous commotion the unraveling of the story has caused, speculation has continued to be rife. There are pitched battles in the recesses of social media with some wondering why an event such as Meltwater allows a 'cheater' to play and they do not blame Magnus for his attitudes, while others bemoan the lack of data to properly render judgement. 

Amateurs have sprung up with their own detailed verdicts, providing spreadsheets comparing the broadcast and offline event results of Hans Niemann, others doing Elo performance charts with lovely bell curves, and others running his games from events such as the Capablanca Memorial (that he won) through engines to prove his guilt beyond any doubt. As one grandmaster confided in private, "The Capablanca video is pretty damning".

The problem with all of these is the lack of scientific rigor and depth to properly make such analyses valid points of reference. On the flip side, with a lack of better sources of information, anything goes. 

Who is Professor Kenneth Regan?

Ken Regan was better known as a chess player before becoming an academic, reaching the IM title as a junior, and qualifying for the US Championship even. He explains he never planned on a career in chess, despite his talent and enjoyment for the game, and eventually obtained a PhD in Mathematics from Oxford University in complexity theory. 

In 2006, at the cusp of the infamous 'toiletgate' controversy during the World Championship, he was brought in to provide expert analysis of the cheating accusations leveraged by the Topalov Team against Vladimir Kramnik. The claims were that Kramnik's moves in Game Two, the point of bitter contention, achieved an unreal 80%+ match with the best engine of the time, Rybka. Dr. Regan confirmed this claim, but debunked the cheating accusations. He demonstrated that many of the moves that matched the engine were in fact quite forced, and if you eliminated these forced moves from the analysis, then the overall number of moves that matched engine choices was quite normal. 

Dr. Ken Regan on the cover of Chess Life magazine for his pioneering work in cheating detection in chess

Showing this greater depth and rigor in his analysis immediately propelled him into the limelight as the new top expert, even if many did not understand or accept the depth of his research. He has since been used by FIDE to supplement expert analysis in cases such as Sebastien Feller, Igor Rausis, and general oversight of major events.

Regan has developed statistical methods to prove cheating in a series of scientific papers which are accessible on his homepage.

The Interview

Dr. Ken Regan was kind enough to agree to an interview in which he not only explains how his methods work, but also the detailed work he did on the Carlsen-Niemann Affair and what his conclusions were. 

This is a slightly abridged version of the full interview (see below), removing notably some of the lengthier technical explanations.

Are broadcast games more suspicious?

In the interview I queried specific points being brought up online and elsewhere such as a possible difference between games that were broadcast and games that were not.

"What I'm saying, as justifying my not needing to take the time to individually look into tournaments to see which were broadcast and which were not, is that if there is any bias in my data, then it's towards broadcast games (i.e more of it is analyzed due to availability) and yet I show something entirely normal."

ChessBase 16 - Starter Package Edition 2021

Your key to fresh ideas, precise analyses and targeted training!

On the quality of his system

"My system deliberately does not use specific chess knowledge, it's all based on the quantitative data from the computer's evaluations of the moves transplanted into a utility function, which is then fed into a predictive analytic model that really works in a broadly similar way to how economists base predictive analytical models on utility functions."

Screenshot of Dr. Regan's computer and the breakdown of the results

He also mentions that the system is also capable of highlighting players who are better than others at inducing errors from their opponents. Notables at this are Mikhail Tal and Alexey Shirov.

On the result in the Capablanca Memorial

You will find this dealt with in great detail, including a breakdown of Niemann's error rate, the error rates of his opponents, and how well his opponents played against him.

It turns out that not only did Niemann perform almost exactly as expected of him in terms of move accuracy, but his opponents are the ones who underperformed against him. The 0.152 on the right in blue is the error rate of his rivals.

Likelihood of a miracle in chess results

"Littlewood's Law says that if you observe a million happenings one of them will have a million-to-one prior probability. which is the common definition of a miracle. So if you see a million things a day, then a miracle happens every day. And in online chess a million games per day is the norm."

The verdict

Dr. Regan analyzed all of Hans Niemann's games over the last two years, including online games, such as played on Chess.com and their events, and his conclusion is there is no reason whatsoever to suspect him of cheating. The wide range of results in a bell curve, with some good and some bad, is actually a sign of a healthy distribution of results. Many of the so-called points of suspicion are in fact quite normal and suspicion is really the result of faulty analysis by zealous amateurs. Even online his play has been quite devoid of anything unusual.

Full results of his data can be found at this link

Full interview

Here are a few mainstream news reports:


Born in the US, he grew up in Paris, France, where he completed his Baccalaureat, and after college moved to Rio de Janeiro, Brazil. He had a peak rating of 2240 FIDE, and was a key designer of Chess Assistant 6. In 2010 he joined the ChessBase family as an editor and writer at ChessBase News. He is also a passionate photographer with work appearing in numerous publications, and the content creator of the YouTube channel, Chess & Tech.

Discuss

Rules for reader comments

 
 

Not registered yet? Register

Lunaticx Lunaticx 9/21/2022 06:01
What an arrogance to write this : "suspicion is really the result of faulty analysis by zealous amateurs." Even Niemann himself admitted to cheating online, propably on several ocassions. What does your analysis and miracle speaking think about that ?
Odinsgrandson Odinsgrandson 9/21/2022 05:57
This already feels like Magnus is abusing his position of influence as a renown world champion to insinuate cheating without evidence. Magnus is a world champion of Chess, and this passive aggressive BS is unbecoming and inappropriate.

It looks very much like Magnus is trying to ruin Hans' career without committing himself to actually making allegations in a sort of Kafkaesque way.

The fact is, allegations of cheating have ALWAYS been around and should have anti-cheating measures in place rather than take the passive aggressive non-word of a world champion throwing a hissy fit.

Regen can no more prove that Magnus wasn't cheating than Nieman. I guess everyone should just refuse to play Magnus after the first move until Magnus proves that he hasn't cheated.
arzi arzi 9/21/2022 05:56
to soulblazer, wouldn't it be simpler not to tell the players the rules at all? If a player does something he thinks is right, but the organizers think is wrong, the player is just thrown out without any explanation. It would be exciting for the players and the audience. Everyone would enjoy it except the ejected player. Cool, I support that system.
soulblazer soulblazer 9/21/2022 05:40
Yes I support any players quitting a tournament or resigning after 1st move. If the player abuses it, then, you just don't invite him anymore.
arzi arzi 9/21/2022 04:34
Ellrond, are you saying that Carlsen was on some nasty drugs that caused him to play so badly in his losing game against Niemann? Ok, now I understand your Lance story. Thanks for sharing.
Marseille07 Marseille07 9/21/2022 04:26
@mc1483 Someone already talked about that. As arzi said, if it were the case then Carlsen needs to investigate his own team and how Niemann might have stolen his opening preparation.

Besides, while it'd explain the Sinquefield Cup withdrawal, it doesn't explain resigning on move 2.
lajosarpad lajosarpad 9/21/2022 03:54
Innocent until proven guilty. It has been analyzed and no reasons were found to think Niemann was cheating. If someone disagrees and finds fault in the analysis, then his position can be one of the following:

1. It is unknowable whether Niemann cheated. In this case, case closed, as Niemann is innocent until proven guilty.

2. The analysis was bad. In this case I advise the proponents of this critique to make a better analysis and show how it was better than Regan's.
Ellrond Ellrond 9/21/2022 02:38
arzis and H.N. reactions to hints and reasonable suggestions look very similar to reactions of Lance Armstrong (TdF winner) in his doping time.
troy878 troy878 9/21/2022 12:14
I have Fritz 11 and he gives 22 lines of analysis. What if Neiman uses the draw line and waits for the opponent to make a mistake and then makes the best move? and win game . Any beginner can do this easily. The one who has been on the throne of chess for four times is not stupid
arzi arzi 9/21/2022 11:48
to mc1483, it's called preparation. It is not forbidden. Maybe it's upsetting if a member of one's own team reveals the opening preparations to the opponent. Even then, it's not Niemann's fault.
arzi arzi 9/21/2022 11:44
hansj, could you explain to me why this guy writes like this. He is just a average level GM from Norway. From the same country as Carlsen.

"It is completely unacceptable to lose on purpose. It's the most unsportsmanlike thing you can do in competitive sports, Grandmaster Jon Ludvig Hammer said on Norwegian TV2's broadcast"

What exactly is he talking about? Does he blame Carlsen for something? For what reason?

By the way, did Niemann kick Carlsen´s leg after the game?
mc1483 mc1483 9/21/2022 11:44
Maybe, just maybe, the problem is not "Niemann is cheating, getting computer help etc". Maybe the problem is "Niemann knew in advance the opening Carlsen would have played against him", that's an entirely different matter. Nobody thought about that?
arzi arzi 9/21/2022 11:15
hansj has not said anything rational.
Nodded his head.
Rolled his eyes.
Grimaced his faces.
All these moves are just reflexes of body. Nothing intelligent there, no language to understand.

Do you mean by those mantras of yours that Carlsen is even more lost? He has aborted one tournament and resigned a game against Niemann just for no reasons? Thanks for explaining the real reason.

By the way, Carlsen can´t accuse Niemann of unsporting behaviour because he does it himself.
hansj hansj 9/21/2022 10:17
Carlsen has not accused Niemann of cheating.
He has not accused Niemann of being rude.
He has not accused Niemann of unsporting behaviour.
He has not accused Niemann of kicking his leg during a game.
He has not, to my knowledge, accused Niemann of anything.
arzi arzi 9/21/2022 08:55
I still have to remind some people what the point is. Did Niemann cheat in his game against Carlsen? Did NOT cheat. There is such overwhelming evidence in favor of it that any suggestion of measures to be taken against Niemann to prove his innocence is ludicrous. Carlsen has not given a single testimony in favor of the scam, NOT ONE. The handing over of the second game wasn't about cheating either. It was a childish reaction from the world champion. Maybe it's good that Carlsen gives up his title. It was sad to see when Fischer lost his marbles and it is sad to see the same happens to Carlsen.

I'm sorry for what I said, Magnus, but I've been a fan of your games for years until now. You have acted rationally over the years, but now you are completely lost. Sorry.
arzi arzi 9/21/2022 08:24
Elrond:"
- some 30 round tournaments which are not broadcasted
- solving 200 chess situations
- anything else suited"

What a nonsense! Prove us that you are not alien or elf, Elrond.

- 24 hours of uninterrupted masturbation
- 1000 push-ups per minute
- self-deprecating banter as live to your parents and friends
Ellrond Ellrond 9/21/2022 08:20
why did ken then didn't find the pattern (some phases in some games with some engine) which Punin found? @Odisngrandson.
arzi arzi 9/21/2022 08:13
Elrond:"For 10-20 moves he can play if needed at 3400 level, otherwise 2500 level. "

Do you mean that computer plays at levels between 2500-3400? Or human plays at 2500 when ever it is needed by him/her? Are you telling me that 2500 player understand the moves made by computer 3400 level? Player 2500 or lower can easily adapt him/herself at the same level as computer 3400 level or stronger? Where do we need a computer then?
Ellrond Ellrond 9/21/2022 08:12
the situation could be resolved easily. Just organize some reference rating tests for H.N., for example
- some 30 round tournaments which are not broadcasted
- solving 200 chess situations
- anything else suited
then we would know his undisputed real ELO and he could prove.
Odinsgrandson Odinsgrandson 9/21/2022 08:08
Regen's analysis can catch someone who cheats for only one key move in each game- over the course of several games.

But even if Hans is cheating, Magnus' response has been childish. It is wrong to pull out of a top level round robin tournament without a very good reason- and suspecting cheating without evidence is not a good reason.
Ellrond Ellrond 9/21/2022 08:03
The most probable scenario is 2). Clever 2500 player who adopted to well known computer help analysis technique.
See the remarkable FM Andrii Punin video:
H.N. turns on the turbo only in some critical phases in some games for 10-20 moves, so that the overall statistics remain "normal". Ken Regan too high level analysis to find such detailed procedures.
For 10-20 moves he can play if needed at 3400 level, otherwise 2500 level. See his performance before / after delay in sinquefield (3400 before / 2500 after). And performance with/without broadcast. And his critical games to catch the GM norm. And and and...
arzi arzi 9/21/2022 07:46
"In the knockout, the players will be paired according to their placings in the preliminaries. following the format 1v8, 2v7, 3v6, 4v5. A statistical analysis shared by Chess by the Numbers shows that Niemann is Carlsen’s most likely opponent in the quarterfinals — not by a wide margin, though."

https://en.chessbase.com/post/generation-cup-2022-d3

Will Carlsen resign once again if he and Niemann play in the quarterfinals? How many half moves the game lasts? One, two? Would that end Carlsen's career as a chess player for good? Is it possible that someone smarter could advise Magnus on this before it's too late? Daddy?
arzi arzi 9/21/2022 06:29
Niemann's performance in the Julius Baer Generation Cup 2022 is now 2730. Is he cheating, Magnus? You know best. Tell us some mysterious words of wisdom, please. You don't have to have proof, we believe you, unquestioningly and blindly. Thanks for sharing.
Leavenfish Leavenfish 9/21/2022 02:38
@physica - nothing is 'fail-safe'...that's a fools endeavor. But top level round robins are going to largely be free of 'cheating' with the technology available. That's all anyone can reasonably ask.
saturn23 saturn23 9/21/2022 02:26
So many people are missing the point of the article. Assuming that his statistical analysis is sound and correct, Ken Regan showed that there's no good evidence to conclude that Niemann cheated in the last two years. This is not proof that Niemann did not cheat, it is simply evidence against any kind of clear and heavy cheating. There are at least a few possible scenarios:

1. Niemann is roughly a 2700 and he never cheated over the board.
2. Niemann is roughly a 2650-2700 and he cheated in a smart way (perhaps in a few key games).
3. Niemann is nowhere near a 2700 and he cheated in many (most) games, in such a way that it is not detectable based on any kind of statistical analysis (and this might not be a hard thing to do).

I believe that the 3rd scenario is unlikely. Given Niemann's history of cheating in online chess and his extraordinary progress, it is not clear to me which of the first two scenarios are more likely.
physica physica 9/21/2022 02:07
I don't think this system is fail-safe. It's like asking 'who is the most accurate chess player according to the engines?' again. Humans are random error generators in chess regardless of level.

Today, there are something like a handful of sound moves in most (top player) positions which maintain the edge or equality before the fireworks start. So mixing up engine's 1-6th best moves in noncritical positions can surely go unnoticed in addition to non-cheating opponent's moves. Obviously, you can't perform much better than your expected score, so you need to lose some games, but stay just above >50%. So is it ok to say in post-interview: "Yuup, I looked this (in situ engine crunched bs) line at home and converted from it..."? This distantly resonates with Kramnik's critic on low quality of play in the last Candidates. Do you trust more in your own intuition or what engines at home told you to play?

Unfortunately, human chess is ruined for good now that we can excuse ourselves: "I saw engines suggesting this...". Personally, I don't understand these affluent online tournaments. Temptation to cheat will increase over time. There will be people who don't aim to become WC nor top players, just making their income from these kinds of tourneys.
Frits Fritschy Frits Fritschy 9/20/2022 11:44
The least Carlsen could have done or do is say he will file an official complaint with the FIDE Ethical and Disciplinary Commission (and do so) about behaviour contrary to the rules by Niemann. Then at least he commits himself to action, and without consequences based on openly accusing.
If he doesn't, he is vulnerable to consequences here again explained, this time by arzi.
JoniCee65 JoniCee65 9/20/2022 11:08
Loving how complete amateurs feel in a position to advance strawman arguments to criticise an expert's findings and feed their fanboy crushes.
The statistical analysis speaks for itself.
tauno tauno 9/20/2022 10:38
@soulblazer, I was wondering if you would support any other top player who would do the same thing - or something similar - that Magnus did. (Personally I cannot even imagine who would.)

It's a normal human reaction when an accident happens that we think it just had to happen. In some weird way, it would feel a bit unnatural if it had not happened. After all, it was a necessity, right? This is how we adapt to reality.

But if Magnus had not done what he did, I'm pretty sure few of us would raise our voices and say that he should do what he did. Think about it.

On some level I can understand what Magnus did, but I don't think it was right and I don't support it. Actually, I think it was quite immature, almost childish. There would have been so many more constructive and elegant ways for Magnus to handle this issue.

But before we judge a person too harshly, we must consider, not only the person's age, but also his level of maturity.

"Our prime purpose in this life is to help others. And if you can't help them, at least don't hurt them."
- Dalai Lama
John Maccormack John Maccormack 9/20/2022 10:25
Thank you tturgut for your personal perspective on Niemann. This sounds like a voice of authority. If Niemann keeps beating the old guard, with or without Carlsen in the equation, he will silence all his critics. The whole fuss may be less about possible cheating that Niemann's somewhat odd and overbearing personality.
Maatalkko Maatalkko 9/20/2022 10:19
A lot of people in the comments are not understanding the point of Regan's analysis when they say "but but but what about if he cheated by receiving one move or what about if he cheated in one game?" Of course statistics can't prove that no cheating occurred but they do show that Niemann plays at a very consistent 2700 level at all formats in all tournaments and even playing informal blitz in a cafe versus MVL. It's not possible to prove anyone has never cheated - for all we know Magnus has cheated. That's why accusations need evidence. Regan put in the rigorous analytical work and found no evidence.
Aighearach Aighearach 9/20/2022 09:51
Resigning after 1 move isn't merely unsportsmanlike.

It is game-fixing. And the purpose of the game-fixing is to influence who is invited to future tournaments, so that he can avoid a player that he finds psychologically difficult to face.

That's simply cheating.
Marseille07 Marseille07 9/20/2022 09:48
@soulblazer Of course Magnus has to justify his move. Resigning on purpose is unsportsmanlike conduct, and he can't say he suspected Niemann of cheating on move 2 obviously, so there's just no excuse this time.
rgorn rgorn 9/20/2022 09:33
@Frederic: You said: "I need to keep this vague and not provide any more details. I do not want to give cheaters detailed instructions on how to better avoid detection in the future."

Magnus could say the same. If the argument is valid for you, it is even more valid for him. I take it for granted that he is making much more money from chess than you and so he has a lot more at stake than you.

Therefore, he didn't even argue, he just quoted Mourinho: "If I speak I am in big trouble."

But now that you have spoken, please go ahead and explain the details that Magnus won't give. We need an expert here. 🤪
tturgut tturgut 9/20/2022 09:15
I know Hans since he was 12.(South Africa World Youth. he played together with my one year younger son Aydin in 4-5 WYCC's- last one in India in 2019.
he is arrogant, but he improved a lot. He is close to a genius in my opinion. Has some psychological problems, but he made significant improvement in the last 3-4 years. Not a perfect person, but he is improving, and he will be atop 10 player.

1) Great analysis, supporting that fact that GM Niemann is not cheating!
2) WC GM Carlsen first decided that he won't be defending his WC title, now acting so irrationally against this kid? Why all this drama? It looks like Carlsen may be experiencing a midlife crisis in his 30s? maybe a girl/woman problem?
3) This has been a serious problem for GM Niemann. he clearly has been a victim , and should have legal bases to sue Carlsen.
4) Any player (World champion or not) should not have the right to leave tournaments, quit after 1 move etc.... This should be in contracts and should be punished.
5) Having said these, I also believe that Carlsen is the best player right now. he was unlucky that there are no players strong enough to challenge him. hope that he recovers from this crisis and starts playing some chess.
6) What a drama.. mostly caused by carlsen... GM Niemann is real, and is there to stay. Time will show this. He is a top 10 player. if he can beat some of is problems, he may even compete for world championship.
soulblazer soulblazer 9/20/2022 08:26
Magnus does not have to justify his move. About cheating, you just need 1 good idea coming from outside to get the better of any opponent at the top of chess. No analysis method could really detect this. Plus, don't you think a cheater at that level would not be expert in covering its track? In any case, I support Magnus decision and I'm not accusing anyone from cheating. Also, let's not feed the trolls.
IntensityInsanity IntensityInsanity 9/20/2022 07:17
IMO: Cudos to chessbase for a good objective reporting on this. I read two other chess news sites but based on their reporting it is very obvious that magnus is a shareholder in those sites because the reports have been anything but objective (ie, K Regan analysis not even mentioned). FIDE leadership response by GM Sutovsky was very unprofessional and biased. Clearly Chessbase is the only major online chess publication taking a balanced approach to this.
arzi arzi 9/20/2022 07:09
Some Fide rules:
9 Conduct of the Players
9.1 Once a player has formally accepted an invitation, he must play except in exceptional
circumstances (force majeure), such as illness or incapacity. Acceptance of another
invitation is not considered to be a valid reason for not participating or for withdrawing.

1.9 General misbehaviour
b) Withdrawal from tournaments: Players withdrawing from a tournament without valid
reason or without informing the tournament arbiter;

https://handbook.fide.com/files/handbook/EthicsAndDisciplinaryCode2022.pdf
Marseille07 Marseille07 9/20/2022 07:05
@Jack Nayer It's not nonsense. Dr. Regan's statement was "there is no reason whatsoever to suspect him of cheating," not Niemann didn't cheat. In your example, if you cheat 2 out of 100, there's also no reason to suspect you of cheating. But you also won't be competing with Super GMs, only cheating 2 out of 100, so this is a good enough safeguard if you will.
Green22 Green22 9/20/2022 07:04
@soulblazer yes so much Hate because now he is tainting his legacy! he's making a mockery of this phantom accusations towards Hans. If Hans was cheating (which he is not) don't ya think he would have a better score at this point? lol! Magnus needs to come forward as a professional as the WC and come out and tall the world why he is doing what he is doing.

You just gift an opponent 3 pts in this tourney why all the other players fighting it out?? Magnus is poor sport right now and acting like big baby wahhhh I won't play Hans but i'll stomp my feet and resign a game, but tell no one why. Gimme a break! 90% of the Chess world is feeling and saying the same things.