Is Hans Niemann cheating? - World renowned expert Ken Regan analyzes

In view of the tremendous commotion the unraveling of the story has caused, speculation has continued to be rife. There are pitched battles in the recesses of social media with some wondering why an event such as Meltwater allows a 'cheater' to play and they do not blame Magnus for his attitudes, while others bemoan the lack of data to properly render judgement. 

Amateurs have sprung up with their own detailed verdicts, providing spreadsheets comparing the broadcast and offline event results of Hans Niemann, others doing Elo performance charts with lovely bell curves, and others running his games from events such as the Capablanca Memorial (that he won) through engines to prove his guilt beyond any doubt. As one grandmaster confided in private, "The Capablanca video is pretty damning".

The problem with all of these is the lack of scientific rigor and depth to properly make such analyses valid points of reference. On the flip side, with a lack of better sources of information, anything goes. 

Who is Professor Kenneth Regan?

Ken Regan was better known as a chess player before becoming an academic, reaching the IM title as a junior, and qualifying for the US Championship even. He explains he never planned on a career in chess, despite his talent and enjoyment for the game, and eventually obtained a PhD in Mathematics from Oxford University in complexity theory. 

In 2006, at the cusp of the infamous 'toiletgate' controversy during the World Championship, he was brought in to provide expert analysis of the cheating accusations leveraged by the Topalov Team against Vladimir Kramnik. The claims were that Kramnik's moves in Game Two, the point of bitter contention, achieved an unreal 80%+ match with the best engine of the time, Rybka. Dr. Regan confirmed this claim, but debunked the cheating accusations. He demonstrated that many of the moves that matched the engine were in fact quite forced, and if you eliminated these forced moves from the analysis, then the overall number of moves that matched engine choices was quite normal. 

Dr. Ken Regan on the cover of Chess Life magazine for his pioneering work in cheating detection in chess

Showing this greater depth and rigor in his analysis immediately propelled him into the limelight as the new top expert, even if many did not understand or accept the depth of his research. He has since been used by FIDE to supplement expert analysis in cases such as Sebastien Feller, Igor Rausis, and general oversight of major events.

Regan has developed statistical methods to prove cheating in a series of scientific papers which are accessible on his homepage.

The Interview

Dr. Ken Regan was kind enough to agree to an interview in which he not only explains how his methods work, but also the detailed work he did on the Carlsen-Niemann Affair and what his conclusions were. 

This is a slightly abridged version of the full interview (see below), removing notably some of the lengthier technical explanations.

Are broadcast games more suspicious?

In the interview I queried specific points being brought up online and elsewhere such as a possible difference between games that were broadcast and games that were not.

"What I'm saying, as justifying my not needing to take the time to individually look into tournaments to see which were broadcast and which were not, is that if there is any bias in my data, then it's towards broadcast games (i.e more of it is analyzed due to availability) and yet I show something entirely normal."

ChessBase 16 - Starter Package Edition 2021

Your key to fresh ideas, precise analyses and targeted training!

On the quality of his system

"My system deliberately does not use specific chess knowledge, it's all based on the quantitative data from the computer's evaluations of the moves transplanted into a utility function, which is then fed into a predictive analytic model that really works in a broadly similar way to how economists base predictive analytical models on utility functions."

Screenshot of Dr. Regan's computer and the breakdown of the results

He also mentions that the system is also capable of highlighting players who are better than others at inducing errors from their opponents. Notables at this are Mikhail Tal and Alexey Shirov.

On the result in the Capablanca Memorial

You will find this dealt with in great detail, including a breakdown of Niemann's error rate, the error rates of his opponents, and how well his opponents played against him.

It turns out that not only did Niemann perform almost exactly as expected of him in terms of move accuracy, but his opponents are the ones who underperformed against him. The 0.152 on the right in blue is the error rate of his rivals.

Likelihood of a miracle in chess results

"Littlewood's Law says that if you observe a million happenings one of them will have a million-to-one prior probability. which is the common definition of a miracle. So if you see a million things a day, then a miracle happens every day. And in online chess a million games per day is the norm."

The verdict

Dr. Regan analyzed all of Hans Niemann's games over the last two years, including online games, such as played on Chess.com and their events, and his conclusion is there is no reason whatsoever to suspect him of cheating. The wide range of results in a bell curve, with some good and some bad, is actually a sign of a healthy distribution of results. Many of the so-called points of suspicion are in fact quite normal and suspicion is really the result of faulty analysis by zealous amateurs. Even online his play has been quite devoid of anything unusual.

Full results of his data can be found at this link

Full interview

Here are a few mainstream news reports:

• Forbes: World Chess Champion Magnus Carlsen Resigns From Match After Just One Move Against Player At Center Of ‘Cheating’ Scandal.
• The Guardian: Carlsen resigns from rematch with Niemann after opening move
• Wall Street Journal: Magnus Carlsen’s Walk-Off Stunner in Rematch With U.S. Teen Fuels More Chess Chaos
• ABC News: Magnus Carlsen resigns from rematch after opening move
• Der Spiegel: Schach-Weltmeister Carlsen sorgt für nächsten Eklat