ICGA/Rybka controversy: An interview with David Levy (2)

2/10/2012 – On Monday we published part one of a lengthy interview we had conducted, via sequential email exchanges, with the President of the International Computer Games Association David Levy. The ICGA had found star programmer Vasik Rajlich guilty of "plagiarism" in his early program versions and banned him for life. Here now is part two of the interview.

ChessBase 14 Download ChessBase 14 Download

Everyone uses ChessBase, from the World Champion to the amateur next door. Start your personal success story with ChessBase 14 and enjoy your chess even more!


Along with the ChessBase 14 program you can access the Live Database of 8 million games, and receive three months of free ChesssBase Account Premium membership and all of our online apps! Have a look today!

More...

Interview with David Levy on the ICGA/Rybka scandal – Part two

Here is part one of this interview

ChessBase: Critics of the ICGA say that at least some of the members of the panel that led the investigation harboured deep personal animosity towards Vasik Rajlich and had been attacking him for years. This can be easily proved by citing many thousands of forum postings before, during and after the investigation process. Was it wise of the ICGA to rely on such members and in fact elect them to lead the investigation and draft the final report? In civil or criminal court this would have lead to an immediate mistrial.

David Levy: The reason that Rajlich had been under attack for years is that it had been known, by some people, for years, that he had used code derived from others. The panel's Secretariat were tasked with writing the report because they were thoroughly familiar with all of the evidence revealed and discussed during the investigation. The ICGA office-bearers who considered the report could themselves judge whether or not the report represented a fair and balanced summary of the investigation and the evidence, and none of us raised any questions or objections to cast doubt on the report as an accurate summary of the evidence and the conclusions of the panel. Furthermore, it is quite typical for courts to rely on "persons with animus" regarding issues of fact (particularly ones that can be checked and verified).

You say that the reason Vasik Rajlich had been under attack for years is that "it had been known, by some people," that he was guilty. Many them have stood to profit from his downfall. Why was it necessary to nominate exactly those people to the jury panel that would pass judgement over him. It meant that the entire investigation and sentencing would in no way represent due process, but become an ad hoc procedure put together by the ICGA, one which one must assume could come to no other conclusion than the one it reached. Are you satisfied that justice was done?

Yes, I am satisfied that justice was done. First I should point out that the panel was not a jury in the usual sense of the word, because its votes were always going to be regarded by the ICGA as advisory rather than a hard-and-fast vote that would determine guilt or innocence. For instance, had the vote been, 9-7 in either direction, rather than 16-0 for "guilty" which it was, the ICGA office bearers would have regarded it as a strong indication that the opinion of panel members was nowhere near as clear cut as it turned out to be, and that would doubtless have affected their thinking.

We did not nominate people to the panel. We asked for people who have an interest and relevant expertise to volunteer to join the panel. The only people who were nominated were the members of the Secretariat. At most, three active competitors of Rajlich voted. None of those three panel members took an active role in the discussions so they did not influence the other members. Even if we had excluded the votes of these three, who make up fewer than one-fifth of the voting members, all the remaining voters agreed that Rajlich was guilty of violating the ICGA's Tournament Rule 2.

At most, three active competitors of Rajlich voted? This is definitely not what is being said in the computer chess forums. Are you sure of the number?

ChessBase has always had access to the ICGA wiki since one of you (Frederic Friedel) were on the panel. If you want to disagree and say there are more than three voting active competitors, why don't you simply name them instead of mentioning the often inaccurate things posted on the forums?

Is this a trap? In January this year programmer Ed Schröder, former ICGA world champion, posted a few names of voting members. He was reprimanded for revealing the information and immediately removed from the panel.

Not a trap. You would name them in a question to me and if there were more than three then I would correct my answer. What Ed did was to name them publicly, which is against the terms of joining the wiki. As I look at the people who voted, I see only three active competitors. Yes, there are other chess programmers on the panel, but of the ones that voted I do not see any selling a competing product.

You justify the procedural decision by saying that "the world of elite chess programmers is a very small one, and these are the people who are best equipped to judge the issues involved in the investigation". Really? The issue was one of copying code, which means there are literally thousands of experts available to make a judgement – in the world of general programming. Would it not have been simple to avoid the impression of cronyism or lynch justice by recusing anyone from the process who stood to gain from a guilty verdict or who otherwise had a vested interest, or harboured personal animosity towards the accused, and instead tapping in on the vast community of general computer science experts?

There are not "literally thousands of experts" on chess programming, and it is obviously necessary for those whose opinions are being considered by the ICGA to have such expertise. In addition to computer chess experts with experience of developing chess programs the panel also included active members such as Wylie Garvin, who is a games programmer at Ubisoft, and Mark Watkins, a research fellow in the School of Mathematics and Statistics at the University of Sydney, neither of whom is from the world of competitive computer chess programmers.

We do not follow the logic: It is necessary for the experts to have chess programming expertise – but the ICGA used at least two who didn't have this expertise?!

Perhaps I didn't make myself clear. There is a limited number of people with a high level of computer chess expertise, and we wanted to avail ourselves of as many of that group as we could. In addition the panel included some people (such as Mark Watkins) whose expertise is relevant even though they have not developed a competitive chess program.

Since the report was published we have seen no evidence to demonstrate that the panel and the report came to the wrong conclusions, or explaining the huge overlap of Rybka and Fruit in a manner that casts doubt on the guilty verdict. Additionally please remember that Rajlich was repeatedly invited to join the panel, he could also have supplied his own experts to join the panel at any time, he was given ample opportunity by way of invitation from me to defend himself against the allegations, and anyone with relevant expertise who believed in Rajlich's innocence was perfectly free to join the panel of their own volition, present whatever evidence and arguments they wished, and attempt to sway the other panel members towards their own point of view. None of Rajlich's supporters felt inclined to do this, which is hardly the fault of the ICGA.

Currently the world of chess engine programming is in a deep crisis: instead of a dozen or so highly competent teams producing new ideas and techniques, there are an increasing number of amateurs who simply reverse engineer top programs, do some optimisation and then either distribute these programs free of charge, or even start to sell them. As a result the commercial engine market has been completely destroyed and the original programmers are out of work. What has the ICGA done about this very acute problem?

First may I say that your question is misleading, suggesting as it does that there are not "a dozen or so highly competent teams producing new ideas and techniques". What about the authors of programs such as Chiron, Fritz, Hiarcs, Junior, Komodo, Pandix, Shredder, Sjeng, Spark, Spike and Stockfish, to name only eleven? Amongst these are four programs sold by ChessBase, so presumably you consider them to be worthy products: Fritz, Hiarcs, Junior and Shredder. As to the core of your question, the commercial engine market, the ICGA is a non-commercial organisation and as such it does not concern itself with commercial matters.

Has it investigated any of the clones?

ICGA investigations will be conducted as and when it is necessary to verify that a program entered for one of our events is the original work of the programmer(s) whose names are submitted with the program entry details.

Was it right for the ICGA to invest a gigantic amount of research and expert capability into checking the fledgling work of the most successful engine programmer, instead of addressing the problem of the illegal clones which are, in the most part, derived from his work?

In general the "illegal clones" to which your question refers have not attempted to enter any ICGA events and so there has been no reason for us to investigate them. The Free Software Foundation would be a much more appropriate organisation to conduct investigations into such programs if there is a prima facie case that a copyright licence has been breached. Prior to the Rybka case the only examples I can recall of an entry into an ICGA event being submitted for an "illegal clone" or anything that could come under a description that is in any way similar, are those I mentioned in my first published article on the subject – "Attack of the Clones".

As to the enormous amount of work and expert capability that went into the Rybka investigation, surely this is something for which the ICGA should be praised rather than criticized. We had to make certain before coming to any conclusions that we had done a thorough job, so that we could be equally certain that the evidence on which we were basing our conclusions and verdict was sufficient in volume and sufficiently accurate in quality. To find someone guilty of such serious infringements of ICGA Tournament Rule 2, with all of the consequences of such a verdict, requires an extremely high level of diligence.

Just like the engine market, the ICGA World Computer Chess Championship would seem to be in a crisis: the top seed and winner of the most recent ICGA event is in place 53 of the CCRL Computer Rating list (January 29, 2012). On equal hardware it is 175 points behind the leader, Houdini, which experts say is a derivative of Ippolit, in turn a derivative of Rybka (which is in third place). How can such "world championships" be meaningful if they do not come close to producing the strongest programs as winners? How can they be distinguished from club championships, where members of a special group keep exchanging titles?

Any program that qualifies under the ICGA rules is welcome to apply to participate. Just as the IOC has certain qualification rules for its events, so does the ICGA.

Why do none of the top seven programs in the world participate in the ICGA world championships?

Most of the top seven original programs do participate. Apart from Rybka none of your "top seven" has been refused entry. If any of them have an author willing to confirm their identity and if they have the necessary permission(s) from the author(s) of any original programs from which they have copied or derived code (either directly or via other programs), then they would be welcome to apply for entry. If they have copied or derived code from another program that likewise contains copied or derived code, they will of course need to obtain permission(s) from all the relevant program authors.

What you advice a new programmer who is interested in building a chess engine: write everything from scratch? Not duplicate or even look at any parts of open source programs? Publish his program as open source if he does? Or simply avoid ICGA events, as the clones are currently doing?

I would advise any new chess programmer to take note of ICGA Tournament Rule 2 and stick to it, if they wish to participate in any ICGA events. And if any ICGA members wish to propose changes to our rules they are free to do so, and we will consider their suggestions. It might be that in the future our rules and/or tournament formats are changed, for example we might decide to add an "Open" event in which any program may take part provided that the team entering it has the necessary permission from anyone from whom code has been copied or derived. But we do not yet have such an event in our calendar, and even if we had done so in recent years Rybka's participation would still have been against the rules because Vasik Rajlich did not ask for or obtain permission from Fabien Letouzey (or Bob Hyatt).

Tell us about the Loop program that was developed under the supervision of the ICGA and earned its author, Fritz Reul, a PhD from Prof. Jaap van den Herik, chief tournament director of the ICGA and editor of its journal. There were suspicions that Loop contained plagiarised code to a much greater extent than that of which Vasik Rajlich has been accused.

Although Reul's thesis was supervised by Professor van den Herik the Loop program was not developed under the supervision of the ICGA. We have received a formal complaint from Fabien Letouzey about Loop and will be making a statement shortly to indicate how we are going to deal with it.

Tell us a little more about the ICGA: in what jurisdiction is it registered, where does it make its tax returns and who is legally and financially responsible for the organisation?

I do not see what connection this has with the Rybka scandal but I am happy to answer the question anyway. The ICGA is a non-profit organisation and as such we have no obligation to register anywhere for tax or legal purposes. We do not make tax returns, and on all our invoices we state that no VAT is payable [For the benefit of readers outside Europe, VAT is a type of sales tax imposed on many goods and services in many European countries.]

As I have stated earlier in this interview, the question about who is legally and financially responsible for the organisation comes under Article III Section 1 of the ICGA Constitution, which is always published in the ICGA Journal prior to the tri-ennial meeting at which ICGA office bearers are elected. Section 1 states that the Executive Committee of the ICGA is charged with the administrative affairs of the association.

Is there anything you would like to add to what you have already said in this interview? A lot of readers of our news page are clamouring to ask questions. Would you be willing to answer them?

I feel that I have answered enough questions in this interview and do not plan to take questions from your individual readers, but I would like to add to what I have already said in this interview.

The first point that I would like to emphasize very firmly relates to what I see as irrelevant comments that have been levelled at the whole ICGA investigation process by supporters of Vasik Rajlich. It is clear that the "Rybka scandal" has aroused strong emotions on both sides of the verdict – what is unfortunate is that the strength of emotion has, in the case of many Rajlich supporters, led to comments and criticisms that are partisan to the point of lacking objectivity and/or irrelevant to the core issue. These Rajlich supporters either fail to realise and accept, or knowingly side-step, the most crucial question in all of this – Did Rajlich or did he not break the ICGA's Tournament Rule 2 when entering multiple World Computer Chess Championships?

The pro-Rajlich lobby has been strong in its criticisms of various aspects of the investigative process, starting with the composition of the investigation panel and of the panel's secretariat, continuing through criticism of Rule 2 itself, and then complaining about the force of the sanctions imposed on Rajlich following the guilty findings of the ICGA. An analogy here would be the supporters of someone being tried in a court of law complaining about the witnesses, about the court officials, the composition of the jury, the law under which the defendant had been brought to trial, and the severity of the sentence passed on a guilty defendant. But what really matters most is the question – Did the defendant commit the offence(s) that led to the investigation or trial?

So much of the pro-Rajlich rhetoric on the forums, and some of the questions in this interview, seem to be aimed at nothing more than discrediting people. Neither the forum postings nor the questions in this interview have seriously questioned the accuracy of the evidence examined by the panel. In fact I have yet to see anything that seriously questions the accuracy of that evidence. Vasik Rajlich did indeed use code derived from Fruit and Crafty, he did indeed fail to declare this with his entry applications for those ICGA tournaments, and he did indeed fail to seek and be granted permission from the authors of those programs. And with the accuracy of the evidence beyond serious question it is hardly surprising that he was found guilty by the ICGA of breaking its Tournament Rule 2. That is the crux of the matter.

Do we think the process used during this investigation was perfect? Of course not. Could it be improved? It can and will be. But it was also not flawed in any way. The panel included several excellent motivated people, with very high levels of technical and computer chess expertise. And the investigation was carried out in a straightforward and open manner, without employing any techniques that experienced programmers could reasonably describe as being biased or incorrect. While the final outcome was painful for everyone, if the integrity of the tournaments that choose to use Rule 2 is not protected, those tournaments would become pointless and no-one would be willing to compete. One important attraction of competition is to compete with your own skills, against others using their own skills. And from the sporting perspective it is not as intellectually honest and stimulating to take bits and pieces from the work of others, as opposed to developing your skills yourself. Then, winning actually means something.

We will, for future panels, limit their size to some extent, and ensure that those who join and remain in the panel actually participate in the discussion rather than stay as spectators to the process, since that seems to have been an issue raised about the Rybka panel. However, the 16 people with the backgrounds of those who participated in the voting within the Rybka panel formed a remarkably expert group with more than enough expertise to fairly and thoroughly investigate potential cases of Rule 2 violation. And in fact, for the future, I do not believe that we would want a working group of more than 30 people, any more than a judge would want to put more than 30 jurors in a room together hoping for a very large majority or a unanimous decision. Some might argue that even the 16 or more who we had actively participating in the Rybka panel is probably pushing the limit a bit, although in this case there were no disagreements at all. The very fact that the 16 who voted generally had a much higher level of understanding of computer chess programming than almost all of those Rajlich supporters whose voices have been heard in the forums, adds significant weight to the ICGA's confidence in the investigation process and in the panel's report. And the fact that those 16 people voted unanimously for guilty verdicts speaks for itself.

Many of the Rajlich supporters posting on the forums have supported the argument about "a new paradigm created by open-source programs", without realising that that concept is just nonsense. Most people still know that copying the work of others and claiming it to be original is wrong, just as it has always been.

Please use this feedback form if you wish to write or comment on this interview.


Previous articles on the ICGA/Rybka scandal

ICGA/Rybka controversy: An interview with David Levy (2)
10.02.2012 – On Monday we published part one of a lengthy interview we had conducted, via sequential email exchanges, with the President of the International Computer Games Association David Levy. The ICGA had found star programmer Vasik Rajlich guilty of "plagiarism" in his early program versions and banned him for life. Here now is part two of the interview.
ICGA/Rybka controversy: An interview with David Levy (1)
06.02.2012 – He is a star chess programmer with an unbroken record of supremacy in computer chess during the past five years. In June 2011 the International Computer Games Association found Vasik Rajlich guilty of "plagiarism" in early program versions and banned him for life. We published a vigorous defence of Rajlich and a rebuttal by ICGA President David Levy, who promised us an in depth interview.
Feedback on the ICGA/Rybka disqualification scandal
13.01.2012 – It's a long read, but many of the letters we received in reaction to the defence of the Rybka program by Dr Søren Riis are quite passionate and well thought out in their content. We start with the summary of a long rebuttal of the Riis paper that was sent to us by the ICGA and circulated on the Internet – with links to the full version and ancillary documents. Take a deep breath.
A Gross Miscarriage of Justice in Computer Chess (part four)
05.01.2012 – As proof that Vas Rajlich had copied program code the ICGA presented pages and pages of Fruit and Rybka code side by side. But, according to Dr Søren Riis, what was labeled as Rybka code was actually fabricated to look like Fruit. He criticizes the ICGA's process failures and ruminates on the reasons behind the unprecedented vendetta launched against the star chess programmer.
A Gross Miscarriage of Justice in Computer Chess (part three)
04.01.2012 – A core accusation against Vas Rajlich is that Rybka and Fruit have very similar positional evaluations, and the use of floating point numbers in Rybka’s time management code had to be copied from Fruit. Søren Riis enumerates the ten substantive evaluation differences and shows how the second accusation boils down to a single misplaced keystroke with zero impact on Rybka's play.
A Gross Miscarriage of Justice in Computer Chess (part two)
03.01.2012 – In this part Dr Søren Riis of Queen Mary University in London shows how most programs (legally) profited from Fruit, and subsequently much more so from the (illegally) reverse engineered Rybka. Yet it is Vasik Rajlich who was investigated, found guilty of plagiarism, banned for life, stripped of his titles, and vilified in the international press – for a five-year-old alleged tournament rule violation. Ironic.
A Gross Miscarriage of Justice in Computer Chess (part one)
02.01.2012 – "Biggest Sporting Scandal since Ben Johnson" and "Czech Mate, Mr. Cheat" – these were headlines in newspapers around the world six months ago. The International Computer Games Association had disqualified star programmer Vasik Rajlich for plagiarism, retroactively stripped him of all titles, and banned him for life. Søren Riis, a computer scientist from London, has investigated the scandal.

Topics: Rybka scandal
Discussion and Feedback Join the public discussion or submit your feedback to the editors


Discuss

Rules for reader comments

 
 

Not registered yet? Register