ICGA/Rybka controversy: An interview with David Levy (2)

Interview with David Levy on the ICGA/Rybka scandal – Part two

Here is part one of this interview

David Levy: The reason that Rajlich had been under attack for years is that it had been known, by some people, for years, that he had used code derived from others. The panel's Secretariat were tasked with writing the report because they were thoroughly familiar with all of the evidence revealed and discussed during the investigation. The ICGA office-bearers who considered the report could themselves judge whether or not the report represented a fair and balanced summary of the investigation and the evidence, and none of us raised any questions or objections to cast doubt on the report as an accurate summary of the evidence and the conclusions of the panel. Furthermore, it is quite typical for courts to rely on "persons with animus" regarding issues of fact (particularly ones that can be checked and verified).

Yes, I am satisfied that justice was done. First I should point out that the panel was not a jury in the usual sense of the word, because its votes were always going to be regarded by the ICGA as advisory rather than a hard-and-fast vote that would determine guilt or innocence. For instance, had the vote been, 9-7 in either direction, rather than 16-0 for "guilty" which it was, the ICGA office bearers would have regarded it as a strong indication that the opinion of panel members was nowhere near as clear cut as it turned out to be, and that would doubtless have affected their thinking.

We did not nominate people to the panel. We asked for people who have an interest and relevant expertise to volunteer to join the panel. The only people who were nominated were the members of the Secretariat. At most, three active competitors of Rajlich voted. None of those three panel members took an active role in the discussions so they did not influence the other members. Even if we had excluded the votes of these three, who make up fewer than one-fifth of the voting members, all the remaining voters agreed that Rajlich was guilty of violating the ICGA's Tournament Rule 2.

ChessBase has always had access to the ICGA wiki since one of you (Frederic Friedel) were on the panel. If you want to disagree and say there are more than three voting active competitors, why don't you simply name them instead of mentioning the often inaccurate things posted on the forums?

Not a trap. You would name them in a question to me and if there were more than three then I would correct my answer. What Ed did was to name them publicly, which is against the terms of joining the wiki. As I look at the people who voted, I see only three active competitors. Yes, there are other chess programmers on the panel, but of the ones that voted I do not see any selling a competing product.

There are not "literally thousands of experts" on chess programming, and it is obviously necessary for those whose opinions are being considered by the ICGA to have such expertise. In addition to computer chess experts with experience of developing chess programs the panel also included active members such as Wylie Garvin, who is a games programmer at Ubisoft, and Mark Watkins, a research fellow in the School of Mathematics and Statistics at the University of Sydney, neither of whom is from the world of competitive computer chess programmers.

Perhaps I didn't make myself clear. There is a limited number of people with a high level of computer chess expertise, and we wanted to avail ourselves of as many of that group as we could. In addition the panel included some people (such as Mark Watkins) whose expertise is relevant even though they have not developed a competitive chess program.

Since the report was published we have seen no evidence to demonstrate that the panel and the report came to the wrong conclusions, or explaining the huge overlap of Rybka and Fruit in a manner that casts doubt on the guilty verdict. Additionally please remember that Rajlich was repeatedly invited to join the panel, he could also have supplied his own experts to join the panel at any time, he was given ample opportunity by way of invitation from me to defend himself against the allegations, and anyone with relevant expertise who believed in Rajlich's innocence was perfectly free to join the panel of their own volition, present whatever evidence and arguments they wished, and attempt to sway the other panel members towards their own point of view. None of Rajlich's supporters felt inclined to do this, which is hardly the fault of the ICGA.

First may I say that your question is misleading, suggesting as it does that there are not "a dozen or so highly competent teams producing new ideas and techniques". What about the authors of programs such as Chiron, Fritz, Hiarcs, Junior, Komodo, Pandix, Shredder, Sjeng, Spark, Spike and Stockfish, to name only eleven? Amongst these are four programs sold by ChessBase, so presumably you consider them to be worthy products: Fritz, Hiarcs, Junior and Shredder. As to the core of your question, the commercial engine market, the ICGA is a non-commercial organisation and as such it does not concern itself with commercial matters.

ICGA investigations will be conducted as and when it is necessary to verify that a program entered for one of our events is the original work of the programmer(s) whose names are submitted with the program entry details.

In general the "illegal clones" to which your question refers have not attempted to enter any ICGA events and so there has been no reason for us to investigate them. The Free Software Foundation would be a much more appropriate organisation to conduct investigations into such programs if there is a prima facie case that a copyright licence has been breached. Prior to the Rybka case the only examples I can recall of an entry into an ICGA event being submitted for an "illegal clone" or anything that could come under a description that is in any way similar, are those I mentioned in my first published article on the subject – "Attack of the Clones".

As to the enormous amount of work and expert capability that went into the Rybka investigation, surely this is something for which the ICGA should be praised rather than criticized. We had to make certain before coming to any conclusions that we had done a thorough job, so that we could be equally certain that the evidence on which we were basing our conclusions and verdict was sufficient in volume and sufficiently accurate in quality. To find someone guilty of such serious infringements of ICGA Tournament Rule 2, with all of the consequences of such a verdict, requires an extremely high level of diligence.

Any program that qualifies under the ICGA rules is welcome to apply to participate. Just as the IOC has certain qualification rules for its events, so does the ICGA.

Most of the top seven original programs do participate. Apart from Rybka none of your "top seven" has been refused entry. If any of them have an author willing to confirm their identity and if they have the necessary permission(s) from the author(s) of any original programs from which they have copied or derived code (either directly or via other programs), then they would be welcome to apply for entry. If they have copied or derived code from another program that likewise contains copied or derived code, they will of course need to obtain permission(s) from all the relevant program authors.

I would advise any new chess programmer to take note of ICGA Tournament Rule 2 and stick to it, if they wish to participate in any ICGA events. And if any ICGA members wish to propose changes to our rules they are free to do so, and we will consider their suggestions. It might be that in the future our rules and/or tournament formats are changed, for example we might decide to add an "Open" event in which any program may take part provided that the team entering it has the necessary permission from anyone from whom code has been copied or derived. But we do not yet have such an event in our calendar, and even if we had done so in recent years Rybka's participation would still have been against the rules because Vasik Rajlich did not ask for or obtain permission from Fabien Letouzey (or Bob Hyatt).

Although Reul's thesis was supervised by Professor van den Herik the Loop program was not developed under the supervision of the ICGA. We have received a formal complaint from Fabien Letouzey about Loop and will be making a statement shortly to indicate how we are going to deal with it.

I do not see what connection this has with the Rybka scandal but I am happy to answer the question anyway. The ICGA is a non-profit organisation and as such we have no obligation to register anywhere for tax or legal purposes. We do not make tax returns, and on all our invoices we state that no VAT is payable [For the benefit of readers outside Europe, VAT is a type of sales tax imposed on many goods and services in many European countries.]

As I have stated earlier in this interview, the question about who is legally and financially responsible for the organisation comes under Article III Section 1 of the ICGA Constitution, which is always published in the ICGA Journal prior to the tri-ennial meeting at which ICGA office bearers are elected. Section 1 states that the Executive Committee of the ICGA is charged with the administrative affairs of the association.

I feel that I have answered enough questions in this interview and do not plan to take questions from your individual readers, but I would like to add to what I have already said in this interview.

The first point that I would like to emphasize very firmly relates to what I see as irrelevant comments that have been levelled at the whole ICGA investigation process by supporters of Vasik Rajlich. It is clear that the "Rybka scandal" has aroused strong emotions on both sides of the verdict – what is unfortunate is that the strength of emotion has, in the case of many Rajlich supporters, led to comments and criticisms that are partisan to the point of lacking objectivity and/or irrelevant to the core issue. These Rajlich supporters either fail to realise and accept, or knowingly side-step, the most crucial question in all of this – Did Rajlich or did he not break the ICGA's Tournament Rule 2 when entering multiple World Computer Chess Championships?

The pro-Rajlich lobby has been strong in its criticisms of various aspects of the investigative process, starting with the composition of the investigation panel and of the panel's secretariat, continuing through criticism of Rule 2 itself, and then complaining about the force of the sanctions imposed on Rajlich following the guilty findings of the ICGA. An analogy here would be the supporters of someone being tried in a court of law complaining about the witnesses, about the court officials, the composition of the jury, the law under which the defendant had been brought to trial, and the severity of the sentence passed on a guilty defendant. But what really matters most is the question – Did the defendant commit the offence(s) that led to the investigation or trial?

So much of the pro-Rajlich rhetoric on the forums, and some of the questions in this interview, seem to be aimed at nothing more than discrediting people. Neither the forum postings nor the questions in this interview have seriously questioned the accuracy of the evidence examined by the panel. In fact I have yet to see anything that seriously questions the accuracy of that evidence. Vasik Rajlich did indeed use code derived from Fruit and Crafty, he did indeed fail to declare this with his entry applications for those ICGA tournaments, and he did indeed fail to seek and be granted permission from the authors of those programs. And with the accuracy of the evidence beyond serious question it is hardly surprising that he was found guilty by the ICGA of breaking its Tournament Rule 2. That is the crux of the matter.

Do we think the process used during this investigation was perfect? Of course not. Could it be improved? It can and will be. But it was also not flawed in any way. The panel included several excellent motivated people, with very high levels of technical and computer chess expertise. And the investigation was carried out in a straightforward and open manner, without employing any techniques that experienced programmers could reasonably describe as being biased or incorrect. While the final outcome was painful for everyone, if the integrity of the tournaments that choose to use Rule 2 is not protected, those tournaments would become pointless and no-one would be willing to compete. One important attraction of competition is to compete with your own skills, against others using their own skills. And from the sporting perspective it is not as intellectually honest and stimulating to take bits and pieces from the work of others, as opposed to developing your skills yourself. Then, winning actually means something.

We will, for future panels, limit their size to some extent, and ensure that those who join and remain in the panel actually participate in the discussion rather than stay as spectators to the process, since that seems to have been an issue raised about the Rybka panel. However, the 16 people with the backgrounds of those who participated in the voting within the Rybka panel formed a remarkably expert group with more than enough expertise to fairly and thoroughly investigate potential cases of Rule 2 violation. And in fact, for the future, I do not believe that we would want a working group of more than 30 people, any more than a judge would want to put more than 30 jurors in a room together hoping for a very large majority or a unanimous decision. Some might argue that even the 16 or more who we had actively participating in the Rybka panel is probably pushing the limit a bit, although in this case there were no disagreements at all. The very fact that the 16 who voted generally had a much higher level of understanding of computer chess programming than almost all of those Rajlich supporters whose voices have been heard in the forums, adds significant weight to the ICGA's confidence in the investigation process and in the panel's report. And the fact that those 16 people voted unanimously for guilty verdicts speaks for itself.

Many of the Rajlich supporters posting on the forums have supported the argument about "a new paradigm created by open-source programs", without realising that that concept is just nonsense. Most people still know that copying the work of others and claiming it to be original is wrong, just as it has always been.

Please use this feedback form if you wish to write or comment on this interview.

Previous articles on the ICGA/Rybka scandal

ICGA/Rybka controversy: An interview with David Levy (2) 10.02.2012 – On Monday we published part one of a lengthy interview we had conducted, via sequential email exchanges, with the President of the International Computer Games Association David Levy. The ICGA had found star programmer Vasik Rajlich guilty of "plagiarism" in his early program versions and banned him for life. Here now is part two of the interview.

ICGA/Rybka controversy: An interview with David Levy (1) 06.02.2012 – He is a star chess programmer with an unbroken record of supremacy in computer chess during the past five years. In June 2011 the International Computer Games Association found Vasik Rajlich guilty of "plagiarism" in early program versions and banned him for life. We published a vigorous defence of Rajlich and a rebuttal by ICGA President David Levy, who promised us an in depth interview.

Feedback on the ICGA/Rybka disqualification scandal 13.01.2012 – It's a long read, but many of the letters we received in reaction to the defence of the Rybka program by Dr Søren Riis are quite passionate and well thought out in their content. We start with the summary of a long rebuttal of the Riis paper that was sent to us by the ICGA and circulated on the Internet – with links to the full version and ancillary documents. Take a deep breath.

A Gross Miscarriage of Justice in Computer Chess (part four) 05.01.2012 – As proof that Vas Rajlich had copied program code the ICGA presented pages and pages of Fruit and Rybka code side by side. But, according to Dr Søren Riis, what was labeled as Rybka code was actually fabricated to look like Fruit. He criticizes the ICGA's process failures and ruminates on the reasons behind the unprecedented vendetta launched against the star chess programmer.

A Gross Miscarriage of Justice in Computer Chess (part three) 04.01.2012 – A core accusation against Vas Rajlich is that Rybka and Fruit have very similar positional evaluations, and the use of floating point numbers in Rybka’s time management code had to be copied from Fruit. Søren Riis enumerates the ten substantive evaluation differences and shows how the second accusation boils down to a single misplaced keystroke with zero impact on Rybka's play.

A Gross Miscarriage of Justice in Computer Chess (part two) 03.01.2012 – In this part Dr Søren Riis of Queen Mary University in London shows how most programs (legally) profited from Fruit, and subsequently much more so from the (illegally) reverse engineered Rybka. Yet it is Vasik Rajlich who was investigated, found guilty of plagiarism, banned for life, stripped of his titles, and vilified in the international press – for a five-year-old alleged tournament rule violation. Ironic.

A Gross Miscarriage of Justice in Computer Chess (part one) 02.01.2012 – "Biggest Sporting Scandal since Ben Johnson" and "Czech Mate, Mr. Cheat" – these were headlines in newspapers around the world six months ago. The International Computer Games Association had disqualified star programmer Vasik Rajlich for plagiarism, retroactively stripped him of all titles, and banned him for life. Søren Riis, a computer scientist from London, has investigated the scandal.