Interview with David Levy on the ICGA/Rybka scandal – Part two
Here
is part one of this interview
ChessBase: Critics of the ICGA say that at least some
of the members of the panel that led the investigation harboured deep personal
animosity towards Vasik Rajlich and had been attacking him for years. This
can be easily proved by citing many thousands of forum postings before, during
and after the investigation process. Was it wise of the ICGA to rely on such
members and in fact elect them to lead the investigation and draft the final
report? In civil or criminal court this would have lead to an immediate mistrial.
David Levy: The reason that Rajlich had been under attack
for years is that it had been known, by some people, for years, that he had
used code derived from others. The panel's Secretariat were tasked with writing
the report because they were thoroughly familiar with all of the evidence revealed
and discussed during the investigation. The ICGA office-bearers who considered
the report could themselves judge whether or not the report represented a fair
and balanced summary of the investigation and the evidence, and none of us raised
any questions or objections to cast doubt on the report as an accurate summary
of the evidence and the conclusions of the panel. Furthermore, it is quite typical
for courts to rely on "persons with animus" regarding issues of fact
(particularly ones that can be checked and verified).
You say that the reason Vasik Rajlich had been under attack for years
is that "it had been known, by some people," that he was guilty.
Many them have stood to profit from his downfall. Why was it necessary to
nominate exactly those people to the jury panel that would pass judgement
over him. It meant that the entire investigation and sentencing would in no
way represent due process, but become an ad hoc procedure put together by
the ICGA, one which one must assume could come to no other conclusion than
the one it reached. Are you satisfied that justice was done?
Yes, I am satisfied that justice was done. First I should point out that the
panel was not a jury in the usual sense of the word, because its votes were
always going to be regarded by the ICGA as advisory rather than a hard-and-fast
vote that would determine guilt or innocence. For instance, had the vote been,
9-7 in either direction, rather than 16-0 for "guilty" which it was,
the ICGA office bearers would have regarded it as a strong indication that the
opinion of panel members was nowhere near as clear cut as it turned out to be,
and that would doubtless have affected their thinking.
We did not nominate people to the panel. We asked for people who have an interest
and relevant expertise to volunteer to join the panel. The only people who were
nominated were the members of the Secretariat. At most, three active competitors
of Rajlich voted. None of those three panel members took an active role in the
discussions so they did not influence the other members. Even if we had excluded
the votes of these three, who make up fewer than one-fifth of the voting members,
all the remaining voters agreed that Rajlich was guilty of violating the ICGA's
Tournament Rule 2.
At most, three active competitors of Rajlich voted? This is definitely
not what is being said in the computer chess forums. Are you sure of the number?
ChessBase has always had access to the ICGA wiki since one of you (Frederic
Friedel) were on the panel. If you want to disagree and say there are more than
three voting active competitors, why don't you simply name them instead of mentioning
the often inaccurate things posted on the forums?
Is this a trap? In January this year programmer Ed Schröder, former
ICGA world champion, posted a few names of voting members. He was reprimanded
for revealing the information and immediately removed from the panel.
Not a trap. You would name them in a question to me and if there were more
than three then I would correct my answer. What Ed did was to name them publicly,
which is against the terms of joining the wiki. As I look at the people who
voted, I see only three active competitors. Yes, there are other chess programmers
on the panel, but of the ones that voted I do not see any selling a competing
product.
You justify the procedural decision by saying that "the world of
elite chess programmers is a very small one, and these are the people who
are best equipped to judge the issues involved in the investigation".
Really? The issue was one of copying code, which means there are literally
thousands of experts available to make a judgement – in the world of
general programming. Would it not have been simple to avoid the impression
of cronyism or lynch justice by recusing anyone from the process who stood
to gain from a guilty verdict or who otherwise had a vested interest, or harboured
personal animosity towards the accused, and instead tapping in on the vast
community of general computer science experts?
There are not "literally thousands of experts" on chess programming,
and it is obviously necessary for those whose opinions are being considered
by the ICGA to have such expertise. In addition to computer chess experts with
experience of developing chess programs the panel also included active members
such as Wylie Garvin, who is a games programmer at Ubisoft, and Mark Watkins,
a research fellow in the School of Mathematics and Statistics at the University
of Sydney, neither of whom is from the world of competitive computer chess programmers.
We do not follow the logic: It is necessary for the experts to have chess
programming expertise – but the ICGA used at least two who didn't have
this expertise?!
Perhaps I didn't make myself clear. There is a limited number of people with
a high level of computer chess expertise, and we wanted to avail ourselves of
as many of that group as we could. In addition the panel included some people
(such as Mark Watkins) whose expertise is relevant even though they have not
developed a competitive chess program.
Since the report was published we have seen no evidence to demonstrate that
the panel and the report came to the wrong conclusions, or explaining the huge
overlap of Rybka and Fruit in a manner that casts doubt on the guilty verdict.
Additionally please remember that Rajlich was repeatedly invited to join the
panel, he could also have supplied his own experts to join the panel at any
time, he was given ample opportunity by way of invitation from me to defend
himself against the allegations, and anyone with relevant expertise who believed
in Rajlich's innocence was perfectly free to join the panel of their own volition,
present whatever evidence and arguments they wished, and attempt to sway the
other panel members towards their own point of view. None of Rajlich's supporters
felt inclined to do this, which is hardly the fault of the ICGA.
Currently the world of chess engine programming is in a deep crisis:
instead of a dozen or so highly competent teams producing new ideas and techniques,
there are an increasing number of amateurs who simply reverse engineer top
programs, do some optimisation and then either distribute these programs free
of charge, or even start to sell them. As a result the commercial engine market
has been completely destroyed and the original programmers are out of work.
What has the ICGA done about this very acute problem?
First may I say that your question is misleading, suggesting as it does that
there are not "a dozen or so highly competent teams producing new ideas
and techniques". What about the authors of programs such as Chiron, Fritz,
Hiarcs, Junior, Komodo, Pandix, Shredder, Sjeng, Spark, Spike and Stockfish,
to name only eleven? Amongst these are four programs sold by ChessBase, so presumably
you consider them to be worthy products: Fritz, Hiarcs, Junior and Shredder.
As to the core of your question, the commercial engine market, the ICGA is a
non-commercial organisation and as such it does not concern itself with commercial
matters.
Has it investigated any of the clones?
ICGA investigations will be conducted as and when it is necessary to verify
that a program entered for one of our events is the original work of the programmer(s)
whose names are submitted with the program entry details.
Was it right for the ICGA to invest a gigantic amount of research and
expert capability into checking the fledgling work of the most successful
engine programmer, instead of addressing the problem of the illegal clones
which are, in the most part, derived from his work?
In general the "illegal clones" to which your question refers have
not attempted to enter any ICGA events and so there has been no reason for us
to investigate them. The Free Software Foundation would be a much more appropriate
organisation to conduct investigations into such programs if there is a prima
facie case that a copyright licence has been breached. Prior to the Rybka case
the only examples I can recall of an entry into an ICGA event being submitted
for an "illegal clone" or anything that could come under a description
that is in any way similar, are those I mentioned in my first published article
on the subject – "Attack
of the Clones".
As to the enormous amount of work and expert capability that went into the
Rybka investigation, surely this is something for which the ICGA should be praised
rather than criticized. We had to make certain before coming to any conclusions
that we had done a thorough job, so that we could be equally certain that the
evidence on which we were basing our conclusions and verdict was sufficient
in volume and sufficiently accurate in quality. To find someone guilty of such
serious infringements of ICGA Tournament Rule 2, with all of the consequences
of such a verdict, requires an extremely high level of diligence.
Just like the engine market, the ICGA World Computer Chess Championship
would seem to be in a crisis: the top seed and winner of the most recent ICGA
event is in place 53 of the CCRL
Computer Rating list (January 29, 2012). On equal hardware it is 175 points
behind the leader, Houdini, which experts say is a derivative of Ippolit,
in turn a derivative of Rybka (which is in third place). How can such "world
championships" be meaningful if they do not come close to producing the
strongest programs as winners? How can they be distinguished from club championships,
where members of a special group keep exchanging titles?
Any program that qualifies under the ICGA rules is welcome to apply to participate.
Just as the IOC has certain qualification rules for its events, so does the
ICGA.
Why do none of the top seven programs in the world participate in the
ICGA world championships?
Most of the top seven original programs do participate. Apart from Rybka none
of your "top seven" has been refused entry. If any of them have an
author willing to confirm their identity and if they have the necessary permission(s)
from the author(s) of any original programs from which they have copied or derived
code (either directly or via other programs), then they would be welcome to
apply for entry. If they have copied or derived code from another program that
likewise contains copied or derived code, they will of course need to obtain
permission(s) from all the relevant program authors.
What you advice a new programmer who is interested in building a chess
engine: write everything from scratch? Not duplicate or even look at any parts
of open source programs? Publish his program as open source if he does? Or
simply avoid ICGA events, as the clones are currently doing?
I would advise any new chess programmer to take note of ICGA Tournament Rule
2 and stick to it, if they wish to participate in any ICGA events. And if any
ICGA members wish to propose changes to our rules they are free to do so, and
we will consider their suggestions. It might be that in the future our rules
and/or tournament formats are changed, for example we might decide to add an
"Open" event in which any program may take part provided that the
team entering it has the necessary permission from anyone from whom code has
been copied or derived. But we do not yet have such an event in our calendar,
and even if we had done so in recent years Rybka's participation would still
have been against the rules because Vasik Rajlich did not ask for or obtain
permission from Fabien Letouzey (or Bob Hyatt).
Tell us about the Loop program that was developed under the supervision
of the ICGA and earned its author, Fritz Reul, a PhD from Prof. Jaap van den
Herik, chief tournament director of the ICGA and editor of its journal. There
were suspicions that Loop contained plagiarised code to a much greater extent
than that of which Vasik Rajlich has been accused.
Although Reul's thesis was supervised by Professor van den Herik the Loop
program was not developed under the supervision of the ICGA. We have received
a formal complaint from Fabien Letouzey about Loop and will be making a statement
shortly to indicate how we are going to deal with it.
Tell us a little more about the ICGA: in what jurisdiction is it registered,
where does it make its tax returns and who is legally and financially responsible
for the organisation?
I do not see what connection this has with the Rybka scandal but I am happy
to answer the question anyway. The ICGA is a non-profit organisation and as
such we have no obligation to register anywhere for tax or legal purposes. We
do not make tax returns, and on all our invoices we state that no VAT is payable
[For the benefit of readers outside Europe, VAT is a type of sales tax imposed
on many goods and services in many European countries.]
As I have stated earlier in this interview, the question about who is legally
and financially responsible for the organisation comes under Article III Section
1 of the ICGA Constitution, which is always published in the ICGA Journal prior
to the tri-ennial meeting at which ICGA office bearers are elected. Section
1 states that the Executive Committee of the ICGA is charged with the administrative
affairs of the association.
Is there anything you would like to add to what you have already said
in this interview? A lot of readers of our news page are clamouring to ask
questions. Would you be willing to answer them?
I feel that I have answered enough questions in this interview and do not
plan to take questions from your individual readers, but I would like to add
to what I have already said in this interview.
The first point that I would like to emphasize very firmly relates to what
I see as irrelevant comments that have been levelled at the whole ICGA investigation
process by supporters of Vasik Rajlich. It is clear that the "Rybka scandal"
has aroused strong emotions on both sides of the verdict – what is unfortunate
is that the strength of emotion has, in the case of many Rajlich supporters,
led to comments and criticisms that are partisan to the point of lacking objectivity
and/or irrelevant to the core issue. These Rajlich supporters either fail to
realise and accept, or knowingly side-step, the most crucial question in all
of this – Did Rajlich or did he not break the ICGA's Tournament Rule 2
when entering multiple World Computer Chess Championships?
The pro-Rajlich lobby has been strong in its criticisms of various aspects
of the investigative process, starting with the composition of the investigation
panel and of the panel's secretariat, continuing through criticism of Rule 2
itself, and then complaining about the force of the sanctions imposed on Rajlich
following the guilty findings of the ICGA. An analogy here would be the supporters
of someone being tried in a court of law complaining about the witnesses, about
the court officials, the composition of the jury, the law under which the defendant
had been brought to trial, and the severity of the sentence passed on a guilty
defendant. But what really matters most is the question – Did the defendant
commit the offence(s) that led to the investigation or trial?
So much of the pro-Rajlich rhetoric on the forums, and some of the questions
in this interview, seem to be aimed at nothing more than discrediting people.
Neither the forum postings nor the questions in this interview have seriously
questioned the accuracy of the evidence examined by the panel. In fact I have
yet to see anything that seriously questions the accuracy of that evidence.
Vasik Rajlich did indeed use code derived from Fruit and Crafty, he did indeed
fail to declare this with his entry applications for those ICGA tournaments,
and he did indeed fail to seek and be granted permission from the authors of
those programs. And with the accuracy of the evidence beyond serious question
it is hardly surprising that he was found guilty by the ICGA of breaking its
Tournament Rule 2. That is the crux of the matter.
Do we think the process used during this investigation was perfect? Of course
not. Could it be improved? It can and will be. But it was also not flawed in
any way. The panel included several excellent motivated people, with very high
levels of technical and computer chess expertise. And the investigation was
carried out in a straightforward and open manner, without employing any techniques
that experienced programmers could reasonably describe as being biased or incorrect.
While the final outcome was painful for everyone, if the integrity of the tournaments
that choose to use Rule 2 is not protected, those tournaments would become pointless
and no-one would be willing to compete. One important attraction of competition
is to compete with your own skills, against others using their own skills. And
from the sporting perspective it is not as intellectually honest and stimulating
to take bits and pieces from the work of others, as opposed to developing your
skills yourself. Then, winning actually means something.
We will, for future panels, limit their size to some extent, and ensure that
those who join and remain in the panel actually participate in the discussion
rather than stay as spectators to the process, since that seems to have been
an issue raised about the Rybka panel. However, the 16 people with the backgrounds
of those who participated in the voting within the Rybka panel formed a remarkably
expert group with more than enough expertise to fairly and thoroughly investigate
potential cases of Rule 2 violation. And in fact, for the future, I do not believe
that we would want a working group of more than 30 people, any more than a judge
would want to put more than 30 jurors in a room together hoping for a very large
majority or a unanimous decision. Some might argue that even the 16 or more
who we had actively participating in the Rybka panel is probably pushing the
limit a bit, although in this case there were no disagreements at all. The very
fact that the 16 who voted generally had a much higher level of understanding
of computer chess programming than almost all of those Rajlich supporters whose
voices have been heard in the forums, adds significant weight to the ICGA's
confidence in the investigation process and in the panel's report. And the fact
that those 16 people voted unanimously for guilty verdicts speaks for itself.
Many of the Rajlich supporters posting on the forums have supported the argument
about "a new paradigm created by open-source programs", without realising
that that concept is just nonsense. Most people still know that copying the
work of others and claiming it to be original is wrong, just as it has always
been.
Please use this
feedback form if you wish to write or comment on this interview.
Previous articles on the ICGA/Rybka scandal
 |
ICGA/Rybka controversy: An interview with David
Levy (2)
10.02.2012 – On Monday we published part
one of a lengthy interview we had conducted, via sequential email
exchanges, with the President of the International Computer Games Association
David Levy. The ICGA had found star programmer Vasik Rajlich guilty
of "plagiarism" in his early program versions and banned him for life.
Here now is part
two of the interview. |
| |
ICGA/Rybka controversy: An interview with David Levy
(1)
06.02.2012 – He is a star chess programmer
with an unbroken record of supremacy in computer chess during the past
five years. In June 2011 the International Computer Games Association
found Vasik Rajlich guilty of "plagiarism" in early program versions
and banned him for life. We published a vigorous defence of Rajlich
and a rebuttal by ICGA President David Levy, who promised us an indepth
interview. |
 |
Feedback on the ICGA/Rybka disqualification scandal
13.01.2012 – It's a long read, but many of
the letters we received in reaction to the defence of the Rybka program
by Dr Søren Riis are quite passionate and well thought out in their content.
We start with the summary of a long rebuttal of the Riis paper that was
sent to us by the ICGA and circulated on the Internet – with links to
the full version and ancillary documents. Take
a deep breath. |
 |
A Gross Miscarriage of Justice in Computer Chess (part
four)
05.01.2012 – As proof that Vas Rajlich had
copied program code the ICGA presented pages and pages of Fruit and Rybka
code side by side. But, according to Dr Søren Riis, what was labeled as
Rybka code was actually fabricated to look like Fruit. He criticizes the
ICGA's process failures and ruminates on the reasons behind the unprecedented
vendetta launched against the star
chess programmer. |
 |
A Gross Miscarriage of Justice in Computer Chess (part
three)
04.01.2012 – A core accusation against Vas
Rajlich is that Rybka and Fruit have very similar positional evaluations,
and the use of floating point numbers in Rybka’s time management code
had to be copied from Fruit. Søren Riis enumerates the ten substantive
evaluation differences and shows how the second accusation boils down
to a single misplaced keystroke with zero
impact on Rybka's play. |
 |
A Gross Miscarriage of Justice in Computer Chess (part
two)
03.01.2012 – In this part Dr Søren Riis of
Queen Mary University in London shows how most programs (legally) profited
from Fruit, and subsequently much more so from the (illegally) reverse
engineered Rybka. Yet it is Vasik Rajlich who was investigated, found
guilty of plagiarism, banned for life, stripped of his titles, and vilified
in the international press – for a five-year-old alleged tournament rule
violation. Ironic. |
 |
A Gross Miscarriage of Justice in Computer Chess (part
one)
02.01.2012 – "Biggest Sporting Scandal since
Ben Johnson" and "Czech Mate, Mr. Cheat" – these were headlines in newspapers
around the world six months ago. The International Computer Games Association
had disqualified star programmer Vasik Rajlich for plagiarism, retroactively
stripped him of all titles, and banned him for life. Søren Riis, a computer
scientist from London, has investigated
the scandal. |