In June 2011 it was widely reported in the global media that the International
Computer Games Association (ICGA), headed by Dr. David Levy, had found chess
programmer Vasik Rajlich in breach of the ICGA’s annual World Computer
Chess Championship (WCCC) tournament rule related to program originality. In
the ICGA’s accompanying report it was asserted that Rajlich’s chess
program Rybka contained “plagiarized” code from Fruit, a program
authored by Fabien Letouzey of France.
Rajlich, who cruised to victory in four consecutive WCCC tournaments in 2007,
2008, 2009 and 2010, was retroactively stripped of all titles he had won in
competition and was slapped with a lifetime ban. He claimed complete innocence
and made it clear that he found the ICGA’s investigatory process and conclusions
to be biased and unprofessional, and the charges baseless and unworthy. Rajlich
refused to be drawn into a protracted dispute with his accusers or mount a comprehensive
defence. This was done by Dr Søren Riis, a computer scientist from London,
in a paper we published in four parts. There was vigorous feedback from the
chess community, including a rebuttal by the ICGA. Links to the relevant articles
are given at the bottom of this page.
In his rebuttal David Levy agreed to do a full one-on-one exclusive interview
with ChessBase on the entire ICGA/Rybka scandal. It was conducted by sequential
emails, and took a fair amount of time, since David was consulting with experts
who live in different time zones around the globe.
Interview with David Levy on the ICGA/Rybka scandal
ChessBase: David, do you believe that the ICGA has conclusively
proved that Vasik Rajlich copied actual code (as opposed to ideas and algorithmic
techniques) from the Fruit program?
David Levy: At the outset I wish to make it clear that this
is not an issue about verbatim copying. We have proven that Rybka is a derivative,
but due to different internal board representations the Fruit code required
modification when being copied. But a conversion such as mailbox to bitboard
still leaves it a derivative, as would changes such as refactoring the code
and tuning the numbers.
IM David Levy, the President of the ICGA
The forums continually mention code copying instead of Rajlich failing to report
a derivative in his ICGA entry submissions. It is important to remember that
the offence for which Rajlich was convicted and banned by the ICGA was failing
to comply with our Tournament Rule 2. The various reports (here
show that Rybka 1.0 beta and Rybka 2.3.2a were clearly derivatives of Fruit
2.1. Postings on the forums and Riis stating that merely retyping or converting
a program to another format are clearly wrong. Merely retyping and/or converting
code from one format to another is still making a derivative, as too are some
changes that affect game play (such as evaluation tuning). So yes, the Rybka
entries were derivative of the Fruit code in a manner that broke Rule 2. And
Rajilch says that "Rybka is original at the level of source code"
meaning "the author either typed his own code or typed the code which generated
his own code", going so far as to say that Rybka derivatives which he had
previously censured are "original at the source code level" if they
were typed by hand. If this were the case then, if instead of cutting and pasting
a programmer just re-types by hand what he has stolen, then according to Rajlich
it is original and does not violate ICGA or copyright rules! Really?
In which versions of his program did this occur?
Rybka versions from 1.0 Beta through 2.3.2a were shown to be clear derivatives
of Fruit 2.1.
Did any of these versions (for which there is definite proof that they
contain derivatives Fruit code) play in an ICCA or ICGA event?
Yes, Rybka 2.3.2a definitely competed in ICGA events. The evaluation of Rybka
2.3.2a clearly was
derived from Fruit 2.1. Note that the report states "Rybka 2.3.2 was
released Jun 13, 2007, and 2.3.2a about five days later. Rybka competed in and
won the ICGA Chess Tournament (Jun 11-18 2007)". ICGA analysis of several
versions of Rybka between 1.0 beta and 2.3.2a show they also contained a derivative
of the Fruit 2.1 evaluation function. During this period a version of Rybka
2.0 competed in the 2006 ICGA World Computer Chess Championship tournament (May
the name "Rajlich". The closest commercial release was the MP-only
Rybka 2.0 Beta 4 with a release date of May 17 (Beta 4).
During the investigation process one member of the ICGA investigation secretariat
asked Rajlich "1. Are you able to provide us with the exact .exes that
played in all ICGA events particularly those from 2006 and 2007? If not, is
it a fair approximation to use Rybka 2.3.2a for the 2007 event?" On the
Rybka Forum, Rajlich said "Yes" to the second question, confirming
that this version was used in an ICGA event.
Were any sold commercially?
Yes, see above. Rybka 2.0 Beta 4 and Rybka 2.3.2a were sold commercially.
What do the ICGA experts think about the current versions of Rybka (i.e.
versions 3 and 4)? Do they also contain code taken from Fruit?
The ICGA has not done extensive analysis of newer versions of Rybka. It was
sufficient for the ICGA to determine that Rajlich had broken Tournament Rule
2, which we did. When the ICGA chose to ban Rybka for life due to violating
the rules in ICGA events in 2006 and 2007, this required all of Rajlich's programs
be removed, since the newer programs would never have been allowed to compete
if the violation of earlier versions was discovered sooner and the lifetime
ban issued earlier. So while it is possible that newer versions of Rybka also
contained Fruit-derived code, it was not essential to prove this once we had
proved the case for the 2006 and 2007 ICGA tournaments.
If Rybka was, as you put it, a derivative of Fruit 2.1, how do you explain
that it was massively stronger? Normally by tuning an existing program, even
your own, you gain 30 point per year. We know that from all our engine programmers.
150 points is unheard of.
The panel did not do much examination as to what made Rybka stronger, so I
would suggest you ask Rajlich directly. But experiments with Strelka, which
Vas has said is a clone/derivative of Rybka, show gains due to bitboard speedup,
64 bit code being more efficient than 32 bit code, the addition of a form of
futility using verification by a qsearch, and tuning the Fruit evaluation. Harder
to test is effect of the the Rybka material imbalance tables, but they could
have caused some gain as well.
If you compare the 32 bit version of Rybka 1.0 (Elo 2885) to Fruit 2.1 (Elo
2793), the Elo difference is only 92. The speedup going to 64 bit code (which
benefits bitboard programs more than mailbox) is about 34 Elo. In versions of
Rybka higher than 1.0, there is speculation that more strength advancement came
through additional evaluation tuning, and search changes.
The use of Fruit gave Rajlich a huge head start, as he only had to improve
a 2793 Elo program instead of writing one from scratch. To get some idea of
what small changes to a strong program can do, consider what an amateur programmer
did with Fruit in creating Toga II (an admitted Fruit derivative). He was able
to add 135 Elo in strength to the Fruit code base, bringing it to an Elo of
2928. The ratings we quote are from are here.
It would appear that Rajlich was punished not for taking code from
Fruit 2.1 verbatim, or "deriving" his program from it, but for failing
to comply with our Tournament Rule 2. If for argument's sake we assume that
Rybka 1.0 Beta through 2.3.2a were clear derivatives of Fruit 2.1, what exactly
would he have had to do to comply with Tournament Rule 2?
As we have intimated on previous occasions he should have revealed, when submitting
his entries, that Rybka was a derivative of Fruit, in order to allow the ICGA
to decide whether it should be allowed entry.
What is the ICCA definition of "derivative"? If a programmer
takes six lines of code from one of the many open source programs (Crafty,
Fruit, Strelka, Stockfish, etc.) must this be specified in the entry form
to comply with Rule 2? What if he takes twelve lines? 24? 200? Or can it be
given in percentage of the entire playing code? Where is this defined in the
ICCA rules or charter?
One doesn't normally define things so precisely, for the same reason that the
United States Constitution is open for interpretation in various places. If
you paint yourself into a corner, you are stuck there, but reasonable people
can come to a reasonable decision about "when is it too much to allow."
And this is really an issue of functionality, rather than counting characters
or lines. For example, if you copy a pawn evaluation procedure, that is copying
too much. If you find a line of code that does something in a clever way you
did not think of yourself, that probably would not even be noticed.
Your question implies that Rajlich might only have copied a small and insignificant
amount of Fruit, while the conclusion of the ICGA investigation was that a lot
of code was copied. When considering the extent of copying no one on the panel
thought that what Rajlich had done was "borderline" in a quantitative
sense. The panel actually discussed how much evidence and which of the evidence
points would be necessary and sufficient to demonstrate Rajlich's guilt, and
looked closely at evaluation feature overlap because this was seen as the most
notable and unarguable from the standpoint of Rule 2 (on program originality).
The essence of the case against Rajlich, as we repeatedly explain, is that when
entering the World Computer Chess Championship a programming team is obliged
to reveal, in their entry submission, if they have used any derived code. The
ICGA can then assess, possibly after questioning the programming team and/or
after examining the entrant's source code, whether or not the entry should be
accepted. That will depend on the nature of what was copied, permissions received
from the author of the original program, and whether another program in the
same family has applied to enter the tournament.
I would also like to stress, as a separate issue, that a programmer should always
comply with the licence of the engine from which he copies the code, as well
as declaring it on the ICGA entry form.
One should also mention here that, even if one were to ignore his copying of
Crafty code, the most serious allegations of infringements by Rajlich do not
start with his 2006 WCCC entry but with his December 2005 Rybka release and
its alleged copyright infringement of Fruit. That release is not, in itself,
part of the ICGA's grievance against Rajlich, since copyright infringement does
not become an ICGA matter unless and until the infringing software is submitted
for entry into the WCCC.
Is it fair to say that Vasik Rajlich ruined his stellar career and his
livelihood in computer chess, and got himself condemned in the world press
(including CNN, Wired) as a liar, cheat, thief, plagiarist, etc. because he
failed to fill out one line in the ICGA entry form correctly? [Note that filling
out that line correctly would have legitimised his participation and prevented
the situation that we currently have]
That "one line" would not necessarily have solved the problem. Letouzey
would also have had to give his permission as well, in order to be listed as
an author. If Rajlich had declared Rybka's Fruit origins on the entry form and
complied with the Fruit GPL he might well have been allowed to enter with Letouzey's
permission. In that case only one Fruit-based engine would have been allowed
– the one which had Letouzey's permission and his name in the entry submission.
For whatever has happened in Rajlich's life due to the ICGA's investigation
and verdict, he has only himself to blame. If people do not follow rules or
laws then they themselves are responsible for the consequences.
Let us now consider what might have followed if Rajlich's entry form had been
filled out correctly in the first place. His entry would certainly have been
rejected unless he had also obtained Letouzey's permission and put Letouzey's
name in his entry submission. What might have happened then? Perhaps Rajlich
would have become frustrated trying to write his own evaluation function, in
which case he might have approached Letouzey and joined forces with him in some
way, or he might have given up trying to create a world beating program and
someone else might have become the world beater for the next four years (either
starting from Fruit code, with Letouzey's permission, or otherwise). Or Rajlich
might have decided after all to develop his own evaluation function, in which
case who knows how strong or weak Rybka might have turned out to be?
Out of curiosity: how many other programs have listed anything or even
filled out that line in the entry form in the past?
Some programmers who enter the WCCC use the Nalimov endgame tablebases and
declare as such in their entry submissions. Some programmers use openings books
developed by others and similarly declare this in their entry submissions. I
do not recall any entry submission that admitted to copying an evaluation function
as Rajlich did (particularly in his early versions).
How many members were there in the panel that investigated Vasik Rajlich?
How many voted? Who were the members who voted and how did they vote?
At the time of voting the panel was made up as follows: There were three members
of the investigation Secretariat; four members of the ICGA board (who did not
vote prior to the completion of the report), and the WCCC Tournament Director
(who also did not vote prior to the completion of the report), and 34 members
of the panel who were invited to vote if they wished. Here is exactly what we
asked the panel members under the topic of "Opinion".
"Panel members are kindly invited to review the Rybka-Fruit
Controversy pages and render opinions. Please answer these questions:
(a) Have you read all of the evidence linked to the Rybka-Fruit Controversy
(b) In your opinion, did Vasik Rajlich's chess program Rybka violate the
Rules, especially rule 2:
"2. Each program must be the original work of the entering developers.
Programming teams whose code is derived from or including game-playing code
written by others must name all other authors, or the source of such code,
in the details of their submission form. Programs which are discovered to
be close derivatives of others (e.g., by playing nearly all moves the same),
may be declared invalid by the Tournament Director after seeking expert advice.
For this purpose a listing of all game-related code running on the system
must be available on demand to the Tournament Director."
Opinions should be presented within two weeks so that the Summary can be
So each panel member who responded to the invitation to vote answered two
questions. Question (a) was to determine if they had read all the important
evidence, so we could judge if the panel member was qualified to answer (b).
Question (b) was to get the opinions of panel members as to whether or not Rajlich
violated Rule 2. It was not really handled like a jury trial, and the final
determination of guilt was up to the ICGA Executive Committee.
Sixteen panel members voted, all of whom answered "yes" to both questions.
The ICGA would not divulge the names of those who voted (other than Ken Thompson)
without first asking their permission and we do not intend to do so.
Why must the names of the panel members (except one) be kept a secret?
Who were the three members of the "Investigation Secretariat", and
did they lead the investigation?
We decided during the investigation process on confidentiality with respect
to the names of those who voted. The Secretariat comprised: Mark Lefler, Robert
Hyatt and Harvey Williamson. They administered the investigation and they wrote
the report. Incidentally, I must re-iterate that the report was written after
the panel members voted, whereas Søren Riis incorrectly states on Chessbase.com
that "It really goes without saying that the panel members voted on the
basis of the findings of the ICGA report." He shouldn't have said that,
given that it is false.
How many votes were required to pass a verdict? Was this defined anywhere
before the investigation was undertaken?
The purpose of the voting by the panel was to assess the weight of opinion
of those panel members who wanted to make their opinions known, so that the
secretariat could take that into account when preparing their report on the
investigation and so that the ICGA office bearers could take it into account
when deciding on their verdict. After the ICGA office bearers studied the report
and the evidence submitted as part of the report, we voted on whether to find
Rajlich innocent or guilty of breaking the ICGA's Tournament Rule 2. There was
unanimous agreement on a guilty decision.
After the guilty vote how was the punishment determined? Where was the
degree of punishment or the procedure to decide on it defined?
The punishment was determined by the Executive Committee of the ICGA (excluding
the Vice-President for Asia): David Levy, Yngvi Bjornsson, Hiroyuki Iida, Rémi
Coulom and Jaap van den Herik, who was the Tournament Director of all the tournaments
under discussion. From the outset there was no doubt about disqualification
and cancellation of World Champion titles, or on the fact that a ban from participating
in future ICGA events was called for. The only matter on which there was any
debate was the duration of the ban. Some ICGA office bearers felt that ten years
was the appropriate period, while others favoured a life ban. By a majority
decision the ICGA decided to make it a life ban.
As to the procedure to decide the degree of punishment – this comes under
Article III Section 1 of the ICGA Constitution, which is always published in
the ICGA Journal prior to the tri-ennial meeting at which ICGA office bearers
are elected. Section 1 states that the Executive Committee of the ICGA is charged
with the administrative affairs of the association.
When the investigation process against Rajlich was initiated and when
the guilty verdict plus sentence was passed, the ICGA issued press releases
and distributed them freely. The banning of Rajlich for life was reported
in chess blogs and in the international media (including the New York Times
and CNN). We cannot understand why the voting members of the jury who found
Rajlich guilty need to be kept a strict secret (except for one, who is proudly
listed). This creates the impression that some of them may have had a vested
interest in the outcome of the investigation. How many panel members that
voted were rival programmers who had been beaten by Rybka in computer chess
tournaments? How many voters received World Championship titles "posthumously"
when Rybka was stripped of its ICGA titles?
We decided at the start of the process that we would not reveal the names
of those who voted, so we did not know at that time who would vote and which
way they would vote.
The suggestion that some of those who voted may have had a vested interest
in a "guilty" verdict overlooks one very important and indisputable
fact. The world of elite chess programmers is a very small one, and these are
the people who are best equipped to judge the issues involved in the investigation.
So Rajlich's actions were being assessed by his peers. If the ICGA had excluded
from the panel anyone who is an elite chess programmer we would have been significantly
diminishing the overall quality of the investigation. Remember also that I,
as President of the ICGA, and the other Board members who I have listed above,
were able to see not only all the evidence and all the votes of the panel, but
also we were able to follow the discussion on the closed Wiki that was set up
for the benefit of panel members. If any of us had seen any comments that led
us to believe that any unfair bias existed within the panel's deliberations,
we would have had the right and the duty to say so at the time.
Finally, on this point, I would like to add that I have always enjoyed a courteous
relationship with Vasik Raljich, and until the evidence leading to this investigation
was presented to me I had absolutely nothing against him. In fact, when I was
first told the rumour (as it then was) that he had used code derived from Fruit,
I did not believe it, largely because I do not regard the person who told me
as a credible source. Furthermore, if I had had a choice of the verdict (in
the absence of the evidence) I would very much have preferred it to have been
a "Not guilty" one, as the investigation and its aftermath could do
absolutely no good for the image of computer chess. The only good that has come
out of the whole affair so far as I am concerned, is that programmers who are
tempted in the future to break the ICGA's tournament rules in a serious way
will now be less likely to do so.
– Part two will follow shortly –
Please do not write in commenting on this interview until part two
has been published!
Previous articles on the ICGA/Rybka scandal
||Feedback on the ICGA/Rybka disqualification scandal
13.01.2012 – It's a long read, but many of
the letters we received in reaction to the defence of the Rybka program
by Dr Søren Riis are quite passionate and well thought out in their content.
We start with the summary of a long rebuttal of the Riis paper that was
sent to us by the ICGA and circulated on the Internet – with links to
the full version and ancillary documents. Take
a deep breath.
||A Gross Miscarriage of Justice in Computer Chess (part
05.01.2012 – As proof that Vas Rajlich had
copied program code the ICGA presented pages and pages of Fruit and Rybka
code side by side. But, according to Dr Søren Riis, what was labeled as
Rybka code was actually fabricated to look like Fruit. He criticizes the
ICGA's process failures and ruminates on the reasons behind the unprecedented
vendetta launched against the star
||A Gross Miscarriage of Justice in Computer Chess (part
04.01.2012 – A core accusation against Vas
Rajlich is that Rybka and Fruit have very similar positional evaluations,
and the use of floating point numbers in Rybka’s time management code
had to be copied from Fruit. Søren Riis enumerates the ten substantive
evaluation differences and shows how the second accusation boils down
to a single misplaced keystroke with zero
impact on Rybka's play.
||A Gross Miscarriage of Justice in Computer Chess (part
03.01.2012 – In this part Dr Søren Riis of
Queen Mary University in London shows how most programs (legally) profited
from Fruit, and subsequently much more so from the (illegally) reverse
engineered Rybka. Yet it is Vasik Rajlich who was investigated, found
guilty of plagiarism, banned for life, stripped of his titles, and vilified
in the international press – for a five-year-old alleged tournament rule
||A Gross Miscarriage of Justice in Computer Chess (part
02.01.2012 – "Biggest Sporting Scandal since
Ben Johnson" and "Czech Mate, Mr. Cheat" – these were headlines in newspapers
around the world six months ago. The International Computer Games Association
had disqualified star programmer Vasik Rajlich for plagiarism, retroactively
stripped him of all titles, and banned him for life. Søren Riis, a computer
scientist from London, has investigated