Do Women Play More Beautiful Chess? – A Response to Critics
By Azlan Iqbal, Ph.D.
In response to the original
article, I received a lot of feedback; most of which was negative and
the rest neutral. This was not entirely unexpected given the subject matter.
Regardless, not all the feedback (including the personal attacks) were seemingly
from militant feminists and men who felt as if I had just insulted their
girlfriends or wives. The former may feel that we are still at risk of regressing
to a time when the roles of men and women in society were more clearly defined
and the latter are probably just succumbing to what some might say are protective
or defensive evolutionary instincts (e.g. mother for child, men for ‘defenseless’
women). There is only so much value or credibility one can attach to anonymous
commentators on the Internet. Are they really who they claim to be? One
also has to wonder if they would have responded similarly, or in stark contrast
praised the work, had I found that women played more beautiful
chess (even if just within the scope of mate-in-3 sequences).
Besides, anyone taking single sentences totally out of context and drawing
conclusions from that about the whole body of work or writing a long, angry-sounding
‘rebuttal’ the very next day (especially with admittedly little
or no background in artificial intelligence or computer science) is doing
a poor job of making a cogent argument or providing constructive criticism.
For the record, and to assuage any concerns, I did not doctor any of the
experimental results and ChessBase did not strike some kind of Faustian
bargain with me to exploit women for the sake of a few more mouse clicks.
Our relationship goes back many years and has virtually nothing to do with
women or money. As academics, we are actually used to negative feedback,
especially when it pertains to new or controversial ideas being proposed.
I recall in Cambridge University back in 2008 when I first met him, Lotfi
Zadeh (who introduced the concept of fuzzy sets) said that when he first
proposed some of his ideas, some of his peers said he “should be lynched”
for trying to promote a ‘lack of precision’ in computing. Fortunately,
to my knowledge, none of my peers feel that way about my work.
Anyway, some of the feedback I received were nevertheless actually genuine
questions and concerns (including about my credentials and credibility)
that I felt I should address in the interest of science; hence this follow-up
article. I will not list the questions individually as many of them overlap.
Instead, I will simply begin to address them collectively based on my interpretation
of what they were trying to get at. Let me begin by saying that the aesthetics
model in chess that I developed for my Ph.D. is considered a seminal piece
of work in then uncharted waters. Therefore a realistic and doable scope
needed to be set, i.e. three-move mate sequences. This is actually how one
goes about doing a Ph.D. You should not bite off more than you can chew
or you will never complete the degree. ‘Aesthetics’ was also
defined as a common ground between the domains of chess problem composition
and real games, since both are not without beauty. The necessary experiments
were performed and the results showed that the computer could indeed apply
the model to recognize aesthetics within the game (given three-movers) in
a way that correlated positively and well with domain-competent human assessment.
This had never been demonstrated before and was a significant contribution
to the pool of knowledge in artificial intelligence. It also had practical
applications, such as allowing the aesthetic analysis of thousands upon
thousands of chess problems and winning move sequences in games that would
be far too difficult for humans to do reliably.
During my study, I worked with many chess experts as well. In order to
understand all this satisfactorily, you will probably have to read
my thesis in its entirety. There are no shortcuts, just as there were
none for me in preparing it. Even though we tend not to read as much as
we used to (we see more pictures and videos now), we are not yet at the
stage where we can depend on a computer to comprehend complex written texts
and answer intelligent questions intelligently thereby saving us a lot of
time and effort. I was awarded my Ph.D. from the University of Malaya which,
when I graduated, was the top university in Malaysia. They also had a policy
that, in addition to my own supervisor, there would be one internal and
two external reviewers, all of whom must be full professors with related
expertise. The external reviewers must also be from overseas, which in my
case happened to be from renowned universities in the UK and Australia.
My thesis was with them for seven months.
Also, unlike some institutions, all four professors (including my supervisor)
must unanimously agree that the Ph.D. be awarded. It is not unusual or uncouth
for one who has successfully attained their Ph.D. to put the ‘Ph.D.’
title at the end of one’s name in scientific reports or articles,
just as medical doctors would put ‘M.D.’ at the end of theirs.
Chess grandmasters are also known for putting ‘GM’ at the front
of theirs in articles and it is ironic and hypocritical that any of them
should think doing this is cocky or ‘showing off’. Given all
this, for anyone to suggest that the aesthetics model developed lacks credibility
is to declare one’s ignorance or preconceptions about me and the region
I am from. While I have enjoyed visiting the West many times on business
and pleasure, I have never had any intention of actually working or staying
there. Not once at any stage of my education or career have I ever even
applied to do so. I am quite happy living in and serving my own country.
Thanks to the growth of the Internet, we are living in a virtually borderless
world anyway.
The three-mover aesthetics model I developed and tested was furthermore
extended in our
2012 IEEE paper, also with the help of chess experts (who also happen
to be Ph.D. holders), to include not only three-movers but also studies
(and logically, longer mates). The paper, of course, was thoroughly peer-reviewed
and had to be revised before being accepted for publication so the extended
model is also, ‘experimentally-validated’. It is true that in
validating the model, the average of three cycles of evaluation by Chesthetica
(as opposed to just one cycle) for each move sequence was tested but using
just one cycle in future experiments is valid as well because, like a human
judge of aesthetics, Chesthetica may or may not deliver exactly the same
evaluation each time it looks at the same sequence (you will have to read
the IEEE paper carefully to learn why this works well). This phenomenon
is of little concern because the program’s consistency and reliability
‘over time’ has already been demonstrated by taking the average
of its evaluations using multiple cycles. It does not imply that multiple
cycles should always be used in the future and that only crisp, unchanging
aesthetic values are acceptable for each sequence. The possibility of slight
variations in an aesthetic evaluation makes the model more dynamic yet still,
on average, consistent and reliable (much like a human judge). As for replication
of experimental results, that depends on the hypothesis. Should the original
hypothesis have stated that a single cycle of evaluation be used, then the
replication of the experiment should use a single cycle as well and the
result accepted, whatever it may be. Analogously, the p-value for
statistical significance should also be determined beforehand and not reset
after the experiment to better suit the results (e.g. changing it from 0.01
to 0.05).
While I have written many papers, to the extent memory serves, I do not
self-publish at all even though admittedly, some of my publications are
certainly better or more prestigious than others (like any academic). For
example, when Britannica invited me to write the entry for “computational
aesthetics” (my PhD field of study) in their encyclopedia or Springer’s
recent publication of our book on the DSNS
approach that my Chesthetica software uses to create original chess problems.
Not to mention many papers published in the ICGA
Journal, a reputable computer games journal with a high standard of
publication. Ken Thompson published there too. I also have papers in high
ranking AI conferences such as the AAAI and IJCAI. I do not ordinarily like
to draw attention to these things but when questioned, I suppose I must
set the record straight. As a side note, it is probably not a good idea
to prepare conference slides at the last minute because typos may show up
and you really cannot tell how seriously some people might take things like
that and use it to draw conclusions about you.
As for ‘impact factor’, academics are well-aware of its limitations
and interested readers might care to look those up as well, such as
explained here. In short, it is not necessarily a good indicator about
the quality of any particular piece of research work. For instance,
a paper essentially reminding us yet again about the dangers of consuming
too many burgers, fries or sodas could have a high impact factor largely
because it is published in a popular medical journal, because medical science
tends to get the most research funding and because many of them tend to
study our eating habits (a lot more people than those looking into say,
the computational aesthetics aspect of chess). On the other hand, I write
for ChessBase (with no impact factor) because it is a kind of ‘community
service’. As academics, we are expected to convey our research to
the public in more palatable and widespread forms than just technical papers
which tend to be of rather limited distribution and out of the layman’s
typical scope of understanding, and in many cases even those from outside
the particular field.
With regard to my chess-playing expertise, I never bothered to obtain an
official chess rating even though I have been playing casually for 30 years
and have won several medals in local tournaments. In fact, I am quite confident
I could last at least 20 moves even against Magnus Carlsen under tournament
conditions. If I had an official Elo rating, my probability of winning that
match (or my ‘expected score’) can indeed be calculated and
would probably be so low one might think I would lose faster than Bill Gates.
I could probably beat him too, by the way. So, yes, I can say that I do
indeed “know how to play” but would not consider myself an ‘official
master’ at the game. The truth is, given my line of work, I simply
do not need to be a chess master as there are many official chess masters
only too happy to assist and work with me on projects. I am frankly quite
amazed at how open-minded and forward-thinking some of them are.
The same can be said about scientists who study say, bodybuilding. They
are not and need not necessarily be renowned bodybuilders themselves (though
they do probably work with a few). Having said all that, I do not think
I am smarter or “more intelligent than everyone else”. It is
not like my IQ is in the 180-range or anything like that. I took a scientifically-accurate
test back in 2003 and it was only 131 with the “unusual distinction
of being equally good at math and verbal skills”. It is unfortunate
that some people interpreted the original article here on ChessBase to be
misogynistic, having “gratuitous sexist content” and claim that
perhaps I did not even know any women. I was also ‘threatened’
that my academic standing and credibility would be undermined by all this
and that I should think about my future in academia. Untrue on all counts,
I would say. I have known plenty of women in my time. At last count, 52
from 23 different countries, as a matter of fact; and most of them would
only have nice things to say about me, I am fairly confident.
As for academic standing, I am more concerned about scientific truth than
what the effects of revealing it might have on my career. Certainly, not
revealing it (the file-drawer effect) or trying to bury it without a good
enough reason would have a greater effect on society (myself included).
Besides, not all academics are so desperately looking for tenure or its
equivalent and would ‘do or conceal anything’ to get it. Some
of us (though I am not necessarily claiming to be in this group) –
and presumably just like some grandmasters – may also be independently
wealthy and could retire tomorrow if we pleased; never having to work another
day in our lives. So now, after hopefully having set the record straight
on these matters, let us look into some of the other concerns about the
experiments in my
paper that suggested men play more beautiful chess than women.
The first thing one should realize when reading a scientific paper is that
there is probably always a scope specified (e.g. three-move mate sequences).
There ought to be. Scientists do not claim to know everything and
the scope serves as an indicator about the extent to which whatever was
being tested was actually tested or could be tested. This does
not mean nothing useful can be said about the subject matter. For instance,
we may only know how certain parts of the brain function with respect to
certain aspects of human activity, but that does not mean those findings
are useless until and unless neuroscientists know how the whole
brain works with regard to all of human activity. Science is a cumulative
and self-corrective process.
Now, some ‘experts’ may feel their personal or collective intuitions
about certain things trump experimental validation. However, from a scientific
standpoint they are wrong. What you need to trump experimental validation
is more or better experimental validation. ‘Common sense’ is
not a scientific argument and has been known to be wrong or misleading.
Just like one might be inclined to think that a bowling ball would hit the
ground faster than a feather dropped from the same height in a vacuum chamber.
So if anyone would like to analyze longer or different types of sequences
in chess using say, some other method, you will first need to develop and
experimentally validate your own aesthetics model for those types of sequences
or all you have is essentially just personal (and quite probably biased)
intuitions. Being a master player does not help you scientifically here.
Moving on to the perfectly valid question about whether playing strength
correlates with aesthetics. In other words, do stronger players play more
aesthetically? In the original study, that was not taken into account but
the study does contrast the aesthetics of play between two engines, i.e.
Rybka 3 vs. Fritz 8 (10+10) and Rybka 3 vs. Fritz 8 (1+1) scoring, on average,
1.979 and 1.992, respectively. The difference was not statistically
significant. So this would suggest that playing strength is not necessarily
relevant to beauty. However, I did happen to have two older databases with
me with 1,000 randomly selected games that ended in mate between players
with an Elo rating above 2,500 and between players with an Elo rating below
1,500. The games were sourced from Big Database 2011 and gender was irrelevant
here, even though most were likely games between men, especially given the
first set (so perhaps the result does not even apply to games between women).
The average aesthetics scores (using the same statistical approaches as
described in the original paper) were 1.815 and 1.693, respectively, and
the difference was indeed statistically significant. So this suggests further
that playing strength is relevant in the aesthetics of three-move mating
sequences that result from play between humans.
What are the implications of this? Should playing strength have been taken
into account in the original study so that only games between women within
the same Elo range as the games between men were used? Perhaps it should
but unfortunately, this was not possible without tampering with the selection
process which is supposed to be random because there is no automatic (and
unbiased) way to search for players based on their gender and there were
simply not enough games between women in the database ending with ‘exclusivity’
(read the original paper to learn what this means) and mate that were also
within any particular Elo range. Of course, most games between strong players
do not even end in mate (they tend to resign) but again, the aesthetics
of games like that are at present not scientifically testable. Besides,
in comparing samples of the same kind (i.e. three-move mate sequences) drawn
from a normal population (i.e. whatever was randomly obtainable from the
database) the differences between men and women are still valid (within
that scope, obviously).
The original study minimized introducing any kind of bias into the samples
(of both men and women) by assuming that whatever was in the 6+ million
game database used was an unbiased representation of games played throughout
the world by both men and women. If it happens that there was a greater
number of strong male players than female players in that database and therefore
the samples of each used also reflected that and thus the games between
females would necessarily score lower aesthetically... well, that begs the
question, why, in a normal population, are there more games by stronger
male players to begin with? This is not something that can be ‘adjusted
for’ without introducing bias into the samples. If I were to select
only specific, strong female players to compare against specific, strong
male players... that would introduce so much bias I would have to justify
how and why each of those players were chosen. It does not reflect what
is typically found in the real-world population of players and what can
realistically be selected at random from that. Now, imagine the
additional biases introduced if arbitrary, ‘manual’ filters
based on age were also applied.
Similarly, if I were to test a mixture of longer mates and study-like endings
(which Chesthetica can also analyze aesthetically now) along with three-movers,
arguments could be made that one sample had more of one type of mate or
ending than the other sample and that affected the outcome because
experiments also show that studies score, on average, higher aesthetically
using the model than mates. Never mind yet the issue of deciding how far
back one needs to go from the ending of a game to determine where the ‘study’
starts and how that decision was made for each game (talk
about introducing bias!). This is why a doable, testable scope and consistency
in experimentation is paramount. Otherwise, it makes the conclusions and
implications of the research only more tenuous. So in summary, the original
study assumed that the database used had an ‘as-fair-as-can-get-without-introducing-bias’
distribution of games between men and between women and compensated further
for bias by using the average aesthetics score.
The two games shown in the original ChessBase article, for instance, should
therefore not be seen as comparing apples and oranges but rather what the
aesthetics model thinks of the sequences themselves, independent
of who the players are or the conditions under which those moves were made
(something humans might find very difficult to ignore). Additionally, by
themselves, these two sequences are not ‘proof’ of anything
and were never intended to be. The samples of 1,069 games each that were
used surely also contained some games between women that were of higher
quality than some of the games between men. This is the beauty of random
selection and the bell curve. Hence the necessity for comparing only averages
and not drawing grand conclusions from individual games or sequences. More
games, I suppose, could have been used (e.g. by artificially flipping the
colors where Black mates and treating the position as White mates in games
that never actually occurred in that form) but again, this would have introduced
bias, especially if playing with the white or black pieces influences the
way people play at all. So since both samples featured only White wins (like
the standard for most chess problems), comparisons between them are technically
still valid. Besides, the randomly-selected 1,069 games in each sample were
considered a sufficient number for experimental purposes.
What, then, about games between men and women or games between higher and
lower rated players? How does the aesthetics analysis account for these?
In the original study, games between men and women were scarcer still and
virtually impossible to obtain automatically and randomly, so that is why
they were not used. As was pointed out to me, it is also the case that there
are ‘women only’ tournaments but no ‘men only’ tournaments.
A strange (perhaps even sexist) double-standard that automatically excludes
men (even low Elo ones) from some tournaments but does not exclude women
from any. So this would further explain the aforementioned scarcity. As
for higher rated players versus lower rated players, this was considered
introducing more variability (read as ‘lack of consistency’)
into the samples compared to using players of about the same rating. For
instance, if the mate occurred as a result of a 2,500 Elo player defeating
a 1,600 Elo player (I am guessing such games rarely take place to begin
with), the larger gap in rating points (i.e. 900) would inherently introduce
more things to be accounted for than if the difference was only, say, 150
Elo points.
There is also no evidence that a large Elo gap necessarily permits the
stronger player to play more beautiful chess but it is certainly something
I could test in future experiments given sufficient data. I do not know
if there were necessarily more games like this in the female sample used
in the original study but trying to find out and then arbitrarily deciding
which ones to include and which ones to reject (and then doing the same
for the male sample) would, once again, introduce more bias than the source
database itself yielded automatically and with no interference from me.
How about the argument that ‘forced’ three-move mate sequences
undermine creativity and aesthetics? Well, in previous research work (Aesthetics
in Mate-in-3 Combinations: Part II: Normality, ICGA Journal, December 2010),
I have shown that forced mates, on average, are actually no different aesthetically,
according to the experimentally-validated model and in the case of games
between human players, than those that are not forced.
A human player or composer may be influenced to think somewhat less of
a sequence that is not forced upon doing some deeper analysis on the position,
however. This is why forced sequences are typically considered more beautiful
and preferred in experiments because eventually, humans are going to perceive
them. Again, as long as both samples are similar in the sense of being forced
(or unforced) mates, the comparisons between them are more credible than
say, if one sample was forced and the other was not. Now, do not get me
wrong. Overall there are probably several dozen if not hundreds of different
variations or permutations of the original study that could also have been
done by filtering this out and compensating for that in order to test specifically
for this with regard to that, but those, precisely, are other experiments
with different scopes and different sets of constraints
and limitations. I really do hope there are people who can find the funding
and time to do them all; armchair commentators included. I would certainly
be interested to read about the results and happy they have contributed
to the literature on the subject, in however small a way.
Finally, the conclusions of the original study are actually supported by
the fact that in the world of chess problem composition (typically having
the highest aesthetics scores, even according to my model), the best compositions
(if not just about all of them) are by men. It could be that the
male ‘patriarchy’ of the composition world are secretly dismissing
some of the most fantastic compositions ever composed simply because they
are submitted by women, or it could also be that women, in general, are
less interested in the aesthetics of chess for reasons that neuroscientists
might be curious about (assuming learning more about the physiological differences
between male and female brains and their implications is not yet considered
forbidden research). I will leave it to readers to decide for themselves
which explanation is more likely. I have no vested interests in the outcome
and am more interested in the truth. I doubt women are so feeble-minded
and lacking confidence as to be discouraged from chess by findings such
as this and even if they turn out to be true, it is not necessarily something
that cannot be compensated for with the right tutelage from a man (or woman)
with greater skill. If anything, I hope the original study motivates even
more women into playing the game and into the world of chess problem composition
to prove they are indeed equal or even superior to men in this regard as
well.
Wrapping up, let me also congratulate Google’s DeepMind on AlphaGo’s
victory over humanity’s Go champion, Lee Sedol. I had absolutely no
doubt this would happen and can only wonder why it took so long to achieve.
By the way, Google, if you happen to have one of your quantum computers
just lying around doing nothing, I would love to plug Chesthetica into it
for a while for some serious computational creativity DSNS processing if
the two are compatible. Just kidding (well, not really). Anyway, good show
and respect to all the industry big boys out there breaking new ground and
taking board game AI seriously.
This article was first uploaded
to ResearchGate and you may contact Dr Azlan Iqbal via e-mail with any
further questions or concerns you may have, at his official
email address with a c.c. to his private
address. Yes, he does reply to the best of his ability.
Previous ChessBase articles by Prof. Azlan Iqbal
-
2/26/2016 – Do
women play more beautiful chess?
Azlan Iqbal, senior lecturer at the Universiti Tenaga Nasiona in Malaysia,
has been working for years in the field of Artificial Intelligence,
trying to program machines to evaluate aesthetics. After making the
Chesthetica software that is able to create an unlimited number of problem-like
chess constructs he has turned his attention to gender-based playing
style. Here are first results.
- 2/24/2016 – Azlan
Iqbal: Recomposition contest result
Over Christmas we showed you an interesting problem: say you have found
some moves somewhere, in coordinate notation without piece names –
is it possible to reconstruct the original supposedly meaningful position
to which they apply? The author, who has a Ph.D. in artificial intelligence,
tried to do it, but with modest success. A reader presented a more plausible
solution and won a valuable prize
-
12/29/2015 – ChessBase
Chrismas Puzzles 2015 (5)
Here's an interesting problem: say you have found some moves somewhere,
in coordinate notation without piece names – e.g. 1.h7g5 d8g5
2.b5d5 d1c2 etc. Can one reconstruct the original supposedly meaningful
position to which they apply? Azlan Iqbal, who has a Ph.D. in artificial
intelligence, retraces his thought processes when he tried, in this
unique exercise in forensic chess. Help him and you can win a special
prize.
-
5/31/2015 – Celebrating
300 machine generated problems
As we reported before, Chesthetica, a program by Azlan Iqbal, is autonomously
generating mate in three problems by the hundreds, and the author is
posting his selections in a very pleasing format on YouTube. The technology
behind the program’s creativity is a new AI approach and Dr. Iqbal
is looking for a substantial research grant for applications in other
fields.
- 4/7/2015 – Switch-Side
Chain-Chess Revisited
The search continues for a chess variant which retains the flavour of
the original game but does not succumb to the brute calculating power
of modern computers. AI researcher Azlan Iqbal has proposed his own unique
variant. Now he provides some test games and shows how Carlsen could have
won (instead of lost) WCCh Game 3 against Anand in Sochi had Switch-Side
rules applied.
-
2/6/2015 – Computer
generated chess problems for everyone
Now they are composing problems that fulfil basic aesthetic criteria!
Chesthetica, a program written by Azlan Iqbal, is churning out mate
in three constructs by the hundreds, and the author is posting them
in a very pleasing format on Youtube. How long will Chesthetica theoretically
be able to generate new three-movers? Quite possibly for tens of thousands
of years.
-
11/7/2014 – A
machine that composes chess problems
Chess problems are an art – positions and solutions, pleasing
to the mind and satisfying high aesthetic standards. Only humans can
compose real chess problems; computers will never understand true beauty.
Really? Dr Azlan Iqbal, an expert on automatic aesthetic evaluation,
imbued his software with enough creativity to generate problems indefinitely.
The results are quite startling.
-
7/26/2014 – Best
‘Chess Constructs’ by ChessBase readers
Chess constructs are basically an intermediate form of composition or
chess problem, lying somewhere between brilliancies from chess history
– and artistic chess problems, between real game sequences and
traditional award-winning compositions. A month ago Dr Azlan Iqbal explained
the concept asked our readers to submit compositions of their own. Here
are the winners.
-
6/29/2014 – Azlan
Iqbal: Introducing ‘Chess Constructs’
People love brilliancies from chess history – and artistic chess
problems. But there is a big gap between the two. Positions from games
demonstrate the natural beauty of actual play, while chess problems
are highly technical, with little practical relevance. The author of
this interesting article suggest an intermediate form, one you can try
your hand at – and win a prize in the process.
-
9/2/2009 – Can
computers be made to appreciate beauty?
Or at least to identify and retrieve positions that human beings consider
beautiful? While computers may be able to play at top GM level, they
are not able to tell a beautiful combination from a bland one. This
has left a research gap which Dr Mohammed Azlan Mohamed Iqbal, working
at Universiti Tenaga Nasional, Malaysia, has tried to close. Here's
his delightfully interesting PhD thesis.
-
12/15/2012 – A
computer program to identify beauty in problems and studies
Computers today can play chess at the grandmaster level, but cannot
tell a beautiful combination from a bland one. In this research, which
has been on-going for seven years, the authors of this remarkable article
show that a computer can indeed be programmed to recognize and evaluate
beauty or aesthetics, at least in three-move mate problems and more
recently endgame studies. Fascinating.
-
2/2/2014 – A
new, challenging chess variant
Ever since desktop computers can play at its highest levels and beat
practically all humans, the interest of the Artificial Intelligence
community in this game has been sagging. That concerns Dr Azlan Iqbal,
a senior lecturer with a PhD in AI, who has created a variant of the
game that is designed to rekindle the interest of computer scientists
– and be enjoyable to humans as well: Switch-Side Chain-Chess.
- 5/11/2014 – Kasparov
in Malaysia
He was mobbed, but in a good way: a large number of chess fans and autograph
hunters sought close contact to the legendary World Champion, who officiated
the opening of the PMB National Age Group Championship 2014, and took
time to discuss a variety of topics with an expert on aesthetics-recognition
technology in chess, our author Dr Azlan Iqbal – who sent us a big
pictorial report.
