Do women play more beautiful chess?

by Azlan Iqbal
2/26/2016 – Azlan Iqbal, senior lecturer at the Universiti Tenaga Nasional in Malaysia, has been working for years in the field of Artificial Intelligence, trying to program machines to evaluate aesthetics. After making the Chesthetica software that is able to create an unlimited number of problem-like chess constructs he has turned his attention to gender-based playing style. Here are first results.

Do Women Play More Beautiful Chess?

By Azlan Iqbal, Ph.D.

It has been a few years since I last worked on my computational chess aesthetics model which enables a machine to evaluate aesthetics in the game in a way that correlates positively and well with domain-competent human assessment. The main reason being that the relevant experimental work there had been completed and I had moved on to other things in the field of computational creativity. In any case, even back when I started work on the aesthetics model for my PhD (around 2006), I had sometimes wondered if there was indeed a difference in the aesthetic quality of play between men and women. Based on official ratings, women are generally inferior to men in terms of strength of play but did they perhaps, on average, play more beautiful games or sequences? For the first time, the technology to test this computationally was available and since I had some free time between projects, I decided to do so.

The result was a short academic paper that appears in the proceedings of the 4th International Congress on Interdisciplinary Behavior and Social Science 2015 (ICIBSoS 2015). The full paper and presentation slides are available for reference. Before I summarize the findings here, perhaps we should ask why the question even matters? So what if men are better at chess? So what if women or men play more beautifully? Well, it matters in a variety of fields to aid our understanding of the differences between men and women; for instance, in gender studies, psychology, and even neuroscience. The more we know the more progress can be made socially, mentally and physiologically.

Having covered that, let me explain the experimental setup that was used. I worked with the ChessBase Big Database 2015 (6,251,221 games) as the main resource for games and the aforementioned computational aesthetics model. The first step was to filter all those six million or so games to those that ended with the white pieces checkmating black. The aesthetics model works with three-movers, longer mates and also studies. However, to be consistent in an experiment, we need to minimize the number of variables, and this could most reliably be done using just three-movers where White wins, which is the usual stipulation. So the first step was intended to use only games that ended in mate where the position could be ‘reversed’ three moves and the aesthetics of that final winning sequence analyzed. This left 157,358 games after filtering using the ChessBase 13 software.

The second step was to filter again for mate-in-3 ‘exclusivity’ (using my Chesthetica software). This means ensuring that those last three moves are actually a forced mate-in-3 line in the position, and not something that happened just because the winner got lucky or the opponent played poorly. This took a few days of continuous processing on a single desktop computer. Exclusivity filtering left 34,868 games. The third step was the difficult task of identifying games between men and games between women. Curiously, there is no easy way to do this using any known software. So, using ChessBase 13 we decided to run a search for the terms “(Women)” and “(Men)” in ‘any field’, and this returned the tournaments that were sensible enough to include those terms in their titles. We also got a handful of hits with the term, “girls” but were less fortunate with the term, “boys” as tournaments are less likely to mention that.

This left 1,069 games between females and only 115 games between males. Age or playing strength was irrelevant to the study. We managed to identify enough additional games between males to bring their total to 1,069 as well. We then created a random subset of the 1,069 games between females consisting of just 115 games. Ultimately we had two sets of games between men and between women that had the same sample sizes of 115 and 1,069 games. We analyzed the aesthetics of the smaller set first. It is worth noting that such evaluations by human experts would not be cost-effective or even consistent and reliable.

The experimental results obtained were interesting because given the first set of 115 games between men and 115 games between women we found the difference between their means (1.847 and 1.810, respectively) to be no different statistically. The aesthetics score is typically used for ranking purposes so even a small difference would rank one composition or game ahead of another. The second set of 1,069 games was analyzed and the difference in means (1.769 and 1.720, respectively) this time was statistically significant. So the larger set exposed a difference between the average aesthetic quality of games between females as opposed to between males. The two positions below show examples of the games between men (left) and between women (right) obtained from the sets analyzed. The solutions should be fairly obvious. There are actually many factors involved in the aesthetics analysis and interested readers may obtain more information here.

#3; position after, 38...Kf8
Karpov vs. Mair, Vienna Lugner City sim,
16 April 2005 Score: 1.756

 

#3; position after, 31. ... Kxg7
Roos vs. Neubacher, Bayern-ch (Women) 67th,
3 August 1996 Score: 1.7

The experimental evidence would therefore suggest that games between men do indeed, on average, rank higher than games between women in terms of beauty. It may be that women have less artistic appreciation of the game or play in a less artistic fashion even though they may generally be good players. This may explain the relative non-existence of master/grandmaster female composers of chess problems, for instance. Naturally, it would also follow that there are likely domains where women fare better aesthetically than men. Understanding these domains and differences better would add to our body of knowledge about the human brain and gender differences and may even help optimize human performance in domains where equal gender distribution is not an issue.

In terms of aesthetics in chess, the difference discovered may or may not extend to games that do not end in mate, but the longer the sequence that needs to be investigated, the less reliable the experiment. There is little reason to believe they do not, however. The availability of data is also a significant issue. We would have been able to analyze games between men and women but these were scarcer still. Similarly, comparing chess problems of the three-mover variety would also be possible but there are simply not enough compositions of that type by female composers and of a similar level to compare against those by men. On the argument that chess has been ‘historically male’ and therefore any aesthetic principles derived from the game are also ‘male’, I believe it first needs to be demonstrated that men and women perceive beauty in the game differently. The default assumption is that they do not.

In general, what we have demonstrated should not be taken too seriously as it only opens a point of inquiry related to the game that may not have been as properly considered before. There is still plenty of room for further work in this area to challenge our preconceptions. On the surface, many people might be inclined to think women, in general, do not even play chess yet even among the most unexpected candidates, we find women who do. For instance, apparently several Playboy USA Playmates, Bunnies and Models, e.g. Miss June 1956 (Gloria Walker), Miss July 1958 (Linnè Nanette Ahlstrand), Miss January 1959 (Virginia Gordon), Miss November 1971 (Daniell de Vabre), Miss March 1975 (Ingeborg Sørensen), Bunny of 1978 (Pamela Bunn), Miss April 1994 (Becky DelosSantos), Model from December 2002 (Sabrina Kassim) and Model from May 2008 (Olga Voch-Mianina). Who would have guessed?

After my presentation of this paper in Jakarta in November 2015, some interesting questions were asked. Among them is whether or not aesthetics of play also relates to cultural differences. In short, do people in say, Germany, play more beautiful chess than people in say, India? I thought it was an interesting question and it is certainly testable using the same methodology described in this paper. However, my response at the time was probably not, as long as the same version of chess was being played by both cultures. It stands to reason that if the rules and objectives of the game are the same, players regardless of their cultural origin would generally come to the same conclusions about good and also aesthetic play. However, the experimental results, whatever they are, would be a better indicator of the truth.

This led to another question. If our objectives in the game tend to be the same, why should men be better at chess than women? My answer was that chess, being a competitive sport and analogous to war and combat, was likely more the forte and even pleasure of men rather than women given the history of most cultures where men fought and died far more than women. It will no doubt take some time for men to mellow and lose that warrior spirit or women to become more combative and when that happens we will probably see less discrepancy in their strength and style of play.

Finally, as an update, Chesthetica (the aesthetics evaluation software used in this research) has her own YouTube, Twitter and Facebook accounts now. Do subscribe and follow to be informed about the latest free computer-generated chess problems you can access, enjoy and learn from anywhere in the world. Chesthetica has also been composing more sophisticated study constructs for a few months now and there will be an article on that soon right here on ChessBase; so stay tuned.

Previous ChessBase articles by Prof. Azlan Iqbal

  • 2/24/2016 – Azlan Iqbal: Recomposition contest result
    Over Christmas we showed you an interesting problem: say you have found some moves somewhere, in coordinate notation without piece names – is it possible to reconstruct the original supposedly meaningful position to which they apply? The author, who has a Ph.D. in artificial intelligence, tried to do it, but with modest success. A reader presented a more plausible solution and won a valuable prize
  • 12/29/2015 – ChessBase Chrismas Puzzles 2015 (5)
    Here's an interesting problem: say you have found some moves somewhere, in coordinate notation without piece names – e.g. 1.h7g5 d8g5 2.b5d5 d1c2 etc. Can one reconstruct the original supposedly meaningful position to which they apply? Azlan Iqbal, who has a Ph.D. in artificial intelligence, retraces his thought processes when he tried, in this unique exercise in forensic chess. Help him and you can win a special prize.

  • 5/31/2015 – Celebrating 300 machine generated problems
    As we reported before, Chesthetica, a program by Azlan Iqbal, is autonomously generating mate in three problems by the hundreds, and the author is posting his selections in a very pleasing format on YouTube. The technology behind the program’s creativity is a new AI approach and Dr. Iqbal is looking for a substantial research grant for applications in other fields.

  • 4/7/2015 – Switch-Side Chain-Chess Revisited
    The search continues for a chess variant which retains the flavour of the original game but does not succumb to the brute calculating power of modern computers. AI researcher Azlan Iqbal has proposed his own unique variant. Now he provides some test games and shows how Carlsen could have won (instead of lost) WCCh Game 3 against Anand in Sochi had Switch-Side rules applied.
  • 2/6/2015 – Computer generated chess problems for everyone
    Now they are composing problems that fulfil basic aesthetic criteria! Chesthetica, a program written by Azlan Iqbal, is churning out mate in three constructs by the hundreds, and the author is posting them in a very pleasing format on Youtube. How long will Chesthetica theoretically be able to generate new three-movers? Quite possibly for tens of thousands of years.

  • 11/7/2014 – A machine that composes chess problems
    Chess problems are an art – positions and solutions, pleasing to the mind and satisfying high aesthetic standards. Only humans can compose real chess problems; computers will never understand true beauty. Really? Dr Azlan Iqbal, an expert on automatic aesthetic evaluation, imbued his software with enough creativity to generate problems indefinitely. The results are quite startling.

  • 7/26/2014 – Best ‘Chess Constructs’ by ChessBase readers
    Chess constructs are basically an intermediate form of composition or chess problem, lying somewhere between brilliancies from chess history – and artistic chess problems, between real game sequences and traditional award-winning compositions. A month ago Dr Azlan Iqbal explained the concept asked our readers to submit compositions of their own. Here are the winners.

  • 6/29/2014 – Azlan Iqbal: Introducing ‘Chess Constructs’
    People love brilliancies from chess history – and artistic chess problems. But there is a big gap between the two. Positions from games demonstrate the natural beauty of actual play, while chess problems are highly technical, with little practical relevance. The author of this interesting article suggest an intermediate form, one you can try your hand at – and win a prize in the process.

  • 9/2/2009 – Can computers be made to appreciate beauty?
    Or at least to identify and retrieve positions that human beings consider beautiful? While computers may be able to play at top GM level, they are not able to tell a beautiful combination from a bland one. This has left a research gap which Dr Mohammed Azlan Mohamed Iqbal, working at Universiti Tenaga Nasional, Malaysia, has tried to close. Here's his delightfully interesting PhD thesis.

  • 12/15/2012 – A computer program to identify beauty in problems and studies
    Computers today can play chess at the grandmaster level, but cannot tell a beautiful combination from a bland one. In this research, which has been on-going for seven years, the authors of this remarkable article show that a computer can indeed be programmed to recognize and evaluate beauty or aesthetics, at least in three-move mate problems and more recently endgame studies. Fascinating.

  • 2/2/2014 – A new, challenging chess variant
    Ever since desktop computers can play at its highest levels and beat practically all humans, the interest of the Artificial Intelligence community in this game has been sagging. That concerns Dr Azlan Iqbal, a senior lecturer with a PhD in AI, who has created a variant of the game that is designed to rekindle the interest of computer scientists – and be enjoyable to humans as well: Switch-Side Chain-Chess.

  • 5/11/2014 – Kasparov in Malaysia
    He was mobbed, but in a good way: a large number of chess fans and autograph hunters sought close contact to the legendary World Champion, who officiated the opening of the PMB National Age Group Championship 2014, and took time to discuss a variety of topics with an expert on aesthetics-recognition technology in chess, our author Dr Azlan Iqbal – who sent us a big pictorial report.

Topics Gender , Azlan Iqbal

Dr. Azlan Iqbal has a Ph.D. in artificial intelligence from the University of Malaya and is a senior lecturer at Universiti Tenaga Nasional, Malaysia, where he has worked since 2002. His research interests include computational aesthetics and computational creativity in games. He is a regular contributor at ChessBase News.
Feedback and mail to our news service Please use this account if you want to contribute to or comment on our news page service



Discuss

Rules for reader comments

 
 

Not registered yet? Register

Adi2010 Adi2010 2/26/2016 04:17
What a waste of space!
gmwdim gmwdim 2/26/2016 04:56
Isn't beauty/aesthetics subjective? How do you even define "beauty" in chess, let alone compare it across genders?
TMMM TMMM 2/26/2016 05:15
Oh, this is the same guy from the "Switch-Side Chain Chess"... Good thing I didn't waste my time reading the article then.
mrstillwater mrstillwater 2/26/2016 05:18
April the 1st isn't for more than a month yet?
timisis timisis 2/26/2016 05:18
A bit of an exercise in futility this one, when the answer to the statement is the obvious one, the more higher rated players. The more accurate "who reaches more beautiful three move mates" is perhaps answered here, but borders on the irrelevant. Call me ignorant (as I only look at chess compositions once a year) but those aesthetics are a world apart from anything any chess player is trying to achieve over the board. Apart from winning, we do appreciate delivering the deep and unexpected blow, and that would really correlate positively with rating, while reaching mate-in-three positions is something best left to weaker players, isn't it? if it were me I would probably try to improve upon previous studies on the quality of play, those for example that judged world champions by letting an engine run for 5 seconds. I feel that "control of the position" has been totally neglected, for example missing a jump from +1.2 to +2 or +5 but never falling back to +0.3 or even -0.3. I would even calibrate an aesthetics engine on such games, I would equate missing the jumps with less beauty, but not with less accuracy.
Queeg Queeg 2/26/2016 05:35
Utter crap.
Aesthetics in chess cannot be found in some random blitz-tactics game finishers.
In order to make something out of his wasted effort, the author might want to investigate the correlation of his 'aesthetics score' and playing strength. This should be trivial with the generated data. I am pretty sure that the result will provide a good argument to debunk his own theory.
PurpleUnicorns PurpleUnicorns 2/26/2016 05:37
There are so many glaring issues with this study. I could start commenting on them but at this point reading this has been waste enough.
Jigz Jigz 2/26/2016 05:45
Intellectual showoff. Most of his articles make no sense.
Mr TambourineMan Mr TambourineMan 2/26/2016 08:45
According to Azlans findings the computers plays more Beautiful chess. That most be the reason for the boom in correspondence chess.... and also not forget the wonderful news that cheating with computers arent a bad thing thou it creates more Beauty! (ironic)
ivan3ivanovich ivan3ivanovich 2/26/2016 10:14
The Roos-Neubacher game should be looked at. The last two moves prior to what's reported here was 29. Rf1-f7 Qe7-d6 30. Rf7-a7! (sacrificing the rook for mate) Qd6-b8 31. Ra7xg7! (sacrificing the rook for a second time).

This was not an "unpleasing" three move combination - it was a brilliant five move combination.

Claiming that this is less "beautiful" than the other game is plain bullshit!

Claiming that you can use a machine to evaluate aesthetics is bullshit!
dhochee dhochee 2/26/2016 10:17
This guy is still getting published? It was silly before, but now that he's introduced gender and reached conclusions like "women have less artistic appreciation of the game", it's just ridiculous. I really should know better than to even start reading, but I can't help but watch the train wreck of illogical conclusions.
ivan3ivanovich ivan3ivanovich 2/26/2016 10:31
The Karpov - Mair game is a simultan by an ex world champion against an unrated amateur that did nothing but move his pieces back and forth aimlessly the entire game.

Using that game for any type of comparison is like drawing conclusions from Mike Tyson beating some guy in a bar brawl.

The selection of games without ANY regard AT ALL for who, where and in what setting they were played makes this entire test useless for any analysis!

The failure to look at the entire game (which the two games presented here is an extremely brilliant example of) makes the test COMPLETELY useless!

This "examination" would be ripped to shreds in any peer review and rightfully so!

On the basis of what I've seen here I seriously question if the author has even played a single game of chess, what's presented here lacks ANY understanding of the game of chess at all.
ivan3ivanovich ivan3ivanovich 2/26/2016 10:42
@Chessbase

Giving this author a platform without even looking at what he claims before publishing it is SHAMEFUL. This piece is totally devoid of any intellectual integrity whatsoever and the analysis presented could be bettered by 99.999% of three year old children.

I urge the webmaster (who I assume plays chess) to have a look at the two games used in the article as "proof" of the conclusion the author claims.

The games are for the first not played by equals (the Karpov simultan), secondly they do not display what the author claims but the EXACT opposite, thirdly the conclusion drawn is complete poppycock and tantamount to fraudulent.

If this website wants to be a website gains any respect AT ALL from the chess-playing community then you should issue an apology and refrain from publishing shit like this in the future.
DropkickIggy DropkickIggy 2/26/2016 11:13
????
Queenslander Queenslander 2/27/2016 12:43
Don't publish articles here unless they are coherent. Don't be blinded by the "PhD" or the subject matter. This piece needed a clear introduction, a brief explanation of the parameters of the research (probably including a short discussion of the notion of beauty/aesthetics), a logical explanation of the methodology and the limitations of the research, and a coherent summary of the discussion. A (rambling) piece "summarising findings" of a research study doesn't make much sense - no wonder many of the other comments here aren't overly complimentary. I review academic articles as part of my day job: I doubt I would have recommended this for publication.
schack schack 2/27/2016 12:59
Your CMS needs fixing. We're over a month out from this article's post date.

For shame.
jackie jackie 2/27/2016 03:11
April 1st!
@ChessBase - this 'research' is flawed in so, so many ways.
Please do not be impressed by a 'PhD' and consider that anything emanating from such may be plausible.
This is so obviously bad I actually suspect a hoax, akin to Sokal.
Apology to readers and retraction please.
And if you wish to publish 'academic' papers of sorts, you'd better get some friendly academic folk on staff to check stuff before you publish.
Speaking as an academic here, this is just laughable.
lajosarpad lajosarpad 2/27/2016 12:38
Some of the main issues which make the study valid/invalid are:

- methodology (statistics)
- validator (mate-in-3 situations)
- logical background

Statistics, as methodology can never be used as proof to general patterns, like "women play more beautiful chess". Statistics will only provide a finite set of data, which can be used as proof if and only if the question is limited to the findings. Example: "did women play more beautiful chess between year x and year y". Therefore, the methodology used is not good-enough for a proof, so it is a bases for assumption at best.

The validator attempts to validate a very subjective term of "beauty" as being quasi-synonymous to "mate". There are several problems with this. The technical problem is that beauty is rarely an absolute concept and most of the cases its value lies inside the open interval of (0, 1), where 0 means "absolutely ugly" and 1 means "absolutely beautiful". A binary domain for a validator results in a game being considered either absolutely beautiful or absolutely ugly. The results of this validator will not be comparable in relative terms, game A is more beautiful like game B and game B is more beautiful like game C will be impossible on this base, even though this kind of comparison is daily routine for us, mortal chess players. The phylosophical problem of the validator is that beauty is a very difficult concept to be converted into numbers. There is no standard for doing so. Some draws are more beautiful like some mates. A forced stalemate, for instance, sounds more interesting than a forced back rank mate. Positional play reaching harmony is sometimes more pleasing to the eyes like the tactical skirmish and vice versa. So the validator did not really catch the concept of beauty.

The logical background is the conclusion that women play more beautiful chess than men, as the compared sets were different in gender. But there was another difference among the genders, mentioned by the author himself as well: playing strength. Apart from my opinion that this paper did not answer its own question, let's assume (reductio ad absurdum) that the conclusion is correct. Even in that case, the author's analyzation lacked a very important ingredient: exclusiveness. If the finding is correct, that is members of Set1 (women) play more beautiful chess than members of Set2 (men), and we intend to draw the conclusion that women play more beautiful chess than men, then we most prove that members of Set1 played more beautiful chess specifically because they are women, that is, we need to find a proof or at least statistics that excludes playing strength from the potential factors. A comparison between women players and male players of the same strength would be useful, just like a comparison between male players as strong as women players in the analyzation and the absolute elite. After that we need to underline the functional dependency by showing why is it probable in the future (besides past findings) that women players will play more beautiful chess.

As a conclusion, the paper does not seem to have the right basis to answer its own question. My modest proposal for the author is to ask himself more analyzable questions, like: "how did accuracy of play changed over time", dividing chess history into historical partitions and comparing them by playing accuracy of different partitions of players (elite, excellent, decent, moderate, modest, beginner), to see the evolution of play. Accuracy could be measured by the average of total evaluation deviation by the player in each game. For instance, if a position having the evaluation of 0.03 and I, as White make a move which changes it to 0.01, then the deviation for the move is 0.03 - 0.01 = 0.02. If we calculate my total deviations and divide it by game number and/or move number, then we get an aggregate value of my inaccuracy.
the vhiz the vhiz 2/27/2016 06:19
A fool's errand...typical 3rd world patriarchy on display..
prail prail 2/27/2016 07:00
"..what we have demonstrated should not be taken too seriously.."

There is no danger of that, chief.

slow news day, chessbase?
KevinC KevinC 2/27/2016 07:23
Beautiful chess, as seemingly defined above, has more to do with having one very strong opponent, and one, who plays badly enough to get slaughtered.

I have been 2200-2300 for about 30 years now. Although I am mostly inactive now, I played many players, who were even in the 2000-2100 ratings, who played very badly. Some played such that the outcome was determined by move 15 (positionally lost), but it may have taken another 40 moves to convert, but others just gave me a way to finish them off early, and in style. That said, it has a lot to do with luck, and how the opposing styles of the two players line up. If you get two tacticians, you are more likely to get "beautiful" fireworks, and a quickly ending game. I was always more of a positional player, so that happened to me less often, but if you gave me the opportunity to beat you quickly, I would happily oblige you. Still, my style made it less likely to have those super short games. I also don't think that I was quite strong enough to play too many incredible brilliancies. I don't think this is a question of being a man, or woman, it is all stylistic, and how much the ratings vary between the opponents.

At the very top (the 2700-club), it is just too hard to beat those guys quickly as the mistakes are just so much more subtle. Although there are exceptions, in many of the stronger women's tournaments, there seems to be a much greater variance from top player to bottom player in skill, and players in the 2350-2550 range are much more likely to make big mistakes than players from 2700-2850. Yes there are lower men's tournaments written about here on chessbase.com, but they are also still a lot stronger than the best women's tournaments, and thus, less likely to have the kind of mistakes necessary for a "beautiful" slaughter.

I agree with the others here: This article, and its premise, are rubbish.
genem genem 2/27/2016 10:09
Dr. Iqbal, Interesting and useful would be a chess game evaluation program that assessed (guesstimated) the Elo rating of both players.
azlan azlan 2/28/2016 12:46
To those having doubts about these experimental results, I would be happy to test PGNs ending in mate between just "expert" female players and between "expert" male players. Of about the same Elo even, if you prefer. Do send them to me (if you can find a sufficient number of them, e.g. >100). This is not to say that weak players cannot execute beautiful combinations on the board as well. You may also send me some of the best chess compositions by women to compare against the best by men. Again, if you can find them, please send them to me. On the other hand, if you doubt the validity of the aesthetics model itself, you really should read and actually understand the IEEE paper linked in the article before commenting further.
brian karen brian karen 2/28/2016 01:17
A rare post on Chessbase.com Unlike the others this was a complete waste of time.
ivan3ivanovich ivan3ivanovich 2/28/2016 03:54
@azlan

In your paper you state: "However, women play the game as well but usually against each other." That statement is false. Women play a vast majority of their games against men. There are more men playing chess therefore women will have more men than other women as opponents. Very few (less than 1%) of tournaments are gender segregated.

You then state: "The second time a sequence
or study is evaluated its score may be slightly
different, much like the second time a human
judge assesses a sequence he may decide to change
his mind slightly. So typically, one cycle is used for
all evaluations."

Your analysis engine is incapable of giving a consistent result when encountering the same position twice so you decide to ignore this flaw and only look at the position once so as not expose these weakness of your software. This is not science! This goes against everything the scientific method requires regarding reproducibility of an experiment.

You then say: "This
means ensuring that the last three moves of the line
played in a game is actually a forced mate-in-3 line
in that position, and not something that occurred
because the winner got lucky or the opponent overlooked
a possible defense. Doing so increases the
likelihood that more thought and skill went into the
actual game and the final winning sequence.

That is bullshit! By choosing a forced line you have ensured that there is no creative thought involved at all in executing the win. The only thing the winning player have to do is to find the winning combination and that has nothing to to with being "beautiful". I will postulate that 99% of rated players or more will find those 3-move wins within 1 minute or less. By limiting yourself to a subset of data that removes all "creativeness" then claiming that finding a simple forced line is a sign of "playing beautiful" without qualifying that assumption in any way at all you have demonstrated that you have no idea of how the decision process involved in chess works or even of what can be considered an achievement by a chess player.

then you go on: "A
forced line is also typically considered more beautiful
than one where the opponent could have defended
longer or escaped checkmate."

More bullshit. Setting up a position so that your opponent is incapable of defending himself can be considered "beautiful" but that is not what you have tested, is it? What you have looked at is what happened AFTER the position was set up and where ONLY the technique is left to secure the win. The "beautiful" part is already over and done with when you start looking at the game.

You next say: "So we decided to run a search
for the terms “(Women)” and “(Men)” in ‘any
field’, which returned the tournaments that were
sensible enough to include those terms in their titles.
We also got a handful of hits with the term,
“girls”. Unfortunately, the term, “boys” returned
nothing as tournament names tend not to feature
that word"

I don't really know if you are thick or simply ignorant but don't you even know that there are no men's tournaments? It's either open for both men and women or for women only. Every chess player that regularly or infrequently plays tournaments know this. But you cannot even be bothered to find out such an important fact?

I highly doubt that you even know how to play chess because if you had you'd know that finding a forced 3 move mate is trivial and has nothing to do with creativity or beauty.

There is NO worth in the experiment you have conducted because it is based on flawed (totally flawed) assumptions about the game and how it is played. The way you have conducted the experiment means that it cannot produce a result that has any bearing AT ALL on the questions that you wanted answered.

Your ignorance (willful or not) about the game of chess is staggering and your failure to acknowledge this is seriously casting doubts on you.
azlan azlan 2/28/2016 04:26
Dear ivan3ivanovich,

Thank you for the long post. It confirms that you really haven't bothered to understand my aesthetics model, are uncomfortable for some reason with the results of this particular research and are using your personal experience as supposed scientific evidence to the contrary. In fact, I doubt you have even read the paper. By the way, if the paper does not meet rigorous scientific standards, it won't get published by an IEEE Journal. The same goes for this entry of mine in Encyclopedia Britannica on the term, "Computational Aesthetics".

http://global.britannica.com/topic/computational-aesthetics

They usually ask the real experts in the area to write things like this. However, I appreciate your feedback. If you have anything constructive actually grounded in science to say for a change, I'm all ears. For example, if you have actually conducted any experiments of your own, for instance. No offense.
ivan3ivanovich ivan3ivanovich 2/28/2016 05:06
@azlan

What are your qualifications regarding chess? What's your Fide Elo?
azlan azlan 2/28/2016 05:09
Again, ivan, if you had read the IEEE paper, you would see immediately that my co-authors are quite qualified in that regard.
Ivan Wijetunge Ivan Wijetunge 2/28/2016 07:25
Review here:

http://www.davidsmerdon.com/?p=1822

utter crap seems to be the consensus
azlan azlan 2/28/2016 08:06
Thank you for that, Ivan Wijetunge. It brought a smile to my face. Apart from the typo of Nxh6+ in the slides (which I have corrected), I found David's review to be lacking of any real understanding of my aesthetics model, the scope or constraints of the research. He probably didn't read the IEEE paper either (perhaps he couldn't understand it). In any case, he should try getting that review/rebuttal published in an actual academic platform (e.g. conference paper, journal) if he thinks it has other merits apart from the correction of the typo.
smurfo smurfo 2/28/2016 10:07
@azlan:

I did read your IEEE article; actually, I first read it a couple of years ago when it was published, and I reread it yesterday. I thought it was quite a nice paper to be honest. You'll notice that my rebuttal didn't criticise the Chesthetica software approach at all.

You're right that I have limited understanding of computer science. I do, however, have a good grasp of research methodology, experiment design, and quantitative (particularly econometric) analysis. And, arguably, chess.
azlan azlan 2/28/2016 10:28
@smurfo

Then you should be well-aware of the constraints of the aesthetics model (e.g. why it was designed that way) and also aware of the material in my PhD thesis upon which the IEEE paper expands upon. I'm not sure you have also read my PhD thesis, but if you have, I salute you. With regard to the present paper, the analysis obtained obviously depends on that aesthetics model (which you claim you have no issue with) but rather are objecting to the quality of the data, which as I have said in the paper, was all I could find. However, I have also mentioned earlier in these comments that I am open to better sets of data (e.g. a sufficient number of games between men and between women or between each other with the same Elo) if anyone is able to find them, because I was unable to using the resources available to me. I would be happy to share Chesthetica's analysis of them in an updated paper or article right here on ChessBase.
jackie jackie 2/28/2016 10:28
Better to leave the ad hominems Azlan, you are making a fool of yourself now, increasingly so. You'd be better to have ChessBase delete this, and you should retract this from wherever you have submitted it to. "He couldn't understand" - really? In fact, it did seem that Mr Smerdon understood all too well. A robust and convincing rebuttal - literate, organised, informed, and ... polite. And your emperor is left naked, and embarrassed. We all see that, seemingly except you.
Instead, better to address the issues raised by Mr Smerdon about your 'research'. Address those one by one, and you may do better here.
Some of us here are academics (too?) and can see straight through the bluster and see simply a piece of terrible research with multiple fundamental shortcomings. I saw several too, like Mr Smerdon:
1. You use mate-in-3 as a proxy for aesthetic sense in chess. That's a shocker, and destroys your 'research' before it even starts. Really? Why? To offer an analogy, this might be like judging the beauty of football as a whole, based on watching videos of goals slotted home from within the 6-yard box. But...
2. ...as Mr Smerdon notes, and if you were a chess player, you would realise. Most GM games do not end in mate. Indeed relatively few at even international level (say 2200+), even at decent club level. So, you'd just be judging games of Sunday league pub football in the local muddy park. Research stops before it starts.
3. Again, as Mr Smerdon notes, 'age and level are irrelevant' state you. Far from it.

Mr Smerdon notes many other points, all of which convince.

Finally, in terms of you seeking context, if that's your point, a grotesque issue you raise about unexpected people who play chess? Really? PlayMate Bunnies? This is terrible writing, and simply inappropriate in any setting. Who'd have thought women could play? Who'd have thought women who some may consider attractive could play? Yes, some women write books too. Others do science. Quite remarkable. It's amazing how far they have come and what those odd creatures are capable of. Women? Wow, some can think, indeed. That is a distressingly unenlightened section - you simply embarrass yourself.
I initially took this piece to be a kind of Sokal wind-up - an offbeat academic joke. But noting that you are serious (are you?), I am disturbed that you are offering this up as research, and I am perturbed that they award doctorates in some places to persons who so lack elementary knowledge of research methodology.
I do plenty of academic reviewing - this gets a straight 'No - fundamental flaws.' Regarding your response earlier that IEEE would not publish anything rubbish. Well, disproved herein, eh. They clearly have done. To those outside academia, I explain. There is a surplus of organisations seeking to publish, and a huge number of publications in all fields and sub-fields. There is little quality control at the lower levels, reviewers are often not too critical, and editors just want pages filled. Content is necessary, so when things are submitted, they are accepted. Presentations at conferences are the same - I've seen a few shockers.
Research is welcomed into chess. But it needs to be robustly conducted by real researchers who know what they are doing. What's written here is just really, really bad research. 'Nuff said.
ivan3ivanovich ivan3ivanovich 2/28/2016 01:10
@Ivan Wijetunge

A very well thought out and well worded review of this article, thank you!

There's a very tiny thing that could be a logical flaw in your reasoning that I'd like to point out though. You say: "research has shown that women play differently against men than they do against other women."

From what I understand the research you are referring to showed that women achieved a different result when playing against other women as opposed to men but that doesn't necessarily mean that it was the women that changed their way of playing, it could just as easily be that men change their way of playing when having a woman as opponent. Both of those scenarios could generate the same result.

Other than that little "thingy" I agree with every word you said.
azlan azlan 2/28/2016 01:22
I think I should point out something else which should be obvious, but maybe isn't given the comments here. It really doesn't matter what the strength or age of the players are when evaluating aesthetics of play as long as there is no intended bias in either sample (i.e. men or women), which was the case in this research. The average score is used, after all. Same with using the last three moves as the scope. If both samples are the same, the comparison is valid between men and women. However, like I've said earlier, if some people really want to know if 2500 Elo men play any more aesthetically than 2500 Elo women under the same tournament conditions, I'd be happy to analyze that as well (but send me the PGNs). If you're looking for an aesthetics score for an entire game, you may need to wait another 50 years until someone like me develops a model like that that can also be experimentally-validated.
ivan3ivanovich ivan3ivanovich 2/28/2016 01:27
@azlan

I asked what YOUR qualifications where regarding chess and how it is played. Would you care to answer that question?

Your entire research is based on assumptions that have no basis in reality and every chess player can spot the flaws and several have pointed them out to you here and in other places. Disregarding all critique from people with knowledge of the subject matter and resorting to ad hominem attacks to try and deflect their criticism is not a way usually considered very effective in defending a point.

You sir, are a fool and you deserve to be told so to your face!
azlan azlan 2/28/2016 01:32
Not every chess player, ivan3ivanovich. I worked with quite a few in developing the model. You sound agitated. Perhaps you need a nap.
ivan3ivanovich ivan3ivanovich 2/28/2016 01:42
@azlan

"It really doesn't matter what the strength or age of the players are when evaluating aesthetics of play as long as there is no intended bias in either sample (i.e. men or women), which was the case in this research. The average score is used, after all. Same with using the last three moves as the scope."

Are you really incapable of seeing the flaw in this reasoning?

What matters are if you are comparing like with like and having NO control over who is playing who and NOT knowing or correcting for any present biases between the two groups means that your result is meaningless.

You haven't even bothered to find out IF there are differences in the rating difference between white and black and IF that difference is the same for the two groups (women and men). You simply DO NOT KNOW if the difference in playing strength between winner and loser is equal in the two groups that you are comparing.

You have as a selection criteria tournaments for women and tournaments for men (something that do not exist) but you include games where a world champion plays a simultan against an amateur but only proper tournament for women (because that is all you find with the label women). Do you not realize that you are introducing a bias here?

Your claim that 3 move mates are a valid criteria for deciding levels of beatifulness is plain nonsense! There is NO basis in reality for this claim!
ivan3ivanovich ivan3ivanovich 2/28/2016 01:44
@azlan

You are STILL not answering my question.

What are your chess qualifications?

Are you afraid or revealing your incompetence?
azlan azlan 2/28/2016 01:49
ivan3ivanovich, science really isn't your forte. Also, if you haven't got the message by now and I have to spell it out for you, my official chess rating (which I have never bothered to obtain), is irrelevant to my research work because I work with people who do have the necessary expertise. In fact, they are probably better and more experienced at chess than you.