Chesthetica Composes Longer Mates!

Preamble

There has been some criticism of the repeated publication of articles by Professor Azlan Iqbal, mainly voiced by problem chess experts. Take for example the following message we received from Marjan Kovačević, chess journalist of the daily Politika Belgrade and grandmaster of solving and composing chess problems:

Each time I see an article about Chesstetics, I get very annoyed and wonder what would be the reaction of eminent problemists. Please, do consult them! Have you ever got an objective evaluation of these "creations"? They have very little in common with the theory of problem chess, with it's standards of beauty and quality. In reality, they are 170 years behind the real development of chess composition, not to talk about recent computer findings... Please, don't loose your faithful readers!

To this I replied:

Prof. Iqbal's work is an interesting and important development in the science of Artificial Intelligence. Your criticism reminds me of the time when people kept telling us how insulting it was to keep saying that computers could play chess – any GM would tell you they understood nothing about the game and played like patzers. But it was the beginning of a development that led to a landmark development in AI history.

Azlan Iqbal is developing new facets of machine intelligence, experimenting with images and chess aesthetic recognition. I think it is shortsighted to discard the results his computer generates, calling them inferior or worthless. It is shortsighted to only compare the constructs with the creme of human problem composition – like comparing the games of Fritz 1.0 to those of the top ten players in the world. It is the process that is interesting, and it may well some day lead to startling results – like chess engines today playing at 3500 level. Grandmasters no longer look down on them or call them patzers.

Marjan wrote back:

There may be a problem of using the term "chess problems" here. We are talking about the field as old as the first chess manuscripts. Even the mansuba's from those times, 1.300 years ago, contained more elements of beauty, surprise and logic than the diagrams posted on YouTube.

I'm very much aware of how computers today create chess problems with much deeper contents than Prof Iqbal, who uses simplest effects, like a single minor-promotion, as a model of surprise. People like Prof Torsten Linss, from Dresden, make programs to create series of promotions + many other sophisticated ideas: surprising long moves, switchbacks, complete circles, flight-giving moves, etc. All these are elements of chess beauty, derived from the long history of chess composition.

Most of Prof Iqbal's creations (only the best) have already been published by some human beings, and in much improved settings. Naturally, I'm not talking about plagiarism, but it is deeply insulting for any chess composer to see much inferior versions of old problems, published as something new. Talking about numbers, Christian Poisson from France published several thousand chess problems extracted from computer bases. There are other composers extracting beautiful chess secrets from the computer bases: Viktor Zheglov (Russia), Viktoras Paliulionis (Litvania), etc. In the field of endgame-studies, there are tens of composers using computer-bases to extract positions with mutual zugzwang and other complex ideas.

The point is, Marjan, that Azlan's program works from scratch, using computer intelligence (and nothing else) to compose chess problems. And he is explaining and showing us how it composes. The following video clip is the most recent footage of Chesthetica at work, with its real-time code viewer open.

One cannot help feeling a sense of awe, watching a computer program coming up with ideas, checking and modifying them, discarding flawed positions, and trying to use (human) aesthetics to extract interesting and worthwhile positions. If it comes up, independently, with ideas that have already been exploited by human composers, so much the better. That shows us that we are dealing with a genuine AI. It is like watching a child playing chess. The results may be inferior to professional players, but we can assume that this will not be the case in the future.

So we continue to publish Azlan Iqbal's articles, grateful that we can document his work in a field that has potentially enormous repercussions in other areas of human endeavour. Read today's article for some indications of what the future holds in store.

Frederic Friedel

Chesthetica Composes Longer Mates!

By Azlan Iqbal, Ph.D.

To those who have been following my work as featured here on ChessBase, Chesthetica and my YouTube channel where I showcase its chess compositions will not come as a surprise or something new. However, if this is the first time you are hearing about this prototype computer program that composes original chess constructs (a type of chess problem), perhaps you would first like to read some of my previous articles on the subject. These are given at the bottom of the report. Regardless of your present predisposition toward my work, I am here to announce that Chesthetica now not only composes three-movers but also longer mates, i.e. typically 4 and 5-movers. Examples of these are included toward the end of article, if you prefer to skip right there for now; and if you do not like the music, just mute it.

The program uses the same Digital Synaptic Neural Substrate (DSNS) technology to spark its creativity in composing the longer mates and there will be more of these featured at my YouTube channel from now on. As I have mentioned before, I upload the ones that I find interesting, educational or possessing some aesthetic merit. Unlike traditional chess problems, constructs may be less ‘spectacular’ but more palatable to a wider audience and players of all levels. This is why the hundreds of problems featured at my channel can be used as training puzzles by anyone, including kids in chess clubs and individuals simply interested in the game and its beauty.

The videos are relatively low in bandwidth requirement, short, easy to follow and available anywhere in the world that has access to the Internet. They are also completely free so please take full advantage of this content by visiting the channel and subscribing. This whole YouTube endeavour certainly costs me (and Google) some time, effort and money. But for now, I think it is a worthwhile investment. Besides, studies have shown that chess lowers your risk of Alzheimer’s disease and boosts brain function in many ways, so a chess puzzle might be a good way to start and end the day, especially if you do not have much time to play full games. If you happen to like what you see do ‘like’ and share with your friends as well. Constructive comments that might lead to improvements are also always welcome. So are comments by those who simply wish to speak their mind. Scientists are no strangers to criticism.

How my YouTube channel looks these days

With these longer mates, Chesthetica therefore now creates more problems than before, because each position it sets up may be a three, four or even five-move checkmate, in contrast to just three-movers previously. Even longer mates (six moves and beyond) are also possible, but I suspect these are too complicated and unappealing to most players. Regardless, I am still, however, publishing at a rate of about two problems per day to keep my viewers from being overwhelmed and losing interest. While I try to ensure what I publish is not too bland, I do try to also include simpler compositions for weaker players and those just learning the game. Put simply, if the problems are way too difficult or complicated, relatively few people will be interested or find them useful. Collectively, there are enough tactics and lessons in these problems to be of benefit to players of all levels; even experts, I would imagine. Should you encounter similar piece structures or ‘chunks’ in your real games, you will know how to win, and somewhat amazingly at that. You could also play toward the formation of such structures, which your opponent will likely not suspect.

As I have mentioned previously, Chesthetica composes chess problems, including the longer mates, entirely on its own and from scratch, so to speak. It does not use endgame ‘tablebases’ or any of the traditional artificial intelligence (AI) approaches that typically focus on positions with relatively few pieces. It does not try to ‘improve upon’ or modify existing chess problems. It also does not apply too much composing knowledge that might stifle its creativity. These approaches are not computationally creative and rather constraining, in my view. This is the same reason I do not apply too many composition convention filters in any single composing session and have given the program some restricted ability in being able to bypass certain user-defined settings if it deems fit. I have found that the program functions more productively and creatively this way. If you apply too many restrictions and conditions to an AI program – even if you think it helps create something more akin to what you are looking for – you inadvertently cut off huge portions of the game-space from the get-go. Similarly, given the very cautious way top players (have to) play chess today, it is quite possible that many very interesting positions, e.g. with highly unbalanced material or balanced material with mismatched piece types, will almost never have the chance to arise.

The most recent version of Chesthetica’s interface ...

... showing the option to compose longer mates

I really have no idea in advance what the chess problems Chesthetica creates will look like, how many pieces will be used, how many moves long it will be or how much time one might take to create. If the option is selected, Chesthetica also chooses the variation that it thinks is the most beautiful as the main line based on its internal aesthetics model. I certainly could not go through all the possible variations for every chess problem myself. Some have thousands upon thousands of variations. In any case, I add and subtract absolutely nothing to what is finally produced by the program. If it does not satisfy my personal taste, I reject it. Otherwise, I collect, process as a video, and later upload it to my channel for everyone to enjoy.

The outputs of the program, for all practical purposes, are one-directional. This means they cannot realistically be traced back and reproduced precisely because with the DSNS, there are many elements that come into play at different times and many different ways the program might deal with them. Not to mention that after running for say, two hours, the program behaves differently than when it just started; and each time it starts, it is very likely a little different from the last time it started. Not necessarily better or worse… just different. So given just a particular composition and knowledge of the composing process, one cannot say for certain what set of inputs and outputs led to it. This is to be expected in a sound approach to computational creativity, or the program is little more than a calculator.

Very rarely, I have to admit, Chesthetica does exhibit some rather unusual behaviour. For example, the identical version of the program can run quite differently on different computers even though they have the same operating system and system type (32/64 bit). It reached a point once where I just knew that the old notebook computer in my office was a better composer (i.e. more prolific in its output) than my office desktop or home notebook. Recently, there was a major bug I unwittingly introduced that prevented all nine instances of Chesthetica I had running on my four computers from composing anything (just three-movers at the time) for a week straight, which was strange even for Chesthetica. For a moment I thought my baby had finally run out of steam and was already taking solace in all that it had managed to create and had been uploaded thus far. Then I realized that in the tenth instance, i.e. an account on my home notebook, the same version was able to bypass that bug somehow and compose as normal, which in programming code seemed to me impossible. In fact, it was composing quite efficiently despite the bug.

An old yet prolific composer running two instances of Chesthetica on different user accounts

This made it even more difficult to ascertain that there was a problem in the first place. You see, as programs become more complex and autonomous, they also become more difficult to debug. Presumably, as had happened once before, there must again have been the perfect confluence of hardware, software and environmental glitches that allowed that particular instance of the program to ‘malfunction’ at that time. I have also noticed occasionally for quite some time a kind of ‘sudden death syndrome’, where the composing process may just stop at any time by itself without the program actually freezing or malfunctioning, whereas most of the time the program can run for weeks or months until I stop it manually. No error message or anything to indicate something went wrong is evident. It is like the program just gave up, and it baffles me to this day. In any case, as long as new and interesting compositions that I like can be generated frequently most of the time, I am happy.

I am also working on getting Chesthetica to compose studies, which are basically positions with longer solutions that usually do not end in mate but are nonetheless rather decisive. These are usually preferred by somewhat stronger players. Actually, Chesthetica can already compose studies with the DSNS technology but she is just getting the hang of it, and there is little to choose from at the moment. These positions seem to be a little more difficult to compose automatically, however. ‘Studies’ is short for ‘endgame studies’ but it seems to me that even a position with many pieces on the board can be considered a study if there is a particular sequence of moves in the position that leads to a rather decisive material (or in some cases, positional) advantage. As an AI researcher, it is frankly amazing to be able to create things that can create things. I have never come across exact duplicates and there is practically no limit to how much Chesthetica can create given the theoretical game-space of chess, most of which has remained uncharted over the last thousand years and probably contains many new ideas yet to be discovered by man or machine, even if only with regard to forced mate and won study positions.

A rough estimate of what is likely reachable in chess given some different approaches

Using the DSNS, Chesthetica is able to cut through and leapfrog to meaningful areas of the chess game-space that would otherwise be unreachable using conventional or ‘hill-climbing’ means, such as recorded real games, convention-laden human compositions and humungous endgame tablebases. Might Chesthetica therefore be the best ‘female’ composer around? Certainly among the most prolific, I would say. In the long-term, I am even considering the idea of giving Chesthetica the ability to communicate with other instances of itself (machine learning) and to roam the Web freely to gather raw materials such as images, music and other chess games that it can use to fuel the DSNS process. This will increase its autonomy and may result in even higher quality output. It may even be necessary when applied to far more complex problems in medicine, for instance. Like humans, computers work better together. With a thousand running instances of Chesthetica and given some time, who knows what wonders and gems it might suddenly produce?

There is another benefit to having problems of the type composed by Chesthetica. Computers tend to think differently than humans (or rather, are not as constrained in their thinking as humans) therefore the discovery of new ideas in our view is more likely to emerge from what computers create. Furthermore, while we have known for some time that computers can also play ‘god-like’ chess (e.g. forced checkmate in 250 moves), such positions are beyond the capability of any human and of little practical use to us. A typical human player is far more likely to be able to see and benefit from tactics 3-5 moves long, and of course, some exposure to studies.

I will conclude this article with a few other updates. The DSNS article that essentially explains how Chesthetica does what it does is finally online. You can download it from ResearchGate or via arXiv. We tried to get it published in a few high-impact journals but the paper was just too long and we did not want to shorten it (reduce its size by about half, in fact, is what they said) lest the contents become more difficult to understand. I suppose people do not like to read as much as they used to. Regardless, we are still trying to get it published perhaps as a book chapter or in a science magazine that does not have an issue with its size or a copy having been uploaded to ResearchGate or arXiv, which is not uncommon for academics today to do.

Readers might also be interested to know that some of Chesthetica’s compositions were featured in the July 2015 issue of Problemskak, the Danish chess problem magazine. These are originals that I should upload to my channel only after they first appear in the magazine. Do grab a copy, if you can. If you are the editor of a science, chess or chess problem magazine or journal and would like to do a feature on the DSNS, Chesthetica or some of her compositions, just send me an e-mail. I am sure we can work something out. Chesthetica, by the way, now also has her own Facebook page which I hope you will ‘like’. Social networking, ladies and gentlemen; it is a necessary part of life and business these days, I’m afraid.

Another aspect of my research work, i.e. Switch-Side Chain-Chess also has an update. I was able to secure a modest commercialization-oriented research grant to develop an Android version of the game (for smartphones and tablets). This came as something of a surprise even to me given that it was a highly-competitive grant with only a 15% success rate. Yes, academics these days are expected to generate product revenue, bring in millions in consultancy, etc., in addition to what they have always been doing. It is a job that never gets boring and universities these days really have no need, for example, to make the terms of sabbatical leave less attractive than before. The budget we requested, as is often the case with AI research, was slashed by about a third but I believe is still sufficient to complete the project after we make some adjustments (or I simply would not have accepted the grant offer, which I have done in the past). The project is 18 months and has essentially just started, so keep a lookout for this new chess variant (and its improved AI) within the next year or so.

Finally, with regard to the larger grant we requested about applying the DSNS technology in the field of protein folding (and consequently drug design), there is still no final decision yet. Then again, when it comes to publications and research grants in academia these days, no news is usually good news. Anyway, I hope you enjoy the longer chess problems as they become available, especially if three-movers are too simple for you (there are plenty of those to choose from as well). Note that the earlier problems at my channel are simpler in their presentation as the format evolves somewhat with time. Feel free to share your thoughts in the comments section of the videos, and at the bottom of this article, of course.

Sample four and five-movers

Complete Playlists

• Mate in 3 Moves Collection
• Mate in 4 Moves Collection
• Mate in 5 Moves Collection

Previous articles by the author

• 5/31/2015 – Celebrating 300 machine generated problems
  As we reported before, Chesthetica, a program by Azlan Iqbal, is autonomously generating mate in three problems by the hundreds, and the author is posting his selections in a very pleasing format on YouTube. The technology behind the program’s creativity is a new AI approach and Dr. Iqbal is looking for a substantial research grant for applications in other fields.

• 4/7/2015 – Switch-Side Chain-Chess Revisited
  The search continues for a chess variant which retains the flavour of the original game but does not succumb to the brute calculating power of modern computers. AI researcher Azlan Iqbal has proposed his own unique variant. Now he provides some test games and shows how Carlsen could have won (instead of lost) WCCh Game 3 against Anand in Sochi had Switch-Side rules applied.

• 2/6/2015 – Computer generated chess problems for everyone
  Now they are composing problems that fulfil basic aesthetic criteria! Chesthetica, a program written by Azlan Iqbal, is churning out mate in three constructs by the hundreds, and the author is posting them in a very pleasing format on Youtube. How long will Chesthetica theoretically be able to generate new three-movers? Quite possibly for tens of thousands of years.

• 11/7/2014 – A machine that composes chess problems
  Chess problems are an art – positions and solutions, pleasing to the mind and satisfying high aesthetic standards. Only humans can compose real chess problems; computers will never understand true beauty. Really? Dr Azlan Iqbal, an expert on automatic aesthetic evaluation, imbued his software with enough creativity to generate problems indefinitely. The results are quite startling.

• 7/26/2014 – Best ‘Chess Constructs’ by ChessBase readers
  Chess constructs are basically an intermediate form of composition or chess problem, lying somewhere between brilliancies from chess history – and artistic chess problems, between real game sequences and traditional award-winning compositions. A month ago Dr Azlan Iqbal explained the concept asked our readers to submit compositions of their own. Here are the winners.

• 6/29/2014 – Azlan Iqbal: Introducing ‘Chess Constructs’
  People love brilliancies from chess history – and artistic chess problems. But there is a big gap between the two. Positions from games demonstrate the natural beauty of actual play, while chess problems are highly technical, with little practical relevance. The author of this interesting article suggest an intermediate form, one you can try your hand at – and win a prize in the process.

• 9/2/2009 – Can computers be made to appreciate beauty?
  Or at least to identify and retrieve positions that human beings consider beautiful? While computers may be able to play at top GM level, they are not able to tell a beautiful combination from a bland one. This has left a research gap which Dr Mohammed Azlan Mohamed Iqbal, working at Universiti Tenaga Nasional, Malaysia, has tried to close. Here's his delightfully interesting PhD thesis.

• 12/15/2012 – A computer program to identify beauty in problems and studies
  Computers today can play chess at the grandmaster level, but cannot tell a beautiful combination from a bland one. In this research, which has been on-going for seven years, the authors of this remarkable article show that a computer can indeed be programmed to recognize and evaluate beauty or aesthetics, at least in three-move mate problems and more recently endgame studies. Fascinating.

• 2/2/2014 – A new, challenging chess variant
  Ever since desktop computers can play at its highest levels and beat practically all humans, the interest of the Artificial Intelligence community in this game has been sagging. That concerns Dr Azlan Iqbal, a senior lecturer with a PhD in AI, who has created a variant of the game that is designed to rekindle the interest of computer scientists – and be enjoyable to humans as well: Switch-Side Chain-Chess.

• 5/11/2014 – Kasparov in Malaysia
  He was mobbed, but in a good way: a large number of chess fans and autograph hunters sought close contact to the legendary World Champion, who officiated the opening of the PMB National Age Group Championship 2014, and took time to discuss a variety of topics with an expert on aesthetics-recognition technology in chess, our author Dr Azlan Iqbal – who sent us a big pictorial report.