Celebrating 300 machine generated problems

Celebrating 300 Chesthetica-Generated Chess Problems

By Azlan Iqbal, Ph.D.

Previously on ChessBase, I brought to public attention my YouTube channel which features original chess problems or constructs created by my prototype computer program, Chesthetica. Yes, you read that correctly… created by Chesthetica, not simply “using” it. I have been creating videos of them and uploading to the channel since late December 2014 (Christmas Eve, incidentally) at a consistent rate of two chess problems every single day. Fortunately, I also have a master’s degree in multimedia systems where I got even more familiar with these things so creating the videos is not too difficult for me and I can practically mass-produce them. These short videos and their presentation style represent a new format in which chess problems or constructs can be created for a wider audience. Imagine not always having to carry a particular book of chess problems around but instead just your smartphone. Imagine also not having to move any pieces about but instead just sit back and let your mind do the work, or simply enjoy the sow.

This article is in celebration of 300 problems created by Chesthetica; by the time you read this there are probably more already. Three hundred is not a small number of creative objects (try counting to 300 using your fingers) and each one was picked by me based on what I considered interesting, educational or having some aesthetic merit at the time. It is quite possible I have dismissed problems that others would have found interesting but somebody has to make these decisions. The problems or constructs generated are generally not as difficult as traditional chess problems, which makes them suitable for training purposes as well. They are positions that reflect what could reasonably happen in actual games. Some of these might otherwise take decades or centuries to occur between players over the board, assuming the players even recorded the games and decided to share them.  Consider it a free service to the chess-playing community. I am not sure if any new strategic or tactical ideas have been introduced to the chess-playing community through these problems but since I am neither a master composer nor master player – though I have worked with quite a few – I would be interested to know if they have.

From an artificial intelligence (AI) standpoint, however, I can indeed comment with a bit more confidence. Chesthetica uses the digital synaptic neural substrate or DSNS approach (patent pending) for its ‘creative spark’ combined with what can be described as probabilistic reasoning algorithms to compose original problems or constructs. So occasionally, it can even bypass certain filters set by the user if it sees fit. This creative process, as suspected, seems to be inexhaustible; at least for the time being. Especially since I can feed it with more photographs and tournament games as raw materials to draw ‘inspiration’ from. Thanks to a relatively small internal research grant I recently obtained, Chesthetica is now able to extract the necessary attributes from these domains automatically in the DSNS process.

As an update, the research paper on the DSNS that I talked about before still requires further revision to convince the ever-sceptical reviewers and the research proposal investigating its application in protein folding has passed the fourth stage (oral defense) recently. The funding we have requested for that is considerably larger and I am now just awaiting the final decision. This project will be a serious effort in the hope of improving the critical defense mechanisms of the body and discovering through computational creativity what might actually fix them should they fail for any reason, even in slim people. It is not just another medical research project in the class of something that say, tries to ‘link’ a disease to tasty foods, waistlines or a lack of exercise, and reminding everyone yet again of this worldwide willpower crisis. There are enough researchers spending precious research dollars on that sort of thing and an ‘outsider’ like me need not try to contribute in the same or similar way, even though it would be much easier to accomplish than what we have planned. Should our proposal ultimately be rejected for whatever reason, the aforementioned work will be dead in the water for another year, at least. Regardless, we will keep trying for as long as we can and in the meantime try to get the DSNS approach published so that others with more luck with large grants may attempt to use it in their domains. Just getting a paper published in a prestigious journal can take over a year, not including the time it took to actually do the research that went into that paper.

Anyway, given the ‘festive’ nature of this article, I thought I would share an interesting story with my readers. As I discovered only much later by chance, the very first composition created by Chesthetica (v9.32 at the time) using the DSNS approach that I decided to upload should not have been possible because I had explicitly created an ‘exclusions file’ to rule out what I thought would be simple and uninteresting endings such as KRk and KRPk. This was not one of the filters that the program was programmed to be able to bypass. I have combed through the code of that version of Chesthetica and to this day it remains the only unexplained bug in my error log. There is absolutely no way around the exclusions file that I could see. In any case, I am glad Chesthetica did somehow bypass the filter at precisely the right time in its composing process because it appears that not all compositions with ‘simple endings’ are dull. Nothing like this, to my knowledge, has happened since and the bug seems to have resolved itself. 

White to play and mate in three

The thumbnail for this article (and also composition 00260 below), by the way, is called ‘Personification of Chesthetica’ and was commissioned under a previous research project. It was sketched by my former research assistant from Iran who was not a professional artist yet willing to do the job. She included in the sketch, at my request, a curious chess position on the board from one of my games a very long time ago, if you can make it out. Her husband wrote Chesthetica Endgame under my supervision. I use this artwork and the original Chesthetica poster he helped me design (shown in the YouTube thumbnail for composition 00017 above) for some of the chess problems.

New compositions are constantly being generated by nine different instances of Chesthetica that I have running 24 hours a day on several machines. Chesthetica is a relatively complex or complicated computer program. It has parts written in Visual Basic 6, ANSI C, Delphi and Python. For such programs where running more than one instance might interfere with the other running instances, one can always run each instance of the program on a separate user account, at least in Windows. It is difficult to say when, exactly, any particular instance of Chesthetica will successfully compose a problem. Sometimes an instance of the program can create 12 compositions in an hour. Other times, two weeks will pass with nothing generated. Incidentally, I managed to capture a rare instance of Chesthetica generating a composition in real time as shown below.

All the generated problems are uploaded automatically to a central FTP server where I collect and review them perhaps once a week or once fortnightly. From these, I upload the ones I find interesting in the order they were composed. The precise date and time a composition was generated is given in the description section of the video. Rarely, I am unable to maintain that order. For instance, a chess problem magazine will be featuring a selection of these compositions soon and I was told they need to appear there first before they can appear on YouTube, which is fine and will be explained in the description section for those problems at my channel or simply put into a separate playlist.

In the near future, I hope to improve Chesthetica further so it is not limited to three-movers but can create longer problems and studies as well. The technology is already there, and it is just the coding that is required. Good programmers can be hard to find. A notable recent improvement, starting with v9.58, is that Chesthetica can choose the most aesthetic winning line based on its internal aesthetics model (what my PhD was about). My colleague and research partner, Matej Guid, actually suggested this to me. So if, for example, promoting to a rook instead of a queen suffices, it might just choose to show that one as the main line in the solution. Even humans seldom do this in real games, even though promotion to a queen is not compulsory and often just overkill.

In the meantime, after 300 chess compositions, I would like to present a selection of the ones that, personally, stood out somewhat. Of course, not everyone will agree they are interesting but this is true with just about every creative domain. Even presented with the most praiseworthy compositions by the best human composers the world has ever known, I would estimate that any one person would perhaps really like or truly appreciate only around 10-15%. This is not to say the other 85-90% are not worthy, except that we all have our personal tastes and biases. Bear in mind also that these compositions were all created by mindless, soulless, godless machines and there is something to be said about that. Regardless, if you do not find anything interesting here, do take your time to check out the other compositions at my channel. Feel free to leave comments and suggestions there, share with your friends and do subscribe for channel updates.

Previous articles by the author

• 4/7/2015 – Switch-Side Chain-Chess Revisited
  The search continues for a chess variant which retains the flavour of the original game but does not succumb to the brute calculating power of modern computers. AI researcher Azlan Iqbal has proposed his own unique variant. Now he provides some test games and shows how Carlsen could have won (instead of lost) WCCh Game 3 against Anand in Sochi had Switch-Side rules applied.

• 2/6/2015 – Computer generated chess problems for everyone
  Now they are composing problems that fulfil basic aesthetic criteria! Chesthetica, a program written by Azlan Iqbal, is churning out mate in three constructs by the hundreds, and the author is posting them in a very pleasing format on Youtube. How long will Chesthetica theoretically be able to generate new three-movers? Quite possibly for tens of thousands of years.

• 11/7/2014 – A machine that composes chess problems
  Chess problems are an art – positions and solutions, pleasing to the mind and satisfying high aesthetic standards. Only humans can compose real chess problems; computers will never understand true beauty. Really? Dr Azlan Iqbal, an expert on automatic aesthetic evaluation, imbued his software with enough creativity to generate problems indefinitely. The results are quite startling.

• 7/26/2014 – Best ‘Chess Constructs’ by ChessBase readers
  Chess constructs are basically an intermediate form of composition or chess problem, lying somewhere between brilliancies from chess history – and artistic chess problems, between real game sequences and traditional award-winning compositions. A month ago Dr Azlan Iqbal explained the concept asked our readers to submit compositions of their own. Here are the winners.

• 6/29/2014 – Azlan Iqbal: Introducing ‘Chess Constructs’
  People love brilliancies from chess history – and artistic chess problems. But there is a big gap between the two. Positions from games demonstrate the natural beauty of actual play, while chess problems are highly technical, with little practical relevance. The author of this interesting article suggest an intermediate form, one you can try your hand at – and win a prize in the process.

• 9/2/2009 – Can computers be made to appreciate beauty?
  Or at least to identify and retrieve positions that human beings consider beautiful? While computers may be able to play at top GM level, they are not able to tell a beautiful combination from a bland one. This has left a research gap which Dr Mohammed Azlan Mohamed Iqbal, working at Universiti Tenaga Nasional, Malaysia, has tried to close. Here's his delightfully interesting PhD thesis.

• 12/15/2012 – A computer program to identify beauty in problems and studies
  Computers today can play chess at the grandmaster level, but cannot tell a beautiful combination from a bland one. In this research, which has been on-going for seven years, the authors of this remarkable article show that a computer can indeed be programmed to recognize and evaluate beauty or aesthetics, at least in three-move mate problems and more recently endgame studies. Fascinating.

• 2/2/2014 – A new, challenging chess variant
  Ever since desktop computers can play at its highest levels and beat practically all humans, the interest of the Artificial Intelligence community in this game has been sagging. That concerns Dr Azlan Iqbal, a senior lecturer with a PhD in AI, who has created a variant of the game that is designed to rekindle the interest of computer scientists – and be enjoyable to humans as well: Switch-Side Chain-Chess.

• 5/11/2014 – Kasparov in Malaysia
  He was mobbed, but in a good way: a large number of chess fans and autograph hunters sought close contact to the legendary World Champion, who officiated the opening of the PMB National Age Group Championship 2014, and took time to discuss a variety of topics with an expert on aesthetics-recognition technology in chess, our author Dr Azlan Iqbal – who sent us a big pictorial report.