Azlan Iqbal: Studies and a Decade in Development

by Azlan Iqbal
5/31/2016 – Chesthetica, a computationally creative chess problem composer, has added studies to its repertoire. Dr. Azlan Iqbal shows us examples and looks back upon a decade of its development. He also describes the challenges he faced in trying to introduce the technology in the field of protein folding which might have yielded cures to diabetes, cancer and other deadly diseases. From chess to computers to medicine.

ChessBase 18 - Mega package ChessBase 18 - Mega package

Winning starts with what you know
The new version 18 offers completely new possibilities for chess training and analysis: playing style analysis, search for strategic themes, access to 6 billion Lichess games, player preparation by matching Lichess games, download Chess.com games with built-in API, built-in cloud engine and much more.

More...

Chesthetica: Studies and a Decade in Development

By Azlan Iqbal, Ph.D.

It has been a while since I have written exclusively about Chesthetica, a computer program I started developing at the beginning of my doctoral studies back in 2006. It was designed initially to evaluate aesthetics in chess and I later enhanced it to actually compose chess problems as well. This was achieved using a variety of improving methods and not really using the aesthetics model in the way one might expect. Just as an art critic may be good at critiquing, that particular knowledge or ability does not lend itself easily to actually creating good or interesting art. Perhaps this is partly why beauty contest judges are seldom predominantly beauty queens themselves. Put simply, one may recognize aesthetics and even be able to judge or evaluate it, but being able to create it is quite another matter.

In the last installment, I explained how Chesthetica was able to compose longer mates from originally being able to compose only three-movers. I had intended to update ChessBase readers about Chesthetica composing study-like constructs as well, which has been taking place since late 2015 but several other article ideas came up that got in the way. So as an official update to readers, Chesthetica is indeed able to compose study-like constructs now. It uses the same DSNS approach I have written about before which does not rely on endgame ‘tablebases’ or traditional artificial intelligence (AI) approaches that focus on positions with relatively few pieces. It does not even try to ‘improve upon’ or modify existing chess problems. It composes essentially from scratch and to this day, I am not able to explain why it works. I can only tell you how it works, as I have done earlier on ChessBase and more thoroughly in the DSNS book linked to above.

As I have explained before, the DSNS approach serves as the ‘spark’ of creativity in the machine, and the rest are basically parameters of requirements, e.g. three-mover, four-mover, study. This is why I believe it can be applied in virtually any domain and can almost certainly play a part in the composition of any type of chess problem as well. So the study-like constructs composed are basically puzzles that resemble traditional composed studies in the sense that the solver is invited to figure out the key or first move that leads to the most decisive advantage, i.e. the ‘best’ move. The difference between a study-like construct, puzzle or problem compared to a traditional composed study is that it appears to be more likely reachable in a real game. Compared to a ‘tactical study’ which is basically a position taken from a critical point in a real game (usually between master players), study-like constructs are more economically-designed. In other words, there are not as many superfluous pieces, if any.

The DSNS approach was actually developed to further illustrate the creative abilities of computers and one of the best and most amenable domains to test creativity, in my view, is the composition of chess problems or puzzles. It was also convenient that computational creativity, to my knowledge, had never actually been applied in this domain before, and certainly not using the DSNS approach which was only recently developed. Here is an example of a relatively complex study composed autonomously by Chesthetica.

Chesthetica is also able to compose the rarer “White to Play and Draw” type of study, like the one here.

The complete study collection (in two parts) which is always being updated is available here. Readers are invited to try the problems which are of varying difficulty. I chose them from the much larger ‘raw’ collection of compositions by Chesthetica simply because they looked interesting to me. However, I did not alter them (or the chosen solution) in any way whatsoever. What you see is exactly what Chesthetica created and chose and this, in itself, is a marvel of computational creativity. Why should a computer be able to create even things like this at all? Are they not supposed to be ‘mindless’ automatons doing precisely what they were programmed to do every single time and nothing more? Are they not supposed to ‘repeat themselves’ after a while? Are they not supposed to be predictable? All these notions are clearly being challenged in one go.

Given the ‘success’ of Chesthetica applying the DSNS approach in the domain of chess problem composition, the next logical step was to see if it could be applied to more ‘important’ fields and none commands more respect and urgency than medicine, for obvious reasons. Readers may not find it at all surprising that my parents wanted me to study medicine instead of computer science but I really had no interest in it. To this day, even if I could rewind the clock 20 years, I would still choose to study computer science over medicine. My extended family is full of physicians, by the way. Anyway, I had actually applied for a sizeable research grant to test the DSNS technology in the field of protein folding (as I had earlier written about on ChessBase). This would theoretically have applications in drug design to treat diabetes, cancer and Alzheimer’s, to name a few diseases.

The result of that application came back several weeks ago after 15 months of deliberation and two defense sessions; first with an AI panel of assessors and then transferred to the ‘biotech’ panel which was deemed more appropriate. The answer was no. The project had been rejected and no official reason was given.

A Protein Structure [Huntingtin and Huntington's Disease]

Here is my take on what probably happened. I had two research partners, by the way. A wealthy and independent researcher overseas and a highly talented (easily borderline genius-level) young woman with a Ph.D. in chemical engineering. My associate overseas has been studying protein structures for years and is also a very talented computer programmer working in the finance sector. The young woman, a Malaysian just like me, I found online while looking for suitable candidates to join the project, and was very impressed by her CV. I asked her to join me and she said yes. So I was probably the least competent of the group, yet in charge of the project. To make a long story short, I had virtually no expertise with regard to protein folding so my young associate had to handle that part.

She understood the DSNS approach after I explained it to her and found it fascinating. She too saw the unlimited potential and even necessity in the domain of protein folding. Our relationship developed over the months and she also followed me to the defense sessions, doing her best to explain the worthiness of our project proposal to over half a dozen grey-haired men, all of whom were probably experts in their fields as well. To be honest, the impression I first got during the biotech defense session was that they perhaps did not take her seriously enough for some reason but later I realized that they simply could not wrap their heads around how the DSNS approach would be able to ‘create’ or ‘propose’ new protein structures that might hold the key to curing some of the worst diseases that have plagued humanity for centuries.

This is something we are told the best medical minds have been struggling with to no avail yet here were two ‘nobodies’ (one of whom is not even a ‘biotech person’) proposing they might have a solution. Regardless, I do think they were interested but there were so many technical hurdles (especially on the medical side) with regard to how each proposed protein structure needed to be tested and the timeframe required in order to do so that the 18-month project proposal did not seem feasible. Also, what would be needed is not the paltry 6 figures in funding I was asking for but more like 8 figures and closer to 60 months with no guarantee of a positive result, which was way beyond their scope and risk threshold.

Having worked in AI for many years now, I am used to being able to test and do things relatively quickly but in medicine, it is a whole different ball game, so to speak. Some aspects of medical research are downright ‘primitive’ in comparison and understandably so because biology and silicon are two different things. Two totally different worlds, in fact. Needless to say, I was not surprised by their final decision to reject the project (with no reason given or even an explanation of how to revise or improve the proposal) because there was probably no reason and no way to improve it that would make a difference. The project was simply ahead of its time and not within the purview of what the grant is designed to facilitate. I was glad, however, that with a decision finally made on this project, I am now free to apply for a different proposal (on another topic) if I choose, as two simultaneous applications are not permitted.

My associate overseas was disappointed and begged me not to give up. He had mentioned earlier that protein structures hold the key to immortality and all we needed to do was find the right structures in the 20^200 universe of possibilities. No amount of brute-force computing in finite time could ever suffice in such a vast search space. Creativity (human or machine) was the only way to leapfrog to meaningful or useful areas of that space and I believe we all understood this. Alas, I do not think it is worth trying again on this topic as the same issues are likely to be brought up. Also, my other project member has plans to possibly move overseas so we are essentially disbanded. I have therefore decided to look into applying the DSNS into perhaps general artwork instead.

Demonstrating its efficacy in another domain (apart from chess problems) would help establish the approach as viable enough to be useful to other researchers and perhaps included in AI textbooks someday; not unlike artificial neural networks and genetic algorithms. However, such things can take a lot of time after they are first introduced; decades, even. In any case, I hope to soon document the specifics of what I had in mind in adapting the DSNS approach to protein folding. I doubt any science journal or magazine would even be willing to publish it without experimental data (no grant = no funding = no experiments) so it will either have to be self-published or filed away in one of my drawers for a time. I might even just pass it on to one of my research assistants. It is important, I think, to write these things down quickly because memory fades and inspiration does not seem to have a predictable schedule or even repeat itself.

To be honest, the whole DSNS idea simply came to me years ago over a few days in precise detail which I decided to write down and sketch almost immediately. As the years went by, whenever I sat down in front of my computer trying to solve a particular problem related to the DSNS in code, very often my fingers would seemingly start moving on their own and before I even realized what was really going on, the problem was solved and the program actually compiled and ran without errors, which seldom ever happened working on other programs or other aspects of Chesthetica. In fact, debugging programs is usually a very tedious experience. Only a few days ago in debugging and enhancing Chesthetica from v10.12 to v10.13, I managed to accomplish everything in under an hour, when I had originally suspected would take at least a week of torture and sleepless nights (and therefore put off the upgrade for over a month!).

After a decade of working on Chesthetica for various purposes, it is now stable and functioning quite well in composing three-movers, four-movers, five-movers and studies. It can also evaluate beauty in these problems using the aesthetics model I developed earlier on (for my Ph.D.). So thousands upon thousands of chess problems can be ranked aesthetically and also seemingly endlessly composed from scratch which is something that would require countless man-hours if domain-competent humans were to attempt it. There may be minor enhancements to the program every now and then but for the most part, the AI work (with respect to the aim of the DSNS) appears done.

There was never really any intention (without say, millions of dollars of support from a company like IBM) to have Chesthetica compose only world-class traditional chess problems because perhaps only 1% of all chess players can fully appreciate those whereas 99% of players can probably find something interesting in the constructs Chesthetica creates. Is what they perceive not considered beauty or aesthetics in the game as well? Analogously, are works only by Picasso and Van Gogh worthy of being considered as having ‘creative value’ in the art world? What about all the art artists we have never heard of create that also sell (albeit for not as much)? I think my point should be clear.

I have kept up the initiative to upload a personal selection of Chesthetica’s creations to YouTube even though at an absolute and significant financial loss in terms of the hours and effort involved but I feel it is important work that may pay off one day. If nothing else, there is at least some historical value with regard to the history of computer chess and AI. I will keep doing so for as long as it interests me. Chesthetica could, in theory, go on forever but I cannot. I have even added a link to a periodically-updated PGN of the compositions (in the “About” > “Links” section of the YouTube channel) for those who find the videos too cumbersome or inconvenient and prefer to use a PGN reader of some kind.

You will probably not find any of these positions having occurred in real games or composed by anyone as they are quite literally the products of the ‘mind’ of an AI. Had some of these constructs occurred in actual tournament games between human masters, one can almost imagine Danny King commenting on the key as, “a beautiful move here” with inflection. There are no plans at the moment to make Chesthetica available to the public even though that would be an interesting prospect. I sometimes wonder how it might run on thousands of other machines and what kind of compositions it might be able to create with such a multitude of instances running and thousands of human minds judging its creations.

ChessBase has sort of approached me a couple of times with regard to the potential of applying Chesthetica in certain aspects of its business but nothing materialized. Despite all the publicity via ChessBase, trade exhibitions and others, I have not received any interest from any other commercial software developers either so I suppose it is unlikely to be available, even for purchase, any time soon. This does not bother me as I am not really in need of the money anyway. Overall, the reception from the public has been quite positive, however. People have contacted me wanting to download and even purchase the software and the feedback I have received with regard to the constructs created is generally quite positive as well (80% positive, at least).

So for now let me end this article and the Chesthetica saga by thanking ChessBase and readers for their support all these years. It has been an interesting ride but it is now perhaps time to move on to other pastures. Just as I moved on from computational aesthetics to computational creativity, it is now time to move the DSNS from chess problems to some other domain. I would have liked to apply it in medicine and protein folding but alas, that was not meant to be. Regardless, there are plenty of other areas of research to look into. Life in academia never gets boring as long as the ideas keep coming. It’s what we do.

Previous ChessBase articles on the subject by Prof. Azlan Iqbal

  • 5/31/2015 – Celebrating 300 machine generated problems
    As we reported before, Chesthetica, a program by Azlan Iqbal, is autonomously generating mate in three problems by the hundreds, and the author is posting his selections in a very pleasing format on YouTube. The technology behind the program’s creativity is a new AI approach and Dr. Iqbal is looking for a substantial research grant for applications in other fields.

  • 2/6/2015 – Computer generated chess problems for everyone
    Now they are composing problems that fulfil basic aesthetic criteria! Chesthetica, a program written by Azlan Iqbal, is churning out mate in three constructs by the hundreds, and the author is posting them in a very pleasing format on Youtube. How long will Chesthetica theoretically be able to generate new three-movers? Quite possibly for tens of thousands of years.

  • 11/7/2014 – A machine that composes chess problems
    Chess problems are an art – positions and solutions, pleasing to the mind and satisfying high aesthetic standards. Only humans can compose real chess problems; computers will never understand true beauty. Really? Dr Azlan Iqbal, an expert on automatic aesthetic evaluation, imbued his software with enough creativity to generate problems indefinitely. The results are quite startling.

  • 7/26/2014 – Best ‘Chess Constructs’ by ChessBase readers
    Chess constructs are basically an intermediate form of composition or chess problem, lying somewhere between brilliancies from chess history – and artistic chess problems, between real game sequences and traditional award-winning compositions. A month ago Dr Azlan Iqbal explained the concept asked our readers to submit compositions of their own. Here are the winners.

  • 6/29/2014 – Azlan Iqbal: Introducing ‘Chess Constructs’
    People love brilliancies from chess history – and artistic chess problems. But there is a big gap between the two. Positions from games demonstrate the natural beauty of actual play, while chess problems are highly technical, with little practical relevance. The author of this interesting article suggest an intermediate form, one you can try your hand at – and win a prize in the process.

  • 9/2/2009 – Can computers be made to appreciate beauty?
    Or at least to identify and retrieve positions that human beings consider beautiful? While computers may be able to play at top GM level, they are not able to tell a beautiful combination from a bland one. This has left a research gap which Dr Mohammed Azlan Mohamed Iqbal, working at Universiti Tenaga Nasional, Malaysia, has tried to close. Here's his delightfully interesting PhD thesis.

  • 12/15/2012 – A computer program to identify beauty in problems and studies
    Computers today can play chess at the grandmaster level, but cannot tell a beautiful combination from a bland one. In this research, which has been on-going for seven years, the authors of this remarkable article show that a computer can indeed be programmed to recognize and evaluate beauty or aesthetics, at least in three-move mate problems and more recently endgame studies. Fascinating.

  • 2/2/2014 – A new, challenging chess variant
    Ever since desktop computers can play at its highest levels and beat practically all humans, the interest of the Artificial Intelligence community in this game has been sagging. That concerns Dr Azlan Iqbal, a senior lecturer with a PhD in AI, who has created a variant of the game that is designed to rekindle the interest of computer scientists – and be enjoyable to humans as well: Switch-Side Chain-Chess.


Dr. Azlan Iqbal has a Ph.D. in artificial intelligence from the University of Malaya and is a senior lecturer at Universiti Tenaga Nasional, Malaysia, where he has worked since 2002. His research interests include computational aesthetics and computational creativity in games. He is a regular contributor at ChessBase News.

Discuss

Rules for reader comments

 
 

Not registered yet? Register

BatBlind BatBlind 6/5/2016 11:49
The 1st of April has come around rather quickly it seems...
hserusk hserusk 6/2/2016 06:59
One doesn't feel the love for Azlan here. Why that?
azlan azlan 6/2/2016 02:01
@Jackie: Interesting observation and it's perhaps possible (but unlikely as biotech people don't care as much about gender issues). On the flip side of that argument, however, is that if the DSNS did indeed have the potential to cure or better treat diabetes, cancer, Alzheimer's etc., millions of suffering and dying people (perhaps even some of the ChessBase commentators here some day) will have the commentators here to thank, at least in part, I suppose.
jackie jackie 6/2/2016 01:42
There is one inadvertent but happy advantage of all this.
Iqbal applies for funding. Claims to be a chess / IT expert. Perhaps the research council knows little about chess. May be *initially* impressed by the bluster and sales talk.
Funding applications for large grants are researched thoroughly, and they Google. See here that Iqbal is accorded no respect, that his methodology is suspect, that he writes badly, has a freakish line in misogyny, and that his research is utter bunkum, and duly deny. Hey presto, thanks Chessbase! :)
Yes, these articles are a big BS grey cloud. But the cloud's silver lining is the funding saved in his country, and diverted to researchers with presumably better ideas, more understanding of what they are doing (it is clear the chap here understands little of chess), and more robust methodology.
Right, a chess portal, now back to some chess...
yesenadam yesenadam 6/1/2016 05:43
Groan. Iqbal telling us about 'domain-competent humans', lol. Amazing. The readership of this site have fairly unanimously seen that you are not one, sir. If we don't appreciate your brilliance, the problem must by definition lie with us. You never stop going on about what a genius you are, huh. Maybe you would learn something by studying what kind of people tend to do that. Seriously.
AgainAgain AgainAgain 6/1/2016 02:27
April fools is a regular occurrence now :)
Van_Phanel Van_Phanel 6/1/2016 10:45
You still let this guy publish on your website? Really? Any respect I've ever had for you is gone.
Chasqui Chasqui 6/1/2016 09:15
Pay attention to the static image before the first video starts: could it be that the reason it's taken so long to perfect this algorithm is that the board isn't setup right? I'm not a CS expert, so I may be way off in my evaluation.
TMMM TMMM 6/1/2016 04:40
Regarding the arXiv paper: "Reviewed favorably by several renowned journals but the size of the paper was an insurmountable issue." That's HILARIOUS! A computer science paper of 39 pages, and claiming that after many favorable reviews, it is rejected because of the size??

Just admit that the paper was rejected many times because reviewers saw it for the nonsense it is. Please go back to the field of study that you may be capable of, and leave chess and computer science to people who know what they are doing.
TMMM TMMM 6/1/2016 04:33
The Joker is back!

I think ChessBase knows these articles lack all quality, and is intentionally trolling the readers into reading this crap. Better to have discussions about nonsense than not to have any discussions (viewers) at all, right?
thlai80 thlai80 6/1/2016 04:31
The article is very difficult to read when the author is full of himself, praising himself at every opportunity and boasting association with geniuses, phd families and so on. Here is a man who undoubtedly very proud of himself, little regards of people who he thinks not as smart, and take no delay in using position and ranking to influence an outcome. A dangerous trait that can leads to corruption and abuse of authority.
MrL2014 MrL2014 6/1/2016 03:54
"finding the right structures in a 20^200 universe of possibilities"; this is so funny! The visible universe is estimated to contain between 10^78 and 10^82 atoms. Good luck in finding those right structures.
kayatoast kayatoast 6/1/2016 02:13
I loved this article. Elegant, eloquent, creative and insightful ! My other tastes in reading include this wiki page on foot fungus. https://en.wikipedia.org/wiki/Onychomycosis.
azlan azlan 6/1/2016 12:39
@genem: Yes, many people might assume that. However, the DSNS has absolutely nothing to do with the concept of natural selection. It has also nothing to do with machine learning (another common misconception). Creativity is not something that is necessarily associated with either of these. A main difference between GAs and the DSNS is that there is no "inheritance" involved. Two parent DSNS strings may also produce at least two 'child' DSNS strings from which new creative objects can be generated using an external generation system. A second difference is that the DSNS can draw "inspiration" (for lack of a better word) from objects completely unrelated to the domain in question (such as photographs and music), not unlike what might inspire a creative human. The actual process is quite a bit more detailed and too long to explain here. However, you could read about it in our Springer book linked to in the article, or a free shorter version published via arXiv at Cornell.

http://arxiv.org/abs/1507.07058
genem genem 5/31/2016 11:25
Azlan Aqbal's DSNS approach/algorithm loosely reminds me of the "genetic algorithm" technique in computer science: for example:
.
http://www.theprojectspot.com/tutorial-post/creating-a-genetic-algorithm-for-beginners/3
.
Perhaps I would have understood the basic idea of DSNS better if these articles had contrasted DSNS with the genetic algorithm approach.
Good luck in finding other applications for DSNS.
dhochee dhochee 5/31/2016 10:06
"So thousands upon thousands of chess problems can be ranked aesthetically and also seemingly endlessly composed from scratch which is something that would require countless man-hours if domain-competent humans were to attempt it."

1. What value is there to ranking by a subjective standard that most chess players reject?
2. How does it save man-hours when it can't create compositions comparable to those created by domain-competent human composers?

The generated problems typically have numerous winning lines and are little different than actual chess positions taken from games. They're messy, without clear solutions. Not entirely useless for study, but if that's the intent, then using actual game positions is superior. It might be interesting from a computer-science standpoint, but certainly not from a chess perspective.

Dr. Iqbal spent a decade working on software to evaluate and create chess beauty while failing to understand the core concepts of what actual chess players find beautiful. Is it any surprise that those responsible for funding medical research would reject his proposal? They made the right choice, and Chessbase keeps making the wrong one by giving this guy a platform.
flonks flonks 5/31/2016 08:45
Chessbase does not do itself any good with these articles.
gmwdim gmwdim 5/31/2016 04:46
In a way I feel some sympathy for Azlan. Here he posts an article which he no doubt put a considerable amount of effort into writing, and gets immediately ridiculed for it. But then I recall his vitriolic responses to legitimate criticism regarding his previous articles, and am reminded that he brought it all upon himself.
jimliew jimliew 5/31/2016 02:19
The compositions are seriously boring
jackie jackie 5/31/2016 01:42
More clickbait muppetry?!




Mr TambourineMan Mr TambourineMan 5/31/2016 12:28
Maybe Ludwig could compose a seranad to Iqbals leaving?
Frits Fritschy Frits Fritschy 5/31/2016 11:52
I congratulate dr. Iqbal with his doubtlessly deep insights in medicine, but as far as I know, this is a chess site. As I'm not interested in his professional frustrations, I don't feel like reading the complete article. Just one thing about the chess content:
"So the study-like constructs composed are basically puzzles that resemble traditional composed studies in the sense that the solver is invited to figure out the key or first move that leads to the most decisive advantage, i.e. the ‘best’ move."
In a composed study, it is not just about a first move, it is about all the moves. It is not about the move(s) that lead to the most decisive advantage, it is about series of moves that in the main variations are the only way to win or draw. There is no such thing as 'the best move' in a composed study. Apart from that unique way to get to the goal, there are only bad moves for white. Poetry is about finding the only word at the only possible place; I remember vaguely the Polish Noble prize for literature winner Szymborska once wrote. The same goes for composed studies.
AgainAgain AgainAgain 5/31/2016 09:55
Why do you let this man publish? He is the single worst contributor at Chessbase.
PurpleUnicorns PurpleUnicorns 5/31/2016 09:37
Please Chessbase, not this guy again.
1