Can computers compose artistic problems? (2)

by GM Pál Benkö
6/16/2016 – Earlier this week we brought you part one of Pal Benko's critique of machine composed chess problems. In part two this world famous problem composer shows us further examples and how they can be improved. He also gives us an example of composing together with a computer, "the first time in my life I did not create a chess problem fully in my own mind," and tells us why he has decided to drop out of problem competitions.

ChessBase 16 - Mega package Edition 2021 ChessBase 16 - Mega package Edition 2021

Your key to fresh ideas, precise analyses and targeted training!
Everyone uses ChessBase, from the World Champion to the amateur next door. It is the program of choice for anyone who loves the game and wants to know more about it. Start your personal success story with ChessBase and enjoy the game even more.


Can computers compose artistic chess problems? Part two

By Pal Benkö

In part one of this article I told you how ChessBase report entitled "A machine that composes chess problems" raised my interest. In this article Dr Mohammed Azlan Iqbal presented ten selected work composed by the machine. As a problem expert I took it upon myself to comment on these examples. Here I look at a few more examples of machine composition. Once again the solutions to most of the problems are hidden and require a click to be revealed. This is in order to offer you the opportunity to try and solve them yourself, as an additionaly little benefit.

Rook promotion

Chesthetica, 20/8/2014 7:04:40 PM

White to play and mate in three: example of a computer generated chess problem

The solution 1.Rg5 Kc6 2.d8R Kb6 3.Rd6 mate. The key takes away three flight squares of the king in the first move.

Here is an alternative I composed today. It has the same material but the key move does not take away any flight squares (which is frowned in problem chess). It also produced many different mating pictures, depending on Black's defence.

Pal Benko, Original, 15.6.2016

White to play and mate in five moves

The solution: 1.f7 Ke7 (1... Ke6 2.f8=R Ke7 3.Raf5 Ke6 4.Kd8 Kd6 5.R8f6#) 2.Rf5 Ke6 (2...Kf8 3.Rf6 Kg7 4.f8=Q+ Kh7 5.Rh6#) 3. f8=R Ke7 4. R8f6 Ke8 5. Re5#.

The stalemate opportunity is already a gnawed bone. Let us see the same theme from the past:

Pal Benko, Chess Life 1977

White to play and mate in four moves

The solution 1.Kd7! Kb7 [1...Kxa7 2.Kc6!] 2.c8R! Kb6 3.a8R Kb7 4.Rcb8 mate.

I suppose that the double rook promotion was not an original idea, even as early as that time. Therefore I made it longer trying to hide the point. The degree of difficulty of the solution is not the most important, but a rather plausible first move worsens the problem value. The computer commits similar errors in the following example too.

Rook mates

Chesthetica, 24/9/2014 1:51:11 PM

White to play and mate in three moves

The solution 1.Kb3 gxf2 2.Kc2 f1Q 3.Ra5 mate. We can teach only beginners this method of mating with the rook. But it is better if we place the rook on b5. In that case 1.Kb3 gives and takes away a flight square for the black king (if 1.Kb3 Kb1 2.Rd5). It also looks better if the rook is placed on f5. Then we can even remove the knight:

The solution 1.Kb3 g2 2.Kc2 g1Q 3.Ra5 mate. It is possible to create good and neat problems with limited material but there must be something new or interesting in them.

I published the following item in my biography My Life, Games and Compositions (see below):

Pal Benko, My Life, Games and Compositions 2003

White to play and mate in nine moves

The solution: 1.Rg6! (1.Ra7? is only mate in ten) 1...Kh7 2.Rg4! Kh6 3.Kf2 Kh5 4.Kg3 Kh6 5.Kh4 Kh7 6.Kh5 Kh8 7.Kg6 Kg8 8.Rf4 Kh8 9. Rf8 mate. The computer is perfect for verifying the compositions and it has confirmed that this solution is the only one that leads to mate on move nine.

Chesthtica, 28/9/2014 3:40:07 AM

White to play and mate in three moves

The solution 1.Kf1 Nc3 2.Bg4 [2.Bh3] mate in next move. Already the first move makes the composition worthless, and there is also a dual. Here is the improvement:

Chesthtica, modification Benko

White to play and mate in five moves

The solution 1.Kf2 Ne5 2.Bh4 (zugzwang) 2...Nf3 3.Be4 Kh2 4.Bg3+ Kh3 5.Bf5 mate.

The solution is not so simple in this case. The knight offers itself to be taken twice, and that is refused by White (thematic). A new mate picture must be discovered, different from the obvious. The bishop with Be4-Bf5 makes a switchback.

All of the above was found by me after some hours search with the computer. It happened for the first time in my life that I did not create a chess problem fully in my own mind, but found part of it with a computer. Such methods are frequently used in endgames with limited material, and it has caused some controversy. In the end such compositions are recognized as having full value. This is why I am not going to take part in chess problem competitions any more. It is becoming more difficult to distinguish human work and computer output. Anyway the revolution of endgame theory started with proving two bishops versus one knight is a win.

All of the above proves that computers are unable to create artistic chess compositions without human assistance. But so far chess problems created by machines do know any of basic rules of problem composition. These rules may be ignored or broken, but there must be something valuably new in return.

I suppose that some of basic principles may be included in the programs in the future (for example not to start the solution with a check or by capturing a piece). However, the artistic content, beauty and harmony that generates pleasant feelings in humans cannot be easily input, since the computer has no feelings. We can see that this aspect is missing in their compositions.

Replay all the above problems

Select games from the dropdown menu above the board

About the author

In addition to his success as a player, Pal Benko is a noted authority on the chess endgame and a composer of endgame studies and chess problems. He is an over-the-board GM and also a FIDE IM of chess composition. The only other person we know who has these two titles is Jan Timman of the Netherlands.

Pal Benkois also a dear friend who keeps in touch with us regularly, sending problems and puzzles for the ChessBase news page on special occasions.

This biography is a celebration of a great man's creative legacy, an amazing collection of 138 deeply annotated games which have been carefully prepared to be entertaining, enlightening, and instructive. They are brought to life by Benko's memoirs of his early years in war-torn Hungary, a world of poverty, chaos, pain, and ultimately, personal triumph. His insights into famous grandmasters transform legends into real people with substance and personality, and his reminiscences of famous tournaments take us on a journey through chess history unlike anything that's been published before. A massive survey of Benko's openings shows us the scope of his theoretical contributions to the game. Photos abound, and 300 of Benko's chess compositions allow lovers of the game to become intimately acquainted with a strikingly beautiful aspect of chess that most have overlooked.

This highly entertaining and instructive book gives competitors who wish to improve their playing strength a dynamic, fun way to deepen their knowledge and understanding.

Some earlier ChessBase articles by and about Pal Benko

4/1/2016 – Pal Benko's April entertainment
Our loyal friend, Hungarian GM and problemist Pal Benkö, who at the age of 87 is still composing wonderfully imaginative problems and studies, has sent us four very unusual (and tricky!) puzzles to solve on this auspicious day. We present them to you without solutions, so you have a few days to try and find the hidden subtleties and traps. One thing is certain: Benko never ceases to delight.

7/12/2015 – Pal Benko: Variations on a Kubbel study (2)
Our good and faithful friend, GM Pal Benko, recently explained to us why one of the most famous studies of all time, composed in 1922 by Leonid Kubbel, was not completely flawless – and indeed worthy of improvement. He showed us how the process works, and in today's second part you can watch one of the greatest composers of our generation polishing flawed studies.

6/23/2015 – Valuation: variations on a famous Kubbel study
One of the greatest chess composers in history was Leonid Ivanovich Kubbel, born in 1891 in St. Petersburg, Russia. One of the greatest contempory composers is GM Pal Benkö, born in 1928. One of the most famous studies of all time is a 1922 composition by Kubbel. It is, however, not completely flawless, and so Benkö set out to polish it. He gives us a unique insight into the process.

4/4/2014 – Benko: Fun problems to celebrate April 1st
Our friend and world famous chess composer GM Pal Benko got into the spirit of the day and sent us three problems to solve. They look deceptively easy, but you must consider the day of publication and not be fooled by the guile of the composer. We will leave you to work things out for a few days, and then give you the answers which may come as a surprise to some.

3/29/2014 – Pal Benko on Richard Réti’s endgames (2)
125 years ago a boy was born in the Austro-Hungarian part of what is today Slovakia. Richard Reti was a mathematician and world class chess master. Reti was also an endgame specialist who composed some of the most original endgame studies ever devised. Some were flawed, and now, almost a century later, his compatriot GM Pal Benko provides revisions to these studies.

3/26/2014 – Pal Benko on Richard Réti’s endgames (1)
At the turn of the last century an Austro-Hungarian mathematician shook up the chess world with revolutionary new ideas ("hypermodernism"), and with some of the most original endgame studies ever devised. To celebrate his upcoming 125th birthday another great chessplayer and endgame specialist, GM Pal Benko, has sent us some examples of Reti's works.

12/23/2013 – Pal Benko: Secrets of Study Composition (2)
One of the greatest study composers – as well as a former world championship candidate – is our friend Pal Benko, who never fails to send us a special Christmas gift. This year it was an article that offers unique insight into the process of chess composition. We brought you the first part a week ago. Today it is about breaking the pin and avoiding stalemate. And there is a remarkable study for you to solve.

12/17/2013 – Pal Benko: Secrets of Study Composition (1)
There is more to chess than tournament games. The area of chess studies and problems is equally creative and breathtakingly imaginative. One of its greatest composers is grandmaster (and world championship candidate 1959 + 1962) Pal Benko. The 85-year-old author of some of the most famous studies of all time has sent us an essay on the remarkable process of chess composition.

7/15/2013 – The Life Gambit à la Benko
Pal Benko (Hungarian: Benkö Pál) is, as 99% of our readers probably know, a legendary chess grandmaster, author, and composer of endgame studies and problems. He was born on July 15 1928, which made him 85 today. Diana Mihajlova met the fit and active octogenarian, who has been a "pal" of our company for a decade, in his home town of Budapest. Here is part one of her birthday report.

7/18/2013 – The Life Gambit à la Benko – Part two
On Monday Pal Benko, legendary grandmaster, author, and problem composer, turned 85. Diana Mihajlova, who recently met with the fit and active octogenarian in his home town of Budapest, sent us a birthday report in two parts. Today we learn of Benkos escape from Communist Hungary to the US, and his relationship with Bobby Fischer. And we get to solve two highly entertaining problems.

5/20/2011 – Greetings from Pál Benkö for 25 years of ChessBase
"Congratulations to ChessBase on your 25th anniversary! Your news page is the the first thing I look at every day when I go on the Internet. You do such wonderful work. Keep up your great service for the whole chess world." Heartening words from legendary great chess player, theorist, author and problem composer – who in addition sent six anniversary puzzles for our readers.

4/24/2011 – Easter puzzles by Benko – a World Champion challenge
Pál Benkö, 82 and still going strong, is a world class grandmaster, author and problem composer. He is also a faithful friend who periodically sends us puzzles for our newspage. This time, for Easter, he has selected four problems which stumped a World Champion. It is a challenge for you to do better, and win a special prize in the process. Enjoy.

12/30/2009 – Pal Benko improves on Troitzky
In 1856 the great Sam Loyd composed a chess problem, which 75 years later inspired Alexey Troitsky, one of the greatest composers of endgame studies, to create a puzzle with a similar theme. It proved to be flawed. 75 years after Troitzky another great composer, Pal Benko, took up his problem, improved on it and submitted it for our Christmas Puzzle page.

World class chess grandmaster, author, and composer of endgame studies and problems. Benko qualified for the Candidates Tournament for the World Championship in 1959 and 1962, and for the 1970 Interzonal tournament, when he gave up his spot to Bobby Fischer, who went on to win the World Championship in 1972. Pal was born in 1928 and lives in Budapest, Hungary.


Rules for reader comments


Not registered yet? Register

dhochee dhochee 6/17/2016 06:50
I have been a vocal critic of Dr. Aslan's work. Nearly every problem he presented as an example of what his software created was low quality, whether you judged according to composition standards or simply wanted a challenging "find-the-best-move" problem.

However, he more recently drew my attention to some problems that were better, at least by my low standards. They were deep, and the winning moves were not the obvious ones. While it still doesn't seem like an earth-shattering accomplishment (you could achieve something similar by using an engine to analyze positions from real games), it caused me to think more about why I and so many others reacted so negatively. Even if we are not amazed by the quality, shouldn't the ability of software to generate chess problems be interesting, and not worthy of scorn?

The main issues are expectations and communication. Dr. Aslan approached this whole topic as a computer-science project to demonstrate an AI-style approach to problem-solving that can have use beyond chess. Chess was selected as a proof-of-concept. As a researcher, he has very different criteria for what constitutes success than does a chess player who wants to solve the problems, and Dr. Aslan oversold what he accomplished to us with his articles on this site. If he says that he created software that can identify chess beauty and replace a human composer, then we want to see problems that can rival what Pal Benko creates, but Chesthetica isn't even close to that.

It would not have been terribly difficult to adjust his approach to suit the audience, but it wasn't his concern. For example, if he had created a restrictive set of criteria to identify what we as chess-enthusiasts consider a good problem, and then applied that filter to the hundreds of thousands of generated problems, he would have ended up with some that were much more pleasing, and resulted in less criticism. But from his standpoint, that wouldn't be proving much. It doesn't contribute to his thesis.

Perhaps most importantly, a more modest and realistic presentation of his work, written with the chess-playing audience in mind, would have done much to improve the reception. I hope Dr. Aslan can find a way to align his expectations with those of his audience and communicate that more effectively.
azlan azlan 6/17/2016 09:04

It's interesting you should mention the need for "taste" in composing. This is a general PhD topic description I've had at my university website for years and years.

Title: Computational Taste: An Investigation into the Mechanics of Personal Preference in Intelligent Agents

Area: Artificial Intelligence

Special requirements: programming skills and some background in AI

Description: Demonstrate an algorithmic approach within a specific domain whereby a computer is able to develop its own 'taste' with respect to a particular feature of that domain. Scope and details to be discussed further.

In the early days of programming Chesthetica, it was designed to be an extension of its composing module. However, this was before the introduction of the DSNS approach, which does not necessarily preclude the concept of 'taste'. One of the risks of such an approach, however, is that the computer may become just another 'master' composer with no conception of a common ground of aesthetic appreciation in the game.
lajosarpad lajosarpad 6/17/2016 08:45
@treetown: While I mostly agree with you, I would like to point out that, if memory serves me well, he did not say he cannot distinguish computer-generated chess problems from human compositions. As far as I remember he mentioned that he cannot distinguish human compositions from hybrids, where composers are checking their ideas with a computer. If his position was that he cannot distinguish human composition from computer-generated problems, then both his articles would be pointless. So no, the Turing test was not passed according to him.

@gmwdim: Actually, beauty is not owned by chess problem composers. With all honesty, I can tell you that I have seen and played combinations by the thousand which I consider to be more beautiful than most of the chess problems. It is because they were closer to my personal taste. I will not consider a chess problem to be more beautiful than such a position just because a chess problemist has a different taste or some rules designed to approach beauty do not apply. By the way, I see nothing wrong with applying for funding, nor with wanting to live well from the work one does. The question is whether this particular research might lead to great results and that is food for a scientific debate. My personal opinion is that such a computer should have a personality to be able to appreciate beauty and to develop a taste. I do not believe that an approach omitting taste and personality would yield very good results.
jcaleb jcaleb 6/17/2016 01:13
1. Computer's can't compose artistic problems because there is no money in it yet and programmers are not solving it rigorously.

2. It would be easier for humans to compose artistic problems with computer aid.
azlan azlan 6/16/2016 05:32
@treetown: I doubt anyone will bother with making automatic chess problem composers that can satisfy the tastes of most master composers today because unlike chess-playing programs (for which there is actually a sizable market to sell to), automatic chess problem composers don't produce anything of "value" and most players would rather not crack their heads trying to solve such puzzles (compared to say, enjoying a swashbuckling blitz game full of risks and mistakes).

Having said that, chess problem composition (even of "weak" compositions) is a very good place or domain to test new ideas and approaches in computational creativity. If only to demonstrate that such approaches are able to come up, unaided by humans, with things of greater aesthetic value than random chance. That's the first step. After that, it's up to anyone interested to invest millions in refining the approach and supplying the necessary (typically massive) computing resources toward a particular purpose or goal.
gmwdim gmwdim 6/16/2016 05:20
"master chess problem composers do not own or even have a monopoly on the concept of "beauty" or aesthetics in chess"

What a joke. Pal Benko, one of the most accomplished and respected chess authorities in the world, points out with clear explanation the shortcomings of the computer-composed problems, and your response is to basically argue that he (along with other master composers) is not a definitive authority on this topic? Then what's the point of even bothering to quantify beauty (which seems to be the focus of your research), if you can always dismiss any criticism with the logic that "beauty is subjective so the master's opinion is just an opinion". How is this scientific in any way? Chess problems are an art, while computer algorithms are a science. Azlan has incredibly managed to combined these two into something that is neither art nor science, but rather more like a pseudo-intellectual activity for the purpose of grabbing research funding and boosting his own ego.
treetown treetown 6/16/2016 05:01
I'm glad part 2 is up because part 1 sort of just ended.

1. Benko mentions that initial moves that take away squares is frowned upon - it is less aesthetic probably because it is more obvious being a forcing move. A move that does not check, capture or restrict is probably hard to find. BUT if that is case, it should be programmable. This suggests that perhaps the programmers should learn more about what are the generally accepted concepts of good composition design.
2. He also notes that he can't distinguish the work done by a person and one done by a machine - so does that pass a form of chess composition Turing test?
3. Finally, that two bishops against a knight is now a known win is the type of information that computers could come up with.
4. Can taste and aesthetics be stated clearly enough to be codified into a program? Maybe not now but... for now I'll have to defer to GM Benko's taste and judgment but never say never. I'm old enough to remember when the early playing programs struggled long and hard just play a legal game.

Nice articles!
azlan azlan 6/16/2016 12:58
@Mr TambourineMan: What does that have to do with anything?
Mr TambourineMan Mr TambourineMan 6/16/2016 12:02
Mr Azlan before we talk can you pls show us even one problem you composed by yourself?
azlan azlan 6/16/2016 11:41
@thlai80: You still don't get it. Let me give you another example you might understand. Computers today are able to see forced mates even as deep as 50-200 moves (and beyond). Now imagine there was a relatively small group of humans or sentient machines that started composing problems along these lines. Problems so deep and beautiful they require 50-100 pages just to explain. Now, imagine they suddenly looked upon compositions like Benko's. Would it be fair for them to say that he didn't understand "beauty" in the game? Or that his aesthetic feelings were meaningless? Or that he didn't even understand chess? Do you see the analogy?

Besides, even the best compositions by the best human composers today, as someone pointed out in the previous article, are not worth any money. Even ChessBase couldn't sell them for a profit. Unlike the artistic works of Van Gogh or Picasso, for example. So I suppose what Chesthetica comes up with are worth less than nothing. :)

Regardless, the generic DSNS technology behind Chesthetica may have applications in other areas some day. Like I've said many times, it was never about creating compositions to rival those of the best human composers. There's simply no incentive for anyone in AI to do that, or someone would have already done it by now. This does not at all mean it can't be done. If IBM or Google would like to pour millions of dollars into a project like that using existing AI technology, I'm pretty sure it could be done. But to what end?

Now, don't get me wrong. I, personally, agree with Benko about the relative quality of Chesthetica's compositions and that what he has created in this article are certainly better. I'm merely pointing out that Chesthetica was not designed to compose the kind of problems he is comparing them against. However, I think Chesthetica is hardly the benchmark of what is possible in AI, even today (given the right amount of funding from the right people).
lajosarpad lajosarpad 6/16/2016 11:37
"Beauty" is a subjective concept. If something is beautiful for me, it might be ugly for the other person and vice versa. Since Benkő did not like the compositions of the machine, we can say that he does not like that, but solely this fact is not enough for any further objective conclusion.

Since beauty is subjective, to make sure that composers will not feel that their composition was unjustly unrewarded, "beauty" was conceptualized. As a result, we have a set of rules which describe how a chess problem becomes more/less valuable in terms of "beauty".

These objective rules could be computed into a software. For instance: do not compose a problem starting with check. Or: Compose problems where the evaluation gets accurate only at the end of the combination. Since the computer will generate tens of thousands of problems, these heuristics of problem selecting could improve the output significantly.

We can see hard words, like "the computer has no feelings". What is the definition of "feeling"? We need this definition to be able to determine whether computers feel. Also: let us suppose computers do not feel. Does that mean that they will not feel in the future either?

I understand Benkő's point that the problems generated by the software were not satisfying to him, but it would be a mistaken induction to believe that another software, or a future version of this one will not produce satisfying results for him.

As so frequently, this debate is reduced to belief. There is the camp of those, who believe that computers are not able to produce fine art and there is the camp of those who believe that computers are able to do so. If one firmly believes in either of the statements then he/she will tend to ignore opposing facts. Instead, let us focus on the problems.

Last, but not least: the artworks created in the XX. century tend to be invaluable for people living prior the era. Art and beauty is subjective, after all.
thlai80 thlai80 6/16/2016 11:00
Hundreds of millions of players worldwide who do not know openings, randomly playing middle game, zero endgame knowledge. You care about this group on what they think is beauty in chess rather than the really playing community! And if your research cater and try to amaze this group of people, your creation is not even chess.

You are trying to re-define chess creativity/beauty on your terms. This is akin to drawing a cartoon to the world kid population below age of 5 and claim it is a masterpiece instead of letting it objectively evaluated by real artistes.

Enough said.
azlan azlan 6/16/2016 09:35

"The chess world is a very small community"

Ummm.... no. There are hundreds of millions of players worldwide. Enough said.
thlai80 thlai80 6/16/2016 07:55
And that is a generous conclusion from the great Pal Benko that this Iqbal creation is overblown than it really is. There's no beauty or aesthetic in its compositions, but just a fake marketing for his own thesis. Best of all, this guy keeps talking about more funding, getting free money feeding from its University and/or govt. A total waste of tax payers money to feed one's ego.

Quote: "The model was never designed exclusively around what only master composers or master players think."
@azlan. No, not even that! You don't call something as chess beauty if you cannot cater to the chess community. It does not take someone of Pal Benko's calibre to understand the beauty in composition, to mere mortals like us patzers, we would analyze for hours and realize the hidden gems behind.

The chess world is a very small community. "Competent" and able to understand most concepts and able to make plans and foresee moves ahead in a game requires at least about ELO 1600 to 1700. Even this group is error prone but able to appreciate chess beauties. Including this very last group, the community numbers below 100,000. A mere 0.00167% of the world population. Dont' even talk about Pal Benko's level of beauty. If you can't cater to this small community, your work is useless. If your work in chess caters to general world population in bell curve, again your work is useless. To quote domain competence, anyone out of the 0.00167% is basically zero competency making random moves. So, to point anomaly of Pal Benko as a base for your argument is, honestly speaking useless. It is a wonder that your level of stubbornness stretches over months after being rejected by virtually everyone in the chess community from the top to bottom. If you do not believe this, try asking Chessbase to sell your product. Let's see how many would even give you a cent.
azlan azlan 6/16/2016 07:30
One thing that suprisingly needs to be pointed out repeatedly is that master chess problem composers do not own or even have a monopoly on the concept of "beauty" or aesthetics in chess. The vast majority of players (whom are not IMs and GMs, actually) also experience beauty and amazement in the game. To a lesser degree perhaps but beauty nonetheless. This "common ground of aesthetics" is, in fact, what my aesthetics model has always been about. You may read about it in section 3.2 of my thesis which was written back in 2008.

The model was never designed exclusively around what only master composers or master players think. How much value would there be in that? It was designed around "domain-competence" which implies people who know how to play reasonably well which need not necessarily include masters or the best in the world (in other words, the vast majority of people who experience the game). What kind of market would a technology that caters only to the tastes of the elite be worth?

I'm not saying it's technologically impossible to model things like "surprise" and "paradox" in the senses that only master composers can fully appreciate but simply that hardly anyone in AI is going to see it as worthwhile because it can probably even be done using heuristics which are nothing new (decades old, in fact) or machine learning techniques which are nothing new either (and don't add to the AI literature in any useful or profitable way).
gmwdim gmwdim 6/16/2016 06:49
So basically the computer has been generating weaker, less interesting variants of problems that Benko has already created many years ago.
Justjeff Justjeff 6/16/2016 04:52
>>computers are unable to create artistic chess compositions without human assistance

Of course. Us humans are not amused by trivial solutions, we are entertained by what surprises us. A computer does not know the difference and computes the fastest win.

The silicon monster is far better at finding the solution objective than we are. Whether it does so in a way that is pleasing and/or surprising depends entirely on human weaknesses.