What Is Creative Transmutation in Computing?

A photograph of a tree might inspire the design of a new antenna. A photo of an old acquaintance combined with vaguely familiar music might lead to the painting of a portrait or the writing of a novella. You get the idea. But if human brains can accomplish this, can the process be replicated or at least simulated in machines? I would argue that the answer is yes. In fact, I would say that my computationally creative computer program, Chesthetica, has been doing that for many years now using the Digital Synaptic Neural Substrate (DSNS) approach. Despite having the word, ‘neural’ in it, it has nothing to do with artificial neural networks (ANNs) or machine learning (ML), by the way. Essentially, this is how the program ultimately arrives at where to put the pieces in its original compositions. 

Still, proof of this creative transmutation phenomenon (in any domain) was non-existent or scant until quite recently. The details and references, for the interested reader, are in a recent conference paper I presented (a personal or institutional IEEE subscription may be required to access the revised, published version). The video presentation itself is unlisted but nonetheless accessible here. In this article, however, I will summarize the main points for the benefit of ChessBase readers. (For those completely unfamiliar with Chesthetica and the DSNS, you may first want to refer to previous articles here on the subject.) I had suspected for some time that the DSNS approach – applied in conjunction with some quantum randomness, to be fair – was somehow able to capture creative elements or aspects of objects from different domains and utilize them in its compositions. That was the most reasonable way to explain how it was composing so many original and challenging chess problems (or puzzles, if you prefer) without any kind of ML employed in the composing process.

I then came across two particular compositions by Chesthetica composed on different machines, using different versions of the program and at different times (about 4 months apart). They were so similar yet not identical that the evidence of creative transmutation was virtually undeniable for me (see diagrams below). For the pundits, I understand these compositions may not conform to the highest standards of traditional chess problem composition but that is not the issue here. Besides, not all of the computer-generated compositions are like this; some are better than others, depending on whom you ask and when you ask them. Most have proven to be aesthetically pleasing or interesting enough for most people.

Anyway, if you consider the first position on the left and the later one in to the right, it is clear that they are not even the same length yet share the same idea or theme. In fact, the one on the right looks like something a human composer, after looking at the first position, might try to compose without violating copyright in the strictest sense. It shares the same idea but is nonetheless different. The question therefore arises, how could these two compositions come about on different computers and at different times if the idea itself was not being ‘transmuted’ in the composing process? Chesthetica was never explicitly programmed to do any of this, mind you. None of these thematic concepts are coded which gives the program the capability to compose literally any kind of composition possible, in theory. In other words, the idea (and potentially many others) already exists in some fragmented or rudimentary form in the ‘raw materials’ utilized in the composing process:

To produce original compositions, Chesthetica uses characteristic details of the images in its collection combined with characteristic details of sequences taken from real games and combines them stochastically using the DSNS approach (which actually supports various domains, including music). Given time, a seemingly endless number of chess problems can be produced. The diagrams below show another example pair I discovered later which were composed by Chesthetica over a year apart. This time the one on the right was using the same computer as the one on the left, but as with the previous pair, a different instance and later version of the program.

There have been other examples as well but none quite this prominent or clear which further suggests gradations in the levels of transmutation. If these similarities were simply due to ‘complex emergent behaviour’ using a purely formalized approach (not unlike the repeating pattern in say, Langton’s Ant), they would be identical… but these compositions are not. Furthermore, we have to keep in mind that this is chess with all its rules etc., not some simple pattern on a grid that will repeat over and over. On the other hand, the odds that such pairs as shown in the examples above could come about purely by chance is astronomical, at best. It may be analogous to a chess engine playing two full 30-move games against two different opponents at different times and on different machines yet the games were strikingly similar. To my knowledge, this typically does not happen, if ever.

Therefore, there must be an underlying phenomenon at work. A relatively unexplored or undiscovered one. Perhaps it is even a part of some greater “theory of intelligence” (or creativity). At least, that is my thinking or best guess at this point. The wider implication is that there are fundamental themes or ideas that manifest themselves within all sorts of objects from all sorts of domains. These can also be transmuted between objects and domains which lead to the creation of other objects that might share some or many similarities. Consider the concept or idea of an aircraft and how many different instantiations that alone might have. Everything from a paper plane to a spaceship (or a bird) could bear some relationship to it.

Again, the human brain seemingly operates on this basis all the time and we think little of it. A particular kind of human brain looking at a bird, for instance, might sometime later envision a revolutionary type of aircraft. The process does not seem to rely on specifics, like so much in computing restrictively does, but rather pieces just ‘coming together’ sometimes or under the right circumstances. Imagination versus knowledge, if you will. Naturalists Charles Darwin and Alfred Russel Wallace credited their central insight (related to evolution) to independently reading Thomas Malthus’ essay, Population from 1798. Darwin apparently read it and realized how it applied to his own work whereas Wallace read it a few years later. Was the idea of evolution somehow already embedded or fragmented in the pages of that book such that if the right type of brain chanced upon it, it would lead to something like the theory of evolution by natural selection?

The main reason human creativity is so difficult, if not impossible, to explain is because the underlying mechanisms of creative transmutation (assuming it is a real phenomenon) are not presently known. Even the DSNS which seems to facilitate creative transmutation to an extent does not lay bare the precise mechanics of how, exactly, this occurs; much less why. Analogously, ANNs such as applied in deep learning are also difficult to explain and trace with precision. The ‘fuzziness’ of such processes is not a bug but a feature. If creative transmutation is true, however, and can indeed be simulated (at least to a degree) in computing, even the most advanced deep learning techniques employed today would appear primitive in contrast. This would be due, primarily, to how inefficient traditional machine learning techniques are at achieving noteworthy creativity in any domain, if they achieve them at all.

Harnessing and then perhaps even enhancing creativity computationally could speed up the discovery of new ideas, concepts, products and solutions. Even if only 1% of what a creative machine produces are particularly noteworthy in any domain of interest, the benefits will likely still outweigh the costs. This is partly why I think computational creativity can also serve as the basis for crypto assets, but that is another story. The simulation or automation of creativity through the “recycling” of our own creative outputs could lead to significant time and energy savings for humanity, just as a lot in computing already has. Perhaps a combined approach of ML and creative transmutation would function better. Even good old-fashioned AI (GOFAI) still has its applications (even in Chesthetica). Whatever works, as the saying goes. Alas, we will never know for sure unless some billion or even trillion-dollar corporation is interested in pursuing that kind of experimental work and unlike many universities these days (especially those obsessed with rankings), willing to look beyond a 12 or 18-month horizon, high citation counts and virtually immediate profitable commercialization opportunities.

Research like this requires a lot of time, trust from administration, plentiful resources and specialized equipment, a large and dedicated skilled team, and a high tolerance for failure… but in the long run it could be worth the risks. In fact, I might have some ideas about how a general-purpose “creative brain” could be built but I can offer no guarantees. In any case, after 16 years of research, 12 years of automatic composing, six books of computer-generated compositions and over 3,600 of them published online already, I am certainly still interested to see what Chesthetica comes up with next and still curious where this rabbit hole might lead.