Game Changer: AlphaZero revitalizing the attack

AlphaZero: Challenging the way we play

Aditya Pai: What made you take up this project? Did your previous book, Chess for Life play a role?

Natasha Regan: Yes, certainly. We had worked together on Chess for Life and we’d written a nice book about ten players. We did a mixture of interviews and chess analysis in that book. It was a popular book, especially in England.

We then heard in December 2017 about AlphaZero. It was the first time we had heard of it. We were watching at the London Chess Classic, at the time. And then, the news came out that there was this paper. Everybody at the tournament was talking about it; that was the big thing. There were these ten beautiful games. We were in the analysis room where they were showing these. So, we thought, we could do the same type of analysis as we had done in Chess for Life for AlphaZero. We wanted to do a mixture of the chess analysis and the human interest behind AlphaZero by interviewing the team and showing their story.

Matthew and I had both known Demis Hassabis, the CEO of DeepMind. When he was a kid, he was a very strong chess player and ranked very high in the junior rankings. So, we gave him the idea, showed him our previous book, and asked if he would like a similar thing on AlphaZero. He said ‘I am not sure yet’ and went away with the book. And then, he liked it; he thought it would be a good idea to have the book. That’s how the project was born.

AP: Which one of you came up with the idea of writing the book first?

NR: It was me. Matthew, of course, when he saw the games, was very excited about them. And when he was talking about the games, I suggested that we should do a book. At first, he said he wasn’t sure.

Matthew Sadler: I said it would be a lot of work (laughs).

NR: Yes, and he asked if we would be able to get interesting things out of it. The question was if we would be able to find themes. Our last book was a very thematic book about how the opening choices of players change as they grow older. In this book, we were thinking of what we could get out of these games [of AlphaZero] if there will be anything interesting.

MS: Already, in the ten games that were released, you sort of felt that there were a number of patterns in the way of play that you could bring out and then explain to people in a normal way, without needing to give variations.

The first thing that we were asked to do was to look through some of the new games of AlphaZero and pick out twenty good ones for DeepMind’s paper that was to be published in December 2018. When I went through the games, I really thought there were so many nice themes, so many chess ideas that could be brought out — not complicated variations but big chunks like the way it attacks, the way it builds up attacks. It had a very schematic way of doing it. And then we really thought that there was a wonderful book in this.

Besides, as you know, there is also the whole story about AlphaZero teaching itself. It’s amazing on several levels because, in nine hours, it taught itself, basically going through all human chess experience in nine hours. Playing itself and working it all out, it came up with lots of ideas. That’s one thing, I think, that people find very fascinating. There are questions like: which openings did it choose or are the openings that we play today the best?… you know, stuff like that.

NR: You also think about how chess developed. There was a time when people wouldn’t be fianchettoing their bishops. Would AlphaZero like these kinds of plans or would it be pushing its central pawns or would it do something completely different? Would it do things similarly or differently from us?

We were sort of amazed to see that some of the games followed exactly the same opening theory as some of the current opening trends. Also, some of its openings were quite quiet. And then you think if it’s the best if one started quietly. But it seemed to be able to build up from these quiet systems to actually conduct an attack. It has also developed its own favourite openings and it has developed them really well to be a very powerful weapon.

AP: So, what are the openings that AlphaZero prefers? I bleakly remember watching some YouTube videos of yours on AlphaZero playing the Ruy Lopez Schliemann.

MS: Actually, there are two sets of games that were released. There were some starting from the starting position where you really see what it likes and what it dislikes, and there were these games where it got the TCEC (the computer chess competition) positions — that’s where the Schliemann comes from. It’s not an opening that AlphaZero will play normally.

NR: It likes, normally, e4-e5 for black but it does not choose f5 by itself.

AP: And what does it prefer for white? At some point in the book, you have also mentioned that it prefers 1.d4 over 1.e4.

NR: Yeah, it likes 1.d4 and 1.Nf3. 1.e4, I think, a bit less so. I don’t think it thinks it’s bad particularly but it definitely prefers 1.d4 and 1.Nf3 and sometimes 1.c4. But the difference in percentages isn’t that huge. It gives itself a percentage score after each move and by that 1.d4 and 1.Nf3 are its two favourite choices. And then, after 1.e4, it will normally go 1…e5. It prefers that to the Sicilian or French or anything like that.

AP: How much time and research did you have to invest in this project?

NR: We made the first draft of the book by end of June and then did lots of refinements to it throughout the summer. Matthew was working on it solidly from January to June.

MS: Yeah, because I’m analyzing games and, basically, every single free moment that I had was spent analyzing those games. I have done nothing else with chess.  And actually, I have just kept on going since those games were such an incredibly rich source of chess. Every time you look at them, in every new game, you will find a new idea or a new theme or relate to a known theme in human chess. Basically, it’s been a solid hobby for the past year.

AP: So, did writing this book energize or exhaust you?

MS: It’s always a bit of both. From the chess point of view, it was super energizing. You are seeing lots of new things, you are looking at super quality games — and very exciting games as well. So that’s brilliant. But yeah, on top of your normal job it’s always a lot.

NR: It has been really, really fun. It was a very exciting project.

Sadler looking giddy as a schoolboy at the presentation of Game Changer in Wijk aan Zee | Photo: Alina l'Ami

AP: Do you think AlphaZero holds the potential to change the way we play chess? If so, how?

NR: I think, there will be lots of changes that we will see. I mean, for club players, we have pulled out lots of ideas about how to conduct an attack that players can try out. We’ve got ideas about pushing your rook’s pawns forward, opening a line of attack against the opponent’s king and just ways of conducting an attack that should be accessible to all players whatever their strengths are.

MS: For professional players, there is a lot of interest in the opening and also some very interesting strategies that flow out from those. If you think of how the knowledge of top players has built up, you could say that every top player knows ten thousand positions, for example. But then, every player, individually, has got his own view of which positions are the most important. Attacking players might consider some positions as important while strategic players might consider some others. That makes your style.

What I found with AlphaZero is that a lot of things that I considered incidental — stuff that I would not really aim for but, if it came up in a game, I would use — became the centrepiece of its strategy. One very good example of it is restricting the movement of the enemy king. There are a lot of games where AlphaZero moves its rook’s pawn all the way up to h6 against the black king on g8. That pins the black king to the back rank, restricts its movement and then, somehow, the whole rest of the game is about opening lines or maybe getting a rook to the back rank. Giving up material doesn’t really matter then because there is a long term advantage to exploit. Those sort of strategies, those sort of things are what I found amazingly instructive and they changed the way I think about stuff.

AP: Natasha, you’d said that these attacking ideas can be used by club players. But these kinds of ideas – to point out one, I would go for the Ng5 idea that AlphaZero deployed in the Leningrad Dutch where it later even gave a queen sacrifice — are very difficult to execute. So, while the aesthetic beauty of such ideas will please a club player, how much can he or she really learn from it? 

Leningrad Dutch

On some occasions it would yield you a perfect win, on others it could cause you "shameful" defeats. This DVD is intended in the first line to offer strategical guidance for Black, based on the examination of the most typical structures.

NR: I will tell you a bit about my experience of trying it. In our book, we’ve got a number of different chapters on the AlphaZero themes. Each one gives you a new tool to think of that you can use in a certain position. So, say before I knew about AlphaZero, I would play my kingside attacks in one way, and in some positions that’s good. Since learning about AlphaZero, I have learnt a bit more about pushing the ‘h’ pawn, restricting the king, opening the lines or closing the centre. There are various new ideas and so, in some games, that’s really suitable. I have got little extra things to think of when I am playing my game. It’s actually quite enjoyable as well because when I am playing my game, I think what AlphaZero might play in that position or how I could play like AlphaZero.

If I do sacrifice a lot of pawns and don’t understand it properly then that is going to be dangerous and it means I might not win the game. But I am more alive to the possibility of sacrificing the pawns. Perhaps, I might not choose to do it. And if I do it and it doesn’t work — I mean for me — I would think I have tried it out. For some game, that is, if it’s not an important game, I might try it out and see what happens.  And that helps me learn because I am thinking of a new idea and seeing how it works in practice. So, it’s giving me a little bit of extra richness around the game and some new ideas and things to try out.

MS: You also need to make a distinction between the stuff like Ng5 or the queen sacrifices and such tactical things. I mean, those are extremely complicated. But the things we are focusing on are actually the strategical themes. It’s about building up your game. And those are the things that you can really try and replicate.

Master Class Vol.9: Paul Morphy

Learn about one of the greatest geniuses in the history of chess! Paul Morphy's career (1837-1884) lasted only a few years and yet he managed to defeat the best chess players of his time.

AP: But even such a strategy entails a fair bit of risk, don’t you agree?

MS: I mean some elements are risky. The pawn sacrifices are obviously quite risky. But there’s also an awful lot of other things that are actually just thematic and are not so difficult to implement and quite intuitive to understand as well. I think everyone knows the great games of Morphy, of Alekhine or all of these attacking players.  But, I think, part of what we’ve lost in the last twenty years with the advent of really strong engines is that you sort of say,  ‘ah well, that doesn’t work and will never work’. And then somehow you see again that there is a whole range of positions where your feeling about a position being too dangerous for the opponent is actually kind of right.

I think, after some of the games I have played now, I’ve really got a lot more confident playing positions that I once had doubts about. You sometimes even feel confident to go against the engine evaluation. You suddenly think ‘I know AlphaZero would like this for white’ and you might play it. A part of it is that it just gives me confidence in a way of playing.

NR: And actually, one time, I was playing a game in which I had lost material by mistake. And then I just thought, ‘oh, hang on a minute, AlphaZero keeps doing these kinds of things deliberately’ and actually it turned out quite nicely. I just kept playing as if nothing happened and actually, it turned into a really nice game in the end.

Master Class Vol.8: Magnus Carlsen

Scarcely any world champion has managed to captivate chess lovers to the extent Carlsen has. The enormously talented Norwegian hasn't been systematically trained within the structures of a major chess-playing nation such as Russia, the Ukraine or China.

"I was thinking at several points during the game how AlphaZero would have approached this" - Magnus Carlsen | Tata Steel Chess

AP: How much of an impact do you think AlphaZero could make on opening theory?

MS: A big impact, I think. Because it has taught itself, it has got a very clear view about chess. It knows exactly the way it wants to play and it tries to put it into all sorts of positions. Sometimes it comes up with ideas that make you think why they have not been played before, and yet it is the case.

The Schliemann is a very nice example. With black, it came up with this idea of Bb7 at some stage, castling queenside and throwing the ‘h’ pawn forward — something that has never been seen before but actually, it’s a very fine idea. We were also seeing the world championship games — these complicated Sveshnikov positions — with AlphaZero and it was finding all sorts of gorgeous plans. 

Ruy Lopez: Attack with the Schliemann

In this DVD Sam Collins provides a complete and detailed repertoire for Black. Numerous novelties and improvements are suggested, with the conclusion that White has no clear route to an opening advantage.

I think that’s one of the strengths of AlphaZero. Its comparison to other engines is not in terms of calculation — it calculates a lot fewer moves than a monster like Stockfish — but it’s in those positions where there is a mixture of strategy, insight into the position and some tactics as well. It has a great way of finding its path through that complexity, of finding a way to move forward. I think what you find with traditional engines is that often if you just leave them to analyze a position for six hours or something like that, you get a massive number of variations. And then it costs you another six hours through work through those variations. And what does that mean for the position? What could I actually play myself? With AlphaZero, it’s a lot clearer. The path it finds is, somehow, intuitive.

NR: So, what we are saying is that it is easier for people to learn from AlphaZero than, perhaps, with those more calculation based machines. AlphaZero’s plan is a little bit clearer. So, it might put its pieces on certain squares and it will do it regardless of what exactly the opponent’s playing and come up with a plan that can be easy to follow. 

AP: Do you think AlphaZero could become even stronger if it was equipped with traditional chess knowledge and tablebases or would it come in the way of its self-learning method?

NR: This is an interesting question and quite a difficult one. I guess DeepMind’s purpose was a bit different. It was a part of a scientific experiment to see how the machine could teach itself and how strong it could get. And they were doing it for chess, go and shogi. So, DeepMind did not want to add any human knowledge at all — zero, as a part of AlphaZero.

With things like the endgame tablebases, you intuitively think that it could make AlphaZero stronger. However, when we looked at the games, it looked like it had learned those positions itself. So, it was getting those positions right. In that case, would it help to add the tablebase in? And even then, it’s not clear what the answer is because if you give it the knowledge then it doesn’t have to learn that for itself. Then, that might make the training more effective.

MS: When we looked at the tablebases, AlphaZero’s evaluations were actually pretty good. It probably doesn’t play endings as perfectly as tablebases but it’s learnt them extremely well for itself. There’s also the question that it’s an engine that is analyzing a lot fewer variations. So the number of times you are going to hit tablebases is also going to be small. And of course, tablebases are also, often, very extreme positions. If you can play the basic endings right, it’s unclear how much the tablebases are going to add. But, in general, you would always think that if you add perfect play to it, it should actually help quite a bit.

NR: There are also quite a few ideas that could be tried to make it stronger. So, another thing is the optimization of its time. It was playing all its games by making a move in a twentieth of its remaining time, irrespective of how complicated the position was. Again you would think that if it took into account how complicated the position was, it could spend more time. There are plenty of ideas to make it stronger which weren’t used. So, to answer your question, probably there are ways to make it stronger. We don’t know if actually any of those will be tried in the future or not.

MS: But for the purpose of the experiment, it wasn’t really necessary. They just made it very strong this way and it sort of proves what they were trying to do.

AP: Now, coming back to your book, who exactly is your target audience? You said that your book can be used by club players but would a reader need to possess a certain playing strength to enjoy the contents of the book?

NR: No. In fact, it’s not even exclusively aimed at chess players. People who are interested in AI, machine learning will also find a lot of things of interest in it. So, we have chapters on how AlphaZero thinks, how it plays, interviews with DeepMind’s CEO and the DeepMind team. There’s quite a lot actually on how the AI was built and that kind of thing.

For players, I think, really a wide range of players can understand the themes and add them to their game. So, Matthew brought out the themes and came out with all that amazing content. And quite a lot of my focus was making sure that it was easily understandable for players of a wide range of strengths.

MS: And although I did a huge amount of analysis, we tried to ensure that the end product is not analysis heavy but sort of heavy on explanations so we could explain the flow of the game and the things that are happening. The aim is to make it interesting for as wide a range of players as possible, with a particular focus on club players.

AP: Since you are good friends of Demis Hassabis, could you tell us if this will be the end of the association between AlphaZero and chess or can we expect more?

NR: Actually, we were really only involved in the project, so we don’t really know what the next steps are.

MS: I mean, DeepMind is busy with a lot of things. I don’t know if you saw it but on Thursday they did a huge demonstration on Starcraft.

NR: They had some professional Starcraft players in the office and they were playing against AlphaStar, which you can imagine, is the AI version for Starcraft.

MS: The interesting thing about Starcraft is that unlike chess, go and shogi, it is an imperfect information game, which means you can scout around but you don’t have sight over the whole battlefield. There was a big live stream on YouTube which is well worth watching. We don’t play Starcraft at all but what interested us was that the commentary that was done by professional commentators. They were talking about strategies that the AI was making and it was remarkably similar to how it played chess.

NR: We have said in the book that AlphaZero would redeploy pieces. Once they had done their job on the square, AlphaZero won’t wait for the piece to be chased away. And then, there was this one amazing bit during the stream when we heard one of the commentators say that AlphaStar would redeploy characters immediately after they had stopped to be useful in one place. It was really funny because we had said exactly the same thing about chess.

MS: There are so many things going on with DeepMind. Basically, we feel quite fortunate that this scientific project has touched chess and it has produced something really nice — some great games and some lovely food for analysis and thought.

NR: But you can think of this being used for other complex problems. It’s sort of amazing that chess was one of the first things that have helped develop that.

AP: If you were to name a chess player — present or past — whose playing style comes closest to that of AlphaZero, who would that player be?

MS: Oh, we’ve got a few actually. First of all, to a modern player, the first person that comes to mind is Garry [Kasparov]. In the book, we have this game by Garry where he won this wonderfully against Predrag Nikolic in the French in Paris in rapid play. And then you see that in one of the games between AlphaZero and Stockfish looks exactly the same, it uses exactly the same strategy.

Master Class Vol.7: Garry Kasparov

On this DVD a team of experts gets to the bottom of Kasparov's play. In over 8 hours of video running time the authors Rogozenko, Marin, Reeh and Müller cast light on four important aspects of Kasparov's play: opening, strategy, tactics and endgame.

NR: We also compared one to Vishy, didn’t we?

MS: Yes, Vishy has a gorgeous knight manoeuvre game and it reminded us very much of an AlphaZero game. But in terms of attacking style, we thought Judith Polgar’s style was quite similar given the way that she built up her attacks.

And just in general strategic terms, I really felt that Botvinnik had some really close parallels. He had this way of closing off the centre to play on the flanks. He also came up with a number of openings and strategies based around that. It’s quite appropriate in a way because Botvinnik was trying to build a self-learning chess machine.

And I have to say that Magnus’ play is somehow similar — the way that he is so flexible with material. Especially recently, he is giving up the exchange and playing those positions without the exchange. You really feel that his play sometimes shows fantastic echoes of AphaZero’s evaluation of positions. 

Photo: Botvinnik in 1969 | Koch, Eric / Anefo [CC BY-SA 3.0 nl], via Wikimedia Commons

AP: What do you think the future holds for chess in the light of such technological advancements?

NR: I guess in the near future, the top players will try out these attacking techniques and try some new styles. I know that in top chess, sometimes, there are quite a lot of draws. So it might bring back some confidence in the possibility of attacking.

In terms of computer chess, I can also imagine some really good games being played between some of the top computers. The match between AlphaZero and Stockfish got very exciting games because there had two completely different styles playing against each other — a great attacker against a great defender. It’s the clash of styles that produced these exciting games.

MS: The amazing thing for me is that despite computers being stronger than humans for twenty years more or less, human chess keeps on going. We have moved from fighting against the computer to using them as analysis tools. And chess is still thriving. Computers have helped humans to understand top-level chess better. Now it’s much more fun to watch. Just imagine, twenty-five years ago, you didn’t understand anything of what was going on. You really needed a strong player next to you.

I think human chess is going to keep on going from strength to strength. An awful lot of ideas can also be discovered by trying to think like AlphaZero, trying to maybe challenge for yourself the conclusions of engines and finding gaps in there. I am sure that will influence the top players.

And if there is a shift simply from ‘the defence always works’ to ‘attack works quite a lot as well’, then that’s absolutely fantastic. At the end of the day, there has always been this little conflict throughout chess history between great attackers and great defenders. At various times in chess history, great attackers like Tal or Alekhine have been on top. And then at other times, we’ve had great defenders like Petrosian at the top.

In the past 20 years, there have been more defenders. Engines have been able to defend every single attack that we do. So, maybe we will now see a period when attack is going to be the most important thing.

NR: You know, I could also imagine these young players who get really, really strong very early get even more terrifying if they study all AlphaZero games and know how to conduct such attacks. I could imagine them being really fearsome.

AP: Do you think programs like AlphaZero would solve chess one day?

NR: Again, a really interesting question. We are definitely not at that stage yet. In a way, people have been surprised by how many new things are coming out with AlphaZero. We thought Stockfish was as strong as you can get pretty much but there is still a lot of richness in the game through some of the moves AlphaZero has found. And, as we’ve said, there is still scope for AlphaZero to improve in various ways. So, I think we are not there yet.

Historically, people have thought about the ability to calculate everything and computers are getting faster and faster. So, could you just get to the point where you just know the answer to everything? That seems absolutely way off.

Then you think about things like tablebases and you know what the answer is for seven pieces. Will that be extended to eight pieces and nine pieces and gradually solved that way?

AlphaZero’s big strength is the strategic route through how to make the most of the position. So let’s say we have a position that is strictly drawn, with best play. And then AlphaZero can make the most of that position and keep improving until it looks like it could be winning. Do you then call that a drawn position or is that actually a win? If you say solved, you know at the beginning whether chess is a draw or a win.

What you are getting at the moment is when the top engines play each other, most of the games are drawn while some end in wins. So you can say you are pretty sure it’s a draw. But it’s not a proof. You’re just sort of gradually getting sure.

MS: Solving chess, I suppose, that is a mathematical thing. If you can calculate everything then you have solved it. AlphaZero finds promising paths through huge amounts of data. It could get so good at doing that, that you could say, essentially, no one is ever going to beat it; so chess is solved. That’s not quite the same thing. It’s not mathematical solving but playing so well that you will never actually be able to beat it.

As Natasha said, what AlphaZero has shown is that we really thought that Stockfish was basically the strongest we can get and then suddenly there seems to be a whole space of positions where there is still lots and lots of scope for all sorts of things. And who knows, maybe, what AlphaZero has found isn’t the last word and that Stockfish is going to come back and maybe something new will come along. I think all we have discovered is that there is lots of fresh potential in chess. I think it’s going to be a long time before it gets solved.

NR: And even if you did know that with the best play it’s a draw, there won’t be many people who would be able to do that. You would still be human and won’t know all the possibilities.

AP: So if, hypothetically, chess was solved, how much of an impact would that have on human chess?

NR: I will compare it to this game I quite like playing. It’s a simple game called connect four. It’s quite a cute game. That was solved in the 1980s but I still find the game quite enjoyable. And it’s not as rich a game as chess. So, you could, in theory, learn that whole solution and know it’s a win from the start. But actually, people still play it and when I tell people it’s solved they say it isn’t.

For chess, it’s just a rich game and you can’t know the whole solution. I guess it’s different when you think of the very top players in the world. I am trying to imagine how the solved chess would look if a series of moves which leads to a forced result is found.

MS: You couldn’t remember everything, of course. But you could know the direction you need to take in every game, in every variation. With a lot of work, you could get to a sufficiently promising position. I mean, they do it already in stuff like the Berlin where they go 40 moves deep or so. I think it will have an impact on the professional game if it was absolutely solved but I don’t think it will have any impact whatsoever on club players.

NR: But then, even if it is solved, you’ve always got Chess960 and there is so much stuff you can do.

Links

• Game Changer (New In Chess shop)
• More stories on AlphaZero