Pattern recognition in chess

by Qiyu Zhou
5/30/2018 – Is there a correlation between the strength in chess of players and their ability to recall a position in chess using short-term memory? This was the research question of a budding young scientist, QIYU ZHOU, who gave professional and casual chess players positions to study and then attempt to reconstruct them within 30 seconds. Her results are meticulously documented in a paper we are pleased to publish. At the end, there is an appeal to our readers to help with associated material.

Mega Database 2018 Mega Database 2018

The "Mega" is the database every serious chessplayer needs. The database contains 7.1 million games from 1500 to 2017, in highest quality standard, full of top level analyses and completely classified.

More...

Personal Engagement

As a former World Youth Chess Champion, being curious about how chess players are able to remember so many positions and what contributes to their ability to play a game of chess is only natural. Therefore, it came to mind to conduct an experiment on the difference in pattern recognition of players of different chess strength. Strength in this study will be evaluated by their standard FIDE rating (A chess rating system used to calculate an estimate of the strength of the player).

Hypothesis

There should be a correlation between the strength of a player and their ability to recall a position in chess, despite using short-term memory. Experienced players should be able to recall a “normal” position better than a novice player. All three groups should recall random positions at around the same accuracy.

Introduction

From experience, generally the more positions one is familiar with, the stronger they will be. As a player one knows that the more often one looks through games of chess the better one is at finding the best moves in positions. Planning future moves utilises pattern recognition in order to identify what should be the best course of action. Pattern recognition should, therefore, be directly correlated to the strength of a player, as strength indicates how much effort they have put into either studying or playing chess.

Since the strength of a player in chess can be assigned a numerical value based on the Elo rating system, players of varying strengths will be present. Generally speaking, the more games one wins the higher their Elo and is thus a good indicator of how strong a player is relative to others. Elo ranges from 1000 to infinity, though the highest rated player currently only has an Elo of 2843. People are chosen based on three categories of experienced, casual, and novice. Experienced players are ones with an Elo rating, based on games played in Canada. Casual players are people who know the rules of the game and have played at least a couple of games in their life. Novice players are those who have just learned the rules, and are able to identify which piece is which, but have hardly ever played chess before.

As each person has to look a position and attempt to memorize it, and then set it up, a 30-second limit is put on the amount of time the subject for observing the position. The limitation of 30-seconds is used because generally accepted values for short-term memory range from 15-30 seconds. The purpose of having two different types of positions is to see if there is a difference between each subject’s ability to remember positions they may be familiar with and one that they are completely unfamiliar with. Logically, there should be a large difference of accurately placed pieces between the normal and random positions. Theoretically, all subjects should have close to the same number of correctly placed pieces during the random position trials, as pattern recognition no longer applies to positions previously not acquainted with.

Methodology

Materials: The equipment used are a chessboard, pieces, two tables and two chairs.

Procedure:

  1. Set up a position on a 3D board. All positions are to be set up from the white side.
  2. They will have 30 seconds to look at the position and remember it to the best of their ability
  3. After the 30 second mark, they will be taken away and put down in front of a completely empty board.
  4. They will be asked to set up the same position that they had just seen to the best of their ability.
  5. They are told they have one minute, but in fact, they have two minutes.
  6. This will be done three times, once for each normal position.
  7. Steps 1-6 are repeated for the three random positions.

Scoring system: The number of pieces will be taken into account. The number of pieces they get right will be scored on a percentage. Thus, there is no error in measurement and therefore no uncertainty figures.

Variables: Independent variable — none; dependent variable — the number of correctly placed pieces.

Controlled variables, degree of control/influence:

Board with an actual position: These are positions that may/can occur naturally in a game of chess. The purpose of the common position is to determine the difference in pattern recognition between novice and experienced players. Theoretically, an experienced player will be able to remember this type of position better than an inexperienced player.

Board with a random position: There is no correlation between the pieces, as they are generated using a random board generator. The pieces may be placed illegally, and appear to defy logic according to commonly accepted strategies in the game. The purpose of the random position is to see if there is a difference between the level of recall of experienced players compared to casual, and novice players.

Place: Since subjects will be from different places, it will be difficult to control the atmosphere of individual places. Basic requirements will be quiet, good lighting. Noise level is to remain fairly quiet, similar to a chess-playing environment with some background noise, such as people talking.

Equipment (chess sets): Generic chess set with green for black squares, and white for white squares. Black and white plastic pieces. A similar type of set used for everyone.

Time: Each person receives 30 seconds to see each position, and 2 minutes to set it up.

Strength of players:

  • Experienced players: Study and/or play chess on a regular, almost daily basis. These people should be able to easily recall the given actual position, whether it is because they have seen it before, or because they can remember the general ideas of the position.
  • Casual players: They know the rules, and have played a couple of games or more. Know some basic ideas about where to place piece.
  • Novice players: They know the names of the individual pieces and basic rules of how they can legally move but have no strategies.

Confounding variables:

  • Age: It is commonly accepted that younger people are better at retaining information than older people. This is more likely to affect subjects who do not have a rating and thus cannot evaluate their strength numerically.
  • Distractions/motivation: People who are determined to do better will likely be more focused. Sudden loud noises may disrupt concentration.

For reference, the three normal positions are:

Click or tap to enlarge

The same procedure applied to the random positions as for the normal position. The three random positions:

Data

Experienced players

Subject's Rating 1st normal pos. 2nd normal pos. 3rd normal pos. 1st random pos. 2nd random pos. 3rd random pos.
2356 30/30 22/24 16/17 21/30 13/24 9/17
2141 25/30 12/24 12/17 17/30 10/24 8/17
2206 30/30 18/24 16/17 11/30 4/24 7/17
2087 22/30 15/24 10/17 9/30 0/24 6/17
2100 19/30 13/24 16/17 15/30 14/24 5/17
1833 21/30 12/24 9/17 6/30 2/24 8/17
1752 14/30 12/24 8/17 6/30 5/24 4/17
2262 29/30 20/24 12/17 20/30 20/24 17/17

Table 1. Number of pieces placed correctly by experienced players

Casual players

Subject Number 1st normal pos. 2nd normal pos. 3rd normal pos. 1st random pos. 2nd random pos. 3rd random pos.
1 (1200) 15/30 2/24 7/17 3/30 2/24 4/17
(1600 chess.com) 6/30 7/24 9/17 7/30 7/24 3/17
3 11/30 12/24 6/17 6/30 9/24 4/17
4 16/30 9/24 7/17 8/30 5/24 4/17
5 14/30 7/24 7/17 5/30 3/24 4/17

Table 2. Number of pieces placed correctly by casual players

Novice players

Subject Name 1st normal pos. 2nd normal pos. 3rd normal pos. 1st random pos. 2nd random pos. 3rd random pos.
6 13/30 1/24 6/17 5/30 1/24 0/17
7 15/30 10/24 8/17 7/30 13/24 4/17
8 6/30 2/24 6/17 11/30 4/24 0/17
9 12/30 7/24 5/17 10/30 8/24 2/17
10 8/30 3/24 8/17 6/30 4/24 1/17

Table 3. Number of pieces placed correctly by novice players

Processed Data: Experienced

Subject's Rating 1st normal pos. 2nd normal pos. 3rd normal pos. 1st random pos. 2nd random pos. 3rd random pos.
2356 100% 91.7% 94.1% 70% 54.2% 52.9%
2141 83.3% 50% 70.6% 56.7% 41.7% 47.1%
2206 100% 75% 94.1% 36.7% 16.7% 41.2%
2087 73.3% 62.5% 58.8% 30% 0% 35.3%
2100 63.3% 54.2% 94.1% 50% 58.3% 29.4%
1833 70% 50% 52.9% 20% 8.33% 47.0%
1752 46.7% 50% 47.1% 20% 20.8% 23.5%
2262 96.6% 83.3% 70.1% 66% 83.3% 100%
Average % 69.98% 60.00% 64.60% 37.20% 34.15% 47.04%

Table 4. Percentage of pieces placed correctly by experienced players

Casual

Subject Number 1st normal pos. 2nd normal pos. 3rd normal pos. 1st random pos. 2nd random pos. 3rd random pos.
1 50% 8.33% 41.2% 10% 8.33% 23.5%
2 20% 29.2% 52.9% 23.3% 29.2% 17.6%
3 36.7% 50% 35.3% 20% 37.5% 23.5%
4 53.3% 37.5% 41.2% 26.7% 20.8% 23.5%
5 46.7% 29.2% 41.2% 16.7% 12.5% 23.5%
Average % 41.34% 30.85% 42.36% 19.34% 21.67% 22.32%

Table 5. Percentage of pieces placed correctly by casual players

Novice

Subject Name 1st normal pos. 2nd normal pos. 3rd normal pos. 1st random pos. 2nd random pos. 3rd random pos.
6 43.3% 4.17% 35.3% 16.6% 4.17% 0%
7 50% 41.7% 47.1% 23.3% 54.2% 23.5%
8 20% 8.33% 35.3% 36.7% 16.7% 0%
9 40% 29.2% 29.4% 33.3% 33.3% 11.8%
10 26.7% 12.5% 47.1% 20% 16.7% 5.88%
Average % 36.00% 19.18% 38.84% 26% 25% 8.24%

Table 6. Percentage of pieces placed correctly by novice players

Qualitative observations

Subject Notes
2356 Also plays competitive Go, focused. Student (17).
2141 Used to play a lot of chess. Middle-aged, has had surgery to remove tumour in brain.
2206 Actively studies chess, plays frequently. Self-motivated to do well. Middle-aged.
2087 Elderly (75+)
2100 Subject said he had a ?long day? ? possibly tired. Has not been active in chess recently.
1833 fairly elderly
1752 Plays chess online frequently.
2262 Actively studies chess, plays frequently. Student (15). Confidently used less than 30 seconds for some positions.
1 Fairly elderly, does not play much chess anymore.
2 Has only played chess online. Has no official Elo rating,
3 Claims positions are difficult to remember. Student (17/18)
4 Relaxed. Student (17/18)
5 Student (17/18)
6 Excited to participate. Student (17/18)
7 Complains positions are difficult to remember. Student (17/18)
8 Liked to get up and look at board from different angles. Student (17/18)
9 Started memorizing the coordinates on the board to locate pieces. Student (17/18)
10 Wished to score well in order to compare to friend. Student (17/18)

Averages are calculated by using the formula:

All average calculations were done using the Excel program.

Standard Deviation

To calculate standard deviation, the excel program was used. In statistics, standard deviation (s) is a measure of the dispersion of the data from the mean. It is important for this lab to see how varied the data can be, and whether or not it is consistent. It is expressed as:

Subject Standard Deviation
Experienced 25.5%
Casual 13.4%
Novice 15.7%

Table 8. Standard deviations for each type of player.

Significance: To test the significance of the data, a paired t-test was used to compare means in which percentages are, so that observations in one sample can be paired with observations in the other sample.

Group   Group One     Group Two  
Mean 0.67487 0.25542
SD 0.23273 0.1572
SEM 0.04249 0.0287
N 30         30  

Table 9. Intermediate values used in calculations for paired t-test between experienced and novice players.

Using an online paired t-test calculator, it was deemed that between experienced and novice subjects the data is extremely statistically significant; with a two-tailed P value of less than 0.0001.

Significance between experienced and casual subjects gave a two-tailed P value less than 0.0001, considered to be extremely statistically significant. The significance between novice and casual subjects gave a two-tailed P value of 0.1589. Not considered to be statistically significant.

Analysis: It is very evident that there is a correlation between the strength in chess of a player and their ability to recall a position in chess using short-term memory. Using statistical analysis, it was determined experienced players are able to recall positions much better than casual and novice players. As one recalls, positions 1 to 3 are the normal positions, and positions 4 to 6 are the randomized positions.

Graph 1. Percentage of correct pieces based on position number


From the graph, one can tell that the three normal positions were remembered much better by experienced players than by casual and novice players. This is represented in the two-tailed P values arising from the paired t-test. As the two-tailed P value between experienced and novice subjects is less than 0.0001, it is considered significant. The same applies to the significance between experienced and casual players, meaning that overall, experienced players are evidently better at pattern recognition than novice and casual players. While it is visible from the graph that casual players did slightly better than novice players on normal positions, the two-tailed P value is only 0.1589 between novice and casual subjects, which is considered to be statistically insignificant.

Though it was hypothesized that all subjects should have close to the same score on the random positions, this was not the case. Everyone did worse on random positions, though experienced players still scored better, but not as much when compared to normal positions. This may be due to the fact that an experienced player is able to use logic to remember where the pieces are, using common chess knowledge such as which piece is being threatened to be captured by another. Compare this to subject 9 (a novice) in table 7, who attempted to use the coordinates (the letters and numbers located on the side of the board, visible in image 1 and 3) to remember where pieces are. This is inefficient, especially with a 30-second time cap for observing a position. A similar finding was determined by Bilalic et al, “with random chess positions, when pattern knowledge could not be used to guide perception, experts nevertheless maintained an advantage” (Bilalic, M., Langner, R., Erb, M., & Grodd, W. (2010)) There is evidently no proof that casual players and novice players can remember the positions any better compared to each other.

On normal positions, take for example this 2141-rated player who scored 12/17 on the third normal position, and compare the result to a novice who scored 6/17 on the same position.

If one compares the setup to the correct position below:

It is evident the positions are very similar. The number of pieces, though set up incorrectly, is the same, and it is evident while the black bishop and rook are switched, they are indeed on the board. In all the cases, experienced players had a much better grasp on the correct number of pieces and the pieces themselves (e.g. queen, king).

Example of a wrong position set up by a novice player

Some examples of problems include the pawns which are located on the wrong squares, the number of pieces being incorrect and black missing a rook. Pattern recognition has obviously come into play here, as experts understand the basic pawn structure (location of the pawn) and not have to commit it to memory but rather (speaking from personal experience) have a feel for where they are.

Looking at just the rated (experienced) players’ percentages of correctly placed pieces, some obvious trends can be noted.

Graph 2. Percentage of correctly placed pieces based on Elo rating


Similar what was determined from Graph 1, the stronger (the higher rated in this case) the player, the higher the percentage of correct pieces. For example, the highest rated player (2356) scored 100%, 91.7%, 94.1% on the first three normal positions, whereas the 2262 person scored 96.6%, 83.3%, 70.1%, which is evidently lower. There was more overlap on correct pieces in the random positions. However, the most significant anomaly is the 2262 rated person scoring 100% on the final random position, while everyone else’s was between 23.5% and 52.9%. Reasons for this could be that they found this specific randomized position easier to remember, because of the lack of pieces (called an endgame), and has studied a lot of endgames specifically.

Evaluation

Limitation Significance of limitation Suggested improvement
Systematic error: Definition of novice and casual players It is difficult to evaluate who is a novice and who is a casual player. Affected the data (considering the paired t-test value of 0.1589) slightly. One can choose to only use people with an Elo rating. From the experiment, it is evident that just using chess players is sufficient, as stronger players had better results. More test subjects with similar ratings can be used, with a greater range of ratings.
Systematic error: Time spent between looking at positions and setting up Since subjects had to turn around to reach the other board, ones who did it faster and immediately got a piece onto the board may have been able to get more pieces correct. Not particularly significant, but enough to raise the number of pieces by at least 1. Have subjects sit in a swivel chair, so they can turn around faster.
Random error: Level of motivation Some people were determined to do well (subject with rating 2206), while others did not care as much. There were also subjects who chose to close their eyes (and clear their mind?) before each position. Mentality has a lot to do with chess, and how people approach pattern recognition is related to how much effort they are willing to put in. Effort means concentration, which means those who wanted to do well theoretically scored higher than those who were just participating for fun. Have everyone be motivated. Similar to the first suggestion, one can choose to only use people with an Elo rating and give them an incentive such as a prize for the best scorer at the end in order to encourage everyone to try their best.
Random error: Mental limitations Age can affect mental functioning. Elderly may respond slower, such as for subject with rating 2087 who was more than 75 years old. Short-term memory may not function as well for them and would have thus meant they remembered the positions worse. Have players all of the same age. People of the same age have similar levels of brain functionality, and therefore pattern recognition should only depend on their skill in chess.

References

  1. de Groot, Adrianus Dingeman. Thought and Choice in Chess. Amsterdam University Press, 2008.
  2. Fukunaga, Keinosuke. Introduction to Statistical Pattern Recognition. Academic Press, 1972.
  3. Pavlidis, T. Structural Pattern Recognition. Springer, 1977.
  4. Weinstein, Yana. “Academic Resource Center.How Long Is Short-Term Memory? Shorter than You Might Think. Duke Academic Resource Center, 13 Apr. 2017.
  5. Bilalic, M., Langner, R., Erb, M., & Grodd, W. (2010). Mechanisms and neural basis of object and pattern recognition: A study with chess experts. Journal of Experimental Psychology: General, 139(4), 728-742.
  6. Anand at Accenture: How Memory Works in Chess.Chess News, 28 June 2012.
  7. Shier, Rosie. “Paired t-Tests.” Mathematics Learning Support Centre, 2004.

Review

We submitted Qiyu's paper to Dr. Vera Spillner, a mathematician, quantum physicist, polyglot and violinist whom we regularly consult on such matters. Vera wrote:

This is a creative and smart approach to a very complex scientific topic. In research, it is very important to simplify complex situations to get a feeling for correlations — which has been nicely done in Qiyu’s analysis. In my opinion, a good second step could now be to further test pattern recognition/short-term memory of chess players in non-chess situations.


Postscript

by Frederic Friedel

Almost 38 years ago, in the summer of 1980, I was making a science documentary for German television on computer chess. One of the sections was about how humans and how machines think. For the former we did a number of experiments, including position recognition, using methods similar to those described by Qiyu above.

The main subject of my experiments was a lad named Nigel Short, who had just turned 15 and was of full IM strength. He was staying in my house and playing in the Hamburg GM tournament. So it was natural that we used him — and the World Championship candidate GM András Adorjan — for our experiments. We showed these players meaningful chess positions for five seconds, and then made them reconstruct them on a second board.

WCh Candidate András Adorjan facing a 15-year-old chess prodigy, in Hamburg, back in 1980 | Photo: Frederic Friedel

András Adorjan and Helmut Pfleger (right) | Photo: Frederic Friedel

We also did some remarkable experiments to record the way GMs and amateurs scan a chess board. Above you see András solving a chess position while we track the movement of his eyes. Helping to conduct the procedure is GM Dr Helmut Pfleger, who has done a number of medical and cognitive experiments with chess players himself.

Now to my request: the science documentary, 43 minutes long, ran in 1980 as part of the series "Bilder aus der Wissenschaft", in the first national TV station, ARD, with Albrecht Fölsing as the anchor. It also featured the Computer Chess World Championship in Linz, Australia, which Belle won, as well as the very first experiment in cheating with machine assistance. This documentary has disappeared from all archives that I searched — and it is not on YouTube, where it definitely belongs. So: I would be most grateful if one of our readers could locate the footage, or if anyone has it on VHS tape, and would send it to us. There will be a very nice autographed program for the first reader to help us out.

In any case, there will be a follow-up article describing our 1980 experiments, with vivid descriptions and B/W photos, if the film material is not to be found.

Links


WGM Qiyu Zhou [pronounced Chee-you Jo], born in 2000, is a Canadian chess player who has competed for team Canada at the Women's Chess Olympiad since 2014 and who won the Canadian women's championship in 2016.

Discuss

Rules for reader comments

 
 

Not registered yet? Register