Elo Meter - The test that calculates your Elo

When I was shown this site, my first thought was that this was a bit of a gimmick, a bit like the 'IQ tests' that crop up here and there, and where you congratulate yourself for being a 'genius'. Insert smiley here. However, the credentials are quite a bit more substantial, as are the supporting papers behind the theory.

What is Elometer

The home page is clean and clear. Start your motors!

In a nutshell, the idea is to answer a test set of 76 positions to the best of your ability, some of which have plain cut-and-dry answers, and others which are far more flexible and allow for a second-best or third-best move. Your second-best choice, if that be the case, will not be adjudicated as a fail, and your final rating will be adjusted accordingly. The positions vary in both difficulty and content, and though the initial positions are all completely elementary in nature, they soon ramp up the challenge after which they vary wildly, or such was my impression.

This is the first position, and as you can see, it is as easy as can be. This won't last long, but it does mean it is almost impossible not to get some right.

After you finish the test, they ask a number of questions regarding your conduct in the test, such as how often you play, how long you have played serious chess (30+ years for me, with caveats), how seriously you took the test, and of course whether you looked up the answers to any of the actual questions with an engine or database. I cannot see the point of doing that frankly, since the only person you'd be fooling is yourself.

Of the 76 positions, I recognized four. Two were actually pawn endgames I saw recently from Dvoretsky's Endgame Manual. 

The science behind the test set and how it arrives at its results is really the most fascinating part of it. The authors of the test are Birk Diedenhofen and Jochen Musch from the University of Duesseldorf, Institute of Experimental Psychology. After you have finished the test, and the small questionnaire, the authors explain:

We used item response theory (or "latent trait theory"; Hambleton, Swaminathan, & Rogers, 1991) to derive an estimate of your playing strength based on your answers to a set of chess problems with known properties. To arrive at this estimate, we employed the two-parameter Birnbaum model (Lord, 1980) which allows items to differ a) in difficulty and b) in discriminatory power. The set of chess problems we used was taken from the "Amsterdam Chess Test" developed by van der Maas & Wagenmakers (2005), who presented their chess problems to a sample of 259 participants at a Dutch open tournament. The national Elo rating of these participants ranged from 1169 to 2629. Using a subset of the items of this test (the Choose-A-Move item set A and B), we were able to compute a maximum likelihood estimate of your ELO rating based on a prediction formula regressing the latent ability estimates of the Birnbaum model on the ELO ratings of the comparison sample. Using the test information function, we were also able to compute a 95% confidence interval for this estimate.

What is Item Response Theory?

In psychometrics, item response theory (IRT) (also known as latent trait theory, strong true score theory, or modern mental test theory) is a paradigm for the design, analysis, and scoring of tests, questionnaires, and similar instruments measuring abilities, attitudes, or other variables.

It is based on the application of related mathematical models to testing data, and is often regarded as superior to classical test theory. It is also the science behind famous higher education tests such as the Graduate Record Examination (GRE) and the Graduate Management Admission Test (GMAT).

IRT is based on the idea that the probability of a correct/keyed response to an item is a mathematical function of person and item parameters. Examples include general intelligence or the strength of an attitude. Parameters on which items are characterized include their difficulty; discrimination (slope or correlation), as well as representing how steeply the rate of success of individuals varies with their ability. (source: Wikipedia)

How good is it?

This does beg the question of how accurate the estimates are. I absolutely don't question the sincerity and integrity of their efforts, but I do know I was last rated 2149 FIDE a bit over a decade ago, my last serious games, and though I recently retook training as shared in a previous article, "Study Chess with Me - the video series", good sense tells me the result I got is more Walter Mitty than reality.

Hard to know what to make of this, but I doubt I am alone

Nevertheless, it is obviously better to have it mistakenly telling me I am higher rated, than have it tell me I am lucky to break 1800, but still.... Insert second smiley here. 

Regardless of the obvious grain of salt it must be taken with, if you don't have any massive ego issues involved with such things, it is certainly a good workout that will take you a couple of hours, and is definitely fun. We're chess players after all, so solving positions is what we are all about, right?

Link to Elo Meter

If you wish to share in my study plan, feel free to join the series. I am certain that at the very least this has helped me produce a better result than without it.