During the pandemic, I have been playing much more online than before. I was embarassed by the number of mistakes I made. Could it be that I perform so much worse when playing at home than over the board? Would that also be the case with top players?
I contacted two colleagues, who are experienced in empirical research and share my interest in chess: Stefan Künn, my colleague at Maastricht University and an Assistant Professor, and Dainis Zegners, Assistant Professor in the Department of Technology at the Rotterdam School of Management. We agreed that a comparison between moves played in a tournament hall and moves played from home could be an indicator of the quality of cognitive work done in the office versus the home office.
We compared the quality of moves in the Magnus Carlsen Invitational – the first online chess tournament with a prize fund comparable to a top live tournament – with the 2015–2019 editions of the World Rapid Championship. All of these tournament were played under the same time control (15 minutes+10 sec/move). Moreover, unlike in blitz, where the time to physically make a move compared to entering a move by mouse of touchpad on a screen might distort the effect, in rapid chess such differences are neglectable.
Using Stockfish on a depth of 25 ply (i.e. 13 moves by one side and 12 moves by the opponent side) we evaluated the quality of 27,000 moves made by the same elite chess players: Magnus Carlsen, Ding Liren, Anish Giri, Alireza Firouzja, Hikaru Nakamura, Ian Nepomniachtchi, and Maxime Vachier-Lagrave. Here we relied on a methodology which Dainis had developed and used in an earlier paper. It evaluates the quality of each move accounting for the current evaluation of the position, a measure of complexity of the position, and the remaining time on the clock. It was fair to assume that the players isolated themselves in their homes to have as little or less distractions as in a rapid world championship venue.
We found that the average error size (on a logarithmic scale) was economically and statistically significantly higher in the online tournament than in earlier over-the-board events. It is very unlikely that this occurred by chance. Even on an individual level, each of the seven mentioned players who competed in both, over-the-board and online, made larger errors when playing in the online event.
This finding raises the question if the effect is permanent. These players are clearly accustomed to playing online, but were not priorily used to do it for significant amounts of money. As a first step, we analyzed whether the quality of play increased during the tournament, but found no evidence for that hypothesis. Yet, the timespan of the tournament does not allow for an evaluation how quality of play will change in the long run. Therefore we plan to analyze further online tournaments with the same time control. I can only speculate if the quality of play might improve with more experience and by how much. My hunch is that it will stay significantly below the over-the-board quality for most players or even all.
Why would we expect a decrease in playing strength online despite the high stakes environment? We cannot give a definitive answer within our framework. Yet, other studies have shown that the lack of peer pressure might lead to a decrease in performance. In this case the players were not exposed to see other fully concentrated players on neighbouring tables. It has been argued that the “normal” environment leads to more focus on the task. In our case a physical playing hall rather than the comfort and context of their appartment. Peter Heine Nielsen mentioned in an interview that he rented a house with Magnus Carlsen for later online tournaments in order to create more of a tournament atmosphere.
Other studies have shown that the lack of peer pressure might lead to a decrease in performance. In this case the players were not exposed to see other fully concentrated players on neighbouring tables.
Chess offers a unique setting for studying performance in a cognitive task, similar to those in many modern professional, managerial, technical, and creative occupations. For nearly all other cognitive job-related tasks, it is either difficult or very costly to measure the performance of an employee, or the task at hand also changes when it is moved from the regular workplace to the home office. Our research thus complements other studies which analyze the impact of home office on productivity of an employee, since most of these studies are based on self-reports.
Carrying over to a work setting, our findings imply that the very fact that in the home-office workers might not experience the same pressure as in a real-work environment, sitting together with co-workers or seeing clients face-to-face, can result in a drop in cognitive performance. For employers and employees there are many factors to consider when comparing the benefits of a permanent home office: Noise and distraction levels, costs of operating office space, costs and time of commuting, long-term consequences of home office on the well-being of employees. Our paper has established another factor that must be considered in this equation.
Christian Seel and Dainis Zegners will present in the session Chess data in scientific research at the online conference ChessTech 2020.