Our Great Idea

Don’t expect too much from home office!

27 Nov 2020

How do we perform at cognitive tasks when we work from home? Not as good as in our usual work environment, if we take the quality of moves by world class players as an indicator, writes economist and study coauthor Christian Seel.

How do Fabiano Caruana, Ian Nepomniachtchi, Maxime Vachier-Lagrave, Anish Giri and Magnus Carlsen perform from their home office?
How do Fabiano Caruana, Ian Nepomniachtchi, Maxime Vachier-Lagrave, Anish Giri and Magnus Carlsen perform from their home office? (photo: chess24)

During the pandemic, I have been playing much more online than before. I was embarassed by the number of mistakes I made. Could it be that I perform so much worse when playing at home than over the board? Would that also be the case with top players?

I contacted two colleagues, who are experienced in empirical research and share my interest in chess: Stefan Künn, my colleague at Maastricht University and an Assistant Professor, and Dainis Zegners, Assistant Professor in the Department of Technology at the Rotterdam School of Management. We agreed that a comparison between moves played in a tournament hall and moves played from home could be an indicator of the quality of cognitive work done in the office versus the home office.

We compared the quality of moves in the Magnus Carlsen Invitational – the first online chess tournament with a prize fund comparable to a top live tournament – with the 2015–2019 editions of the World Rapid Championship. All of these tournament were played under the same time control (15 minutes+10 sec/move). Moreover, unlike in blitz, where the time to physically make a move compared to entering a move by mouse of touchpad on a screen might distort the effect, in rapid chess such differences are neglectable.

Comparing the quality of his play online and over the board inspired Christian Seel (pictured at a Bundesliga match in September) for this study.
Comparing the quality of his play online and over the board inspired Christian Seel (pictured at a Bundesliga match in September) for this study. (photo: private)

Using Stockfish on a depth of 25 ply (i.e. 13 moves by one side and 12 moves by the opponent side) we evaluated the quality of 27,000 moves made by the same elite chess players: Magnus Carlsen, Ding Liren, Anish Giri, Alireza Firouzja, Hikaru Nakamura, Ian Nepomniachtchi, and Maxime Vachier-Lagrave. Here we relied on a methodology which Dainis had developed and used in an earlier paper. It evaluates the quality of each move accounting for the current evaluation of the position, a measure of complexity of the position, and the remaining time on the clock. It was fair to assume that the players isolated themselves in their homes to have as little or less distractions as in a rapid world championship venue.

We found that the average error size (on a logarithmic scale) was economically and statistically significantly higher in the online tournament than in earlier over-the-board events. It is very unlikely that this occurred by chance. Even on an individual level, each of the seven mentioned players who competed in both, over-the-board and online, made larger errors when playing in the online event.

[Discussion paper][1] of the IZA Institute of Labor Economics 


  [1]: http://file:///C:/Users/journ/Downloads/SSRN-id3654928.pdf
Discussion paper of the IZA Institute of Labor Economics

This finding raises the question if the effect is permanent. These players are clearly accustomed to playing online, but were not priorily used to do it for significant amounts of money. As a first step, we analyzed whether the quality of play increased during the tournament, but found no evidence for that hypothesis. Yet, the timespan of the tournament does not allow for an evaluation how quality of play will change in the long run. Therefore we plan to analyze further online tournaments with the same time control. I can only speculate if the quality of play might improve with more experience and by how much. My hunch is that it will stay significantly below the over-the-board quality for most players or even all.

Why would we expect a decrease in playing strength online despite the high stakes environment? We cannot give a definitive answer within our framework. Yet, other studies have shown that the lack of peer pressure might lead to a decrease in performance. In this case the players were not exposed to see other fully concentrated players on neighbouring tables. It has been argued that the “normal” environment leads to more focus on the task. In our case a physical playing hall rather than the comfort and context of their appartment. Peter Heine Nielsen mentioned in an interview that he rented a house with Magnus Carlsen for later online tournaments in order to create more of a tournament atmosphere.

Other studies have shown that the lack of peer pressure might lead to a decrease in performance. In this case the players were not exposed to see other fully concentrated players on neighbouring tables.

Chess offers a unique setting for studying performance in a cognitive task, similar to those in many modern professional, managerial, technical, and creative occupations. For nearly all other cognitive job-related tasks, it is either difficult or very costly to measure the performance of an employee, or the task at hand also changes when it is moved from the regular workplace to the home office. Our research thus complements other studies which analyze the impact of home office on productivity of an employee, since most of these studies are based on self-reports.

[Christian Seel][1] is Associate Professor of Microeconomics and Public Economics at Maastricht University, an IM and first board player of the bundesliga club Aachener SV, and German Shogi champion 2012.


  [1]: https://sites.google.com/view/christianseel
Christian Seel is Associate Professor of Microeconomics and Public Economics at Maastricht University, an IM and first board player of the bundesliga club Aachener SV, and German Shogi champion 2012. (photo: private)

Carrying over to a work setting, our findings imply that the very fact that in the home-office workers might not experience the same pressure as in a real-work environment, sitting together with co-workers or seeing clients face-to-face, can result in a drop in cognitive performance. For employers and employees there are many factors to consider when comparing the benefits of a permanent home office: Noise and distraction levels, costs of operating office space, costs and time of commuting, long-term consequences of home office on the well-being of employees. Our paper has established another factor that must be considered in this equation.

Christian Seel and Dainis Zegners will present in the session Chess data in scientific research at the online conference ChessTech 2020.