The Semantic Theory of Survey Response

Words like “leadership” and “motivation” are constructs. It means that they are constructed – we invented them. Just like constructions (or constructs) such as shareholding companies, car brands, and justice, these constructs are only real if we act as if they are. The facts that they are constructed does not mean that such phenomena are nonsense. To the contrary, like most other inventions they serve useful purposes. They still remain linguistic constructions, though. We can now show that questionnaires, frequently used in research on leadership, tend to be predictable before we ask people to fill them out. This is because they mostly tell us about how people talk about leadership and motivation, and not so much about what actually happens in practice. Through the use of digital text algorithms, we can now show how these words are constructed, and how the research on leadership tends to be research on language more than anything else.

Most of these research articles are published as Open Access, which means that you may download them for free. Here are a few of them if you want to read more about it:

 

Can we trust what surveys tell us about leadership? Around the year 2012, my friend Kai Larsen and I started wondering about the data stemming from Likert-scale surveys. In 2014, we published this article, demonstrating how most survey studies on leadership are picking up self-evident data patterns. The relationships in the data are given a priori through language. It means that we can use computers to predict what people will answer. Here is the publication:

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0106361

After publishing this, we could demonstrate another weird phenomenon as well. If the structures in the data can be known in advance – before asking anyone – it should be possible to guess what people will answer before they actually do! This is a bit more complicated but in the article that follows, we have whown how it could principally be possible. If we know a person’s first answers to a survey, we can use semantic algorithms to guess pretty well what the rest of the answers might be. The article is here:

https://journals.sagepub.com/doi/abs/10.1177/2158244018764803

If the text algorithms can predict the data structures in one language, it is because the statistics simply reflect the meaning of the questions. Therefore, if the questionnaire is correctly translated (and the correlations are indeed due to semantics), the text algorithms will predict across languages. We tested this out among Chinese, Pakistanis, Norwegians, Germans, native English speakers and a bunch of other people and this is exactly what we found: The algorithms predicted the bulk of the statistics across all languages. There was virtually nothing left that could count as “culture”. Leadership surveys (and similar instruments) will be culture blind if they are based on semantic relationships:

https://www.frontiersin.org/articles/10.3389/fpsyg.2020.00176/full?

To the extent that survey results are predictable, it will be because of their embededness in language. We can therefore try to trace how constructs like leadership, motivation and results have emerged over the years and among groups of people. The following article published in 2018 showed how the development of workplace-related language also shapes responses to surveys on leadership:

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0207643

Our ideas about leadership are so strongy determined by langauge that we tend to expect things of leaders simply because of the associations that the words evoke. One funny (or ominous) effect of this is how we quickly will believe that leaders are a sort of heroes. Or that heroes also should be leaders. Both ideas lead to exaggerated expectations abut leaders. This in turn seems to make most people disappointed by real flesh-and-blood leaders. Our own bosses are usually diappointingly different from the linguistic stereotype. You can read about it in this article:

https://psykologisk.no/sp/2015/03/e3/

When people hold such exaggerated expectations for leaders, it should be no wonder that leadership development also can go awry. Here is a study we have spent years carrying out. It shows that leadership development can be not only wasted time – it can even be damaging to people’s health, even if that is luckily a rare outcome:

https://psykologisk.no/sp/2016/12/e18/