By Claus Holm

Quarterly has met a man on the up and up, standing tall as part of a minority. His name is Anders Holm and he is a professor of statistics at the Department of Educational Psychology at the School of Education, University of Aarhus.
"I am just a stupid statistician, you know", he describes himself coyly, smiling, while he explains the advantages of being a hard numbers guy and not a soft academic occupied with making qualitative interviews.

What is the problem of using qualitative methods as investigative interviews?

"The problem with the qualitative methods is that you cannot generalise based on them. As a statistician, I will have to ask how you can generalise based on only a few interviews. Recently a colleague said that less than five respondents in a survey would render the use of statistics impossible. You may choose to interpret that statement to mean that if you have less than five respondents, you will have to conduct a qualitative interview.

I think that is wrong. If you make a survey involving only five subjects, you cannot say anything about anyone else than the exact five subjects you have interviewed. Based on the theory of probability, certain statistical rules have been established for when and how generalisations can be made. And if you maintain the statistical paradigm, it means that there are clear and unmistakeable rules for when you can generalise a phenomenon to apply to an entire population. If you want to generalise, you have to count. There is no other way," says Anders Holm.

The regime of denumerability

Anders Holm's statements may not bring out a smile in the large majority of researchers within sociology and social science. Because they use qualitative methods. But he is definitely not the only one demanding the use of quantitative methods. It seems that we are heading for a situation where denumerability, as a scientific regime, is becoming increasingly prevalent.

Again. Actually, we need to go 50 years back in time, to the 1950s, when sociology was established as a science in the Scandinavian countries, to find a similarly favourable situation for the quantitatively-oriented sociology. Then, as now, sociologists needed to contribute to the social art of engineering. The scientific ideal was found in natural science, and sociology was dissociated from the humanities and historical research. Sociology was intended to provide knowledge about general mattes, not about unique cases.

In the 1960s, the situation changed. The use of quantitative methods met with criticism.  The planners did not get the information they needed. Instead, an alternative method of doing sociological research emerged in the mid-1960s. It came to be known as soft data or qualitative methods. At a certain point, the criticism and approach coincided with increased influence from Marxist theory. The consequence was that a number of qualitative methods saw the light of day: hermeneutics, phenomenology, grounded theory, ethnographic method and feminism were - and are - different, but share a severe criticism of positivism.

Here, in the 2000s, the trend is to provide larger scope for quantitative research again. In 2006, the Danish Social Science Research Council published the report Future Prospects of Danish Sociology (Dansk Sociologis muligheder) in which a group of experts, chaired by Heine Andersen, recommended that sociology should give high priority to the development of methods of quantitative data collection and analysis in the coming years.

The reason was that particularly quantitative empirical methods had occupied a relatively weak and marginal position in Danish sociological research for a number of years. In 2007, the Danish Evaluation Institute followed up with its evaluation of Sociology and social science at the Universities of Copenhagen, Aalborg and Roskilde.

The conclusion was i.a. that the students' competencies in quantitative methods needed strengthening at the Universities of Roskilde and Copenhagen. From 2007, the University of Copenhagen has offered a postgraduate study specialising in methods, but the report pointed out the basic problem that the supervisors on the programme at the University of Copenhagen use the qualitative method themselves. And consequently, they encourage their students to do the same in their projects.

From figments of the brain to numbers

The philosopher Karl R. Popper distinguished between the context of discovery and the context of justification. Anders Holm builds upon this distinction: "Many ideas which we want to study scientifically are based on feelings which we cannot quite explain, but that is precisely the purpose of social science; to find out whether these ideas represent merely figments of the brain," says Anders Holm.

In their article Life forms in Denmark: Prevalence and development trends 1981 - 2005 (Livsformer i Danmark: Udbredelse og udviklingstendenser 1981 - 2005) Anders Holm and his colleague Mads Meier Jæger set out to find out whether Thomas Højrup's idea about life forms - the wage earner's life form, a career life form and the self-employed life form - actually made sense in real life. They did so by checking whether there was any systematic connection between the position of individuals in terms of occupation and their work values.

Anders Holm explains:

"Højrup's life forms represent a good idea based on qualitative studies, which it is possible to examine in detail.
This means that Thomas Højrup has been talking to somebody and discovered something interesting. But further examination is required. Because Højrup's life forms are a purely Danish phenomenon. That alone should set the scientific alarm bells ringing. Very rarely do we come across something that is purely Danish. Well, perhaps with the exception of liver paste."

The study of the existence of the life forms actually showed that there was a noticeable connection between the occupational position of individuals and a wage earner's life form where the individuals attach importance to leisure rather than work, or a career life form where they attach importance to fulfilling themselves through their work.

"It is true that our study showed that Højrup's discovery was generalisable. But the study also showed that Højrup was wrong about something. Højrup has a thesis that the life forms are idiosyncratic with mutually exclusive attitude patterns. Our empirical study showed the opposite to be true: Individuals could be bearers of values from the wage earner's life form as well as of the career life form.

When we presented these findings at a research seminar, one researcher said that Højrup was not wrong merely because we could produce results to the contrary. I disagree, but there is a tedious tendency among soft academics to always want to agree with each other. This is a consequence of the fact that it is never possible to pin them to data which are antagonistic compared to their theses and assumptions," says Anders Holm and continues: "The case with economists and statisticians is different. They start from some assumptions and something logical follows from the process. This means that you can check whether we are right or wrong."

Use causality against tea leaves

According to Anders Holm, his basic problem with qualitative methods is that he does not know how to prove causal relations by means of these methods:

"I quite simply have not come across one single qualitative study which I could say to myself establishes a convincing causal relation. The funny thing is that when I talk to researchers who advocate the use of qualitative methods, they say:

'Now you have found some relations among your many data. Now what we need to do is ask people what the relations really are.'

I approach things the other way round: Now I have talked to some people, been inspired and now I need to use the quantitative methods to find out what causal relations actually exist here."

"I have an example of the type of absurd situation that can arise when you try to project a development based on qualitative data. In 1926, the German cultural researcher Sigfried Kracauer writes the article Two Planes, which first describes the light sides of Marseilles and then the dark sides by describing the city mortuary.

At some point in time, a professor from the Department of Sociology at the University of Copenhagen interpreted Sigfried Kracauer's descriptions as a premonition of Nazism. But if you read in the Danish encyclopaedia, Salomonsens Leksikon, the edition from 1924 that is, you can find the same text. This means that Sigfried Kracauer and Salomonsen are copying someone else - and Sigfried Kracauer did not predict anything at all.

The danger of qualitative studies is that they depend on us being put into a mood that may turn out to be nothing at all. And where it is impossible to establish any causal relation at all.Sometimes this tends towards telling fortunes by tea leaves."

But why are you so preoccupied with the issue of causality?

"If the world did not repeat itself in some way or other, then we might as well turn it off. How can we for instance make young people drink less alcohol? We do not know if they do so for a different reason tomorrow. Then we cannot intervene in a sensible way. If immigrants do not respond in a fairly similar way when they are received hostilely, then we might as well stop making scientific studies thereof. Then there is no reason for us to attempt to understand it."

One thing at a time

But is it possible to quantify complex social relations and make them subject to simple causality? Is this not exactly where social science differs fundamentally from scientific research?

"Social science is not what is complex. What is complex is the data available, which are full of all kinds of other things than what we want to examine them for. Nor are the social sciences preoccupied with things more complex than what preoccupies natural science. Physics is not monocausal either. There are thousands of matters that decide how long it will take for a plastic bag to degrade in nature.

This means that it is true that everything is complex, but then we as researchers must try to reduce the complexity and say something about one thing at a time. It is not possible to comprehend things in all their complexity. For instance, I am preoccupied with the impact the number of pupils in the classroom has on learning. It may very well be that their learning is also affected by other factors, but what I want to explain is the actual impact of the number of pupils in the class. I do not take a seat in the classroom and try to get a sense of the entire learning situation in all its complexity. Because that cannot be done, I would say."

About

Anders Holm, professor at the Department of Educational Psychology at the School of Education, University of Aarhus.

Quote

"Very rarely do we come across something that is purely Danish. Well, perhaps with the exception of liver paste.
If you want to generalise, you have to count. There is no other way".