Case study research

I was quite pleased to hear about the new edition of The Sage Handbook of Qualitative Research. In particular, there is a chapter in there by Bent Flyvberg on Case Study research, which should be quite helpful, as it dispels many myths about case study research:

1. First misunderstanding: General, theoretical knowledge is more valuable than concrete case knowledge.
2. Second misunderstanding: One cannot generalize on the basis of an individual case; therefore, the case study cannot contribute to scientific development.
3. Third misunderstanding: The case study is most useful for generating hypotheses; that is, in the first stage of a total research process, while other methods are more suitable for hypotheses testing and theory building.
4. Fourth misunderstanding: The case study contains a bias toward verification, that is, a tendency to confirm the researcher’s preconceived notions.
5. Fifth misunderstanding: It is often difficult to summarize and develop general propositions and theories on the basis of specific case studies.

If you are contemplating research that might benefit from a case study approach, then I recommend this book most strongly.

Research methods in social sciences

One useful resource for learning about research methods in the social sciences is the Research Methods Knowledge Base. It provides a really helpful introduction to many of the basic concepts that are useful in defining research. Particularly useful is the section on the language of research, providing clear explanations for what is meant by phrases such as "unit of analysis". Have a look and see if you find it helpful.

Poorly designed surveys

I am intrigued by the increasing flow of surveys that come across my desk. I wonder how many practitioners subscribe to academic construction research journals? Worse, how do you write up a survey when you have not defined a sampling frame? What would you state about the population, the sample and the return rate? I recall recently seeing one correspondent getting quite irate with people in his industry because not enough people had completed his survey.

One recent survey was sent to the CNBR list from a research student of smoe kind. It was an interesting case in point. His first question, to an international list of construction academics is: “Have you ever worked on a construction project outside of the United States?” It was clear from the context of this question that he identifies overseas as being outside the United States. But if I had never worked overseas, I would have to answer yes to this question, since I only worked on projects at home, in UK. So, having answered yes to this question, because I have worked on projects outside the United States, the next screen asks a whole load of questions with drop-down options that make little sense to me, but clearly make a great deal of sense to the researcher.

I particularly wondered about the question “Have you worked in any country which was affected by terrorism?” I guess that the UK has been affected by terrorism – but is this really what the researcher is looking for, given that his next question asks if my family travelled with my while I worked abroad? Very confusing. Until this point I was answering questions about working on sites in the UK, a country that has been affected by terrorism (as has the USA). Now, when I think about a question about taking my family with me while I worked abroad, I have to think about when my family came with me when I travelled as an academic to countries that were not affected by terrorism. In other words, all of the questions about my working abroad elicit answers that are nothing to do with the survey. So I aborted it.

I think it is important that before sending surveys out to mailing lists, students and researchers should be encouraged to think about the traditional steps in designing surveys. I wonder what we are teaching our students that leads them to make so many errors in the design of a simple survey. There are so many good books on this topic, but, for now, here are a couple of web resources to help researchers to get to grips with the basics of survey design:

• http://www.statpac.com/surveys/
• http://iss.leeds.ac.uk/info/312/surveys/217/guide_to_the_design_of_questionnaires
• http://sysurvey.com/tips/arsham.htm
  (This last one has a long list of useful links at the end)

The reason that I am going into this detail and posting these links is because so many surveys we receive in papers submitted to Construction Management and Economics are so badly designed that they contribute nothing to our collective knowledge. I frequently think that these poorly designed surveys do nothing other than confirm what the researcher thought in the first place, and as such they are simply a waste of paper.

I look forward to the day when we see fewer badly designed surveys...

Finding out what is being published

There are some very useful resources for researchers that will inform you about what is being published in journals and conferences that you are interested in.

Zetoc Alert - provides access to the contents tables of the British Library's collection of 20,000 journals and 16,000 conference proceedings publihsed each year.

Current Contents - provides access to leading scholarly journals (I think all the ISI-listed journals) and about 7,000 websites.

Informaworld - provides access to all the Taylor & Francis and Informa journals and periodicals.

In each case, you can choose which journals you are interested in and you will automatically receive an alert the moment they publish an article. This is a good way of keeping track of what is being published, as it comes out.

Reviewing research papers

The text below originally appeared in an ARCOM Newsletter. I reproduce a revised version here to make it more accessible to new researchers who might not be combing through back-numbers of ARCOM newsletters at www.arcom.ac.uk

There are many things that contribute to the quality of a research paper. Any report of research begins with a review of the relevant body of literature, and there is excellent guidance for undertaking a literature review in Silverman (2000: 12) and in Rudestam and Newton (1998: 50-51). Based upon this guidance, this paper offers a personal view of what we should be looking for in research papers, with the aim of prompting wider debate.

In all papers, a structure is required and the argument should flow from one section to the next. Obviously, clear English should be used throughout and jargon should be avoided. Good papers will move from the general to the particular and begin with the context of the work, move through the statement of the problem being investigated, deal with the empirical and/or analytical aspects of the work, then develop the discussion and draw conclusions based upon what has been covered in the paper, relating these back to the original context of the work. Generally, papers will either develop theory or test theory. A paper that does neither will not add to the sum of knowledge and therefore will not fall into the category of a research paper. Issues connected with style, structure and presentation are dealt with extensively elsewhere in the literature (for example, Turk and Kirkman 1982) and there is no need to reiterate that guidance here, other than to state that the easiest questions can be the most difficult to answer: what have you done, why is it important and how have you gone about it?.

The focus of this article is on whether the material that is included in a paper is suitable for a research paper, rather than whether it is well-written. Silverman’s (2000) headings form an excellent basis for a discussion of what constitutes a good research paper:

• Conceptualization and theoretical basis of the work
• Analytical framework and hypotheses
• Research design
• Results and discussion
• Conclusions of the paper

Conceptualization and theoretical basis of the work

The first questions to ask about a paper are connected with the problem or issue being investigated. There should be a clear statement near the beginning of a paper explaining what problem the paper seeks to resolve. Authors often leave this until half way through the paper. Indeed, some leave it out altogether, perhaps assuming that it is self-evident or simply not realizing that although it is obvious to the author, a reader with no previous knowledge of the work only has the paper to go on.

Any serious piece of research will involve concepts that are specific to the issue being investigated, or to the investigative approach that has been taken. These need to be summarized at the very least, and explained if they are not common within the field of the target audience. This is not just a case of explaining the concepts related to the particular construction industry phenomena under investigation, but, more importantly, to identify the methodological basis of the work. Thus, a research paper is not place for “text-book” explanations. Of course, the nature of the investigation is inevitably connected to some issue of relevance, but, while it may seem heretical to some, it is not necessary for a piece of construction management research to be practically relevant in industry. A piece of research may hold relevance only for other researchers, but that should not detract from our judgment of its value.

In assessing a report of research, it is always helpful to be able to see clearly how the reported work builds upon previous work. There should be explicit connections to an existing body of knowledge or body of theory, although these may not reside in the literature of construction management. Indeed, it is helpful if there are references to bodies of research and knowledge outside our own “domain”, since ours is not an academic discipline in its own right, with its own research techniques and theories (Hughes, 1999). While there are some emerging strands of theory that are unique to construction management or construction economics, most research in this area builds upon theoretical models developed elsewhere in the social sciences. These connections must be identified in order to make clear where a particular piece of work is rooted and to ensure that we are not simply re-inventing theories and models that are well-known in more mainstream disciplines. Without such connections, we run the risk of consigning our research to an academic backwater. With such connections, we may even be able to influence developments in mainstream thinking. In determining the theoretical basis of a piece of research, it is useful to think about knowledge domains. At the very least, it might be helpful to think about where would you expect to find a particular piece of work in a library so that connections and antecedents are clear.

These issues are important because progress in our understanding often depends upon our ability to generalize from specific examples. One question I often ask research students when they are trying to make these connections is “what is the general class of problem of which your chosen topic is a specific example?”. Understanding this enables some kind of view to be developed about the extent to which findings might be generalized into a wider context. Thus, good papers will begin with what is well-known and move gradually deeper into the less well-known (Latour, 1987: 57).

Analytical framework and hypotheses

All research papers have an analytical framework. Unfortunately, it is not always clearly articulated. The extent to which a particular approach is authoritative is often judged in terms of where it has come from. Thus, further connections to the research literature should be expected in the passages describing the analytical framework. When this is done well, it helps to establish the credibility of the paper by showing the usefulness of the particular approach, or approaches that precede it. The articulation of the analytical framework helps in judging the usefulness of the research question. It explains how the concepts and theories are being applied in this particular case.

One perennial problem with research papers in our field is the question of whether there should be hypotheses. They are certainly not a pre-requisite for a good research paper. In fact, they may not belong at all. The question about whether there should be hypotheses is, perhaps, a wrong question. Their presence or absence depends upon the methodological stance of the research. It is not intended to enter into the methodological debate here, other than to point out the dangers of not understanding the methodological implications of different approaches to research (see, for example, Seymour and Rooke, 1995). Given one methodological stance, hypotheses may be irrelevant. Given another, they may be indispensable.

If there are hypotheses, they should be clearly stated. If there are no hypotheses, then this, of itself, is not a problem, but it should be clear whether the paper is a review, a case study, a contribution to theory development or some other type of study. Without clear articulation, the reader stands no chance of determining the value of the contribution. In the presence of hypotheses, the relationships between the main variables should be explicit and reasonable. They should be stated in a way that makes them testable and the results, no matter what they are, interpretable. If the research is not built on hypotheses, the significance of the paper’s contribution to the development of theory must be explained.

Research design

In undertaking research, there are many methods that can be used to find answers to questions. Some are more suitable than others. In answering certain types of question, one particular method may be very powerful, but the same method may be weak in dealing with other types of question. Therefore, the relevance of the methods of research will be judged in terms of their appropriateness to the nature of the question being asked. Similarly, the sensitivity of the methods must match the needs of the research question. A good paper will make clear the type of research design, perhaps by reference to earlier, similar studies from different regions, different industries or different disciplines.

The research must be focused on an appropriate unit of analysis. It is useful to describe the criteria by which this was chosen, as well as the criteria by which the cases were chosen. For example, the unit of analysis could be a person, a finished building, a project, a firm, an industry or a country. Each would result in an entirely different study from the others. Moreover, cases might be selected from a large number of similar cases, which would imply one kind of approach, or the question might be framed in such a way that there is only one case, implying an entirely different approach. Neither, of itself, is more or less valid than the other. Indeed no judgement can be made about the validity of a piece of research simply by counting the cases or referring to the unit of analysis. Each characteristic depends on the other.

It is always important to address whether the research design isolates what is being measured from other effects, or, at the very least, identifies the inter-relationships between the effect under scrutiny and other effects. If the research design involves the identification of variables, they need to be clearly and reasonably operationalized (i.e. translated into simple descriptions of what is measured and how it is to be measured) and the reliability and validity of the measures should be discussed. Similarly, there will be issues related to the appropriateness of the population for the research question being studied, the sample size used and the extent to which the results can reasonably be generalized on the basis of this particular sample.

Again, not all research is as deterministic as this, but there are traditions in different types of work and if a phenomenological or ethnographic approach is being adopted, then the author should take this stance clearly and confidently and not try to dress it up in hypothetico-deductive clothes! These issues are well-articulated by Johnson and Duberley (2000), who warn against the dangers of not dealing with the epistemological positions that are implicit in different approaches to empirical research. It is important to emphasize that none of us wishes be prescriptive about the kind of research that is encouraged in ARCOM. We welcome research outputs that add to our understanding of construction management issues. But, the relevance and appropriateness of research outputs can only be evaluated in the light of the epistemological background to the research and this is why it is important to discuss the design of the research.

Results and discussion

Within the research paper, the data or evidence of the field-work must be present in some guise. But there are limits on the length of papers, whether for conferences or for journals. It is inevitable that the data will not be reported in their entirety through these outlets. Thus, one technique is to describe what the data is like, rather than what it is. Perhaps sample sections of interview transcripts can be given; examples of diaries or other source documents can be used to illustrate the approach taken. The full record of the data can be maintained elsewhere, perhaps in a departmental library or on the internet, so that the interested reader can interrogate the data further.

In any event, there must be sufficient information within the paper itself for the reader to evaluate whether the data were appropriate for the study and whether the data collection and record keeping were systematic. Similarly, the validity and robustness of the results of the study will depend upon whether the analytical techniques were appropriate and adequately described. Most importantly, there should be reference to accepted procedures for analysis. This helps the reader to understand what kind of tradition there is in the particular kind of analysis and how such research is generally reported.

In assessing how systematic the analysis has been, one of the main ideas is to persuade the reader that if he or she were to have done the same things, then the same conclusions would have been reached (Latour, 1987). Again, it is important that this very statement implies a certain epistemological stance, so the researcher and the reader need to be clear about whether they are working from the same basis in coming to their views about the results and their discussion.

Conclusions

Conclusions can be the most difficult part of a paper to write, particularly if the context and research design have not been addressed properly in the first place. It is often the case that those who have the greatest difficulty writing conclusions, can trace their difficulties to poor research planning. When research is well planned, the conclusions become obvious from the work that has been reported.

No new facts should be introduced in the conclusions. The conclusions of the study should be consistent with the results of the analysis. Where there is no numerical analysis, the conclusions should be consistent with, and follow from, the development of the argument in the paper. I am not quite sure why, but many authors commence their conclusions with a summary of the paper. This is usually not required, if there is already a summary at the beginning.

The conclusions may be expected to reconsider the purpose of the research, summarize what was discovered and provide a discussion about the implications of the findings. In many cases, conclusions can be bolstered by considering whether there are alternative conclusions that are consistent with the data or arguments that have been presented. Also, it is useful to consider both theoretical and practical implications of the results. If the research has been properly contextualized at the beginning of the paper, the theoretical implications of the reported research can be adequately connected to the literature discussed there. It is usual to include limitations of the study and future research needs, to the extent that the research has revealed further gaps in our collective knowledge. The limitations of the study should be noted in terms of the parameters of the research and applicability of the findings. Authors sometimes misinterpret the purpose of a section on limitations of the work and attempt to indulge in soul-searching self-criticism, identifying faults in the execution and reporting of their own work. This is simply not required. The section on limitations should make clear that, for example, the conclusions do not apply to all construction activity in all places at all times. The approach taken in the research enables certain generalizations to be made, but what are they?

Conclusions can also be bolstered by including discussion of the evidence for and against the researcher's arguments and making a clear distinction between the data and their interpretation.

Finally…

Any research paper is capable of being summarized succinctly. Papers are expected to include an abstract or summary at the beginning, especially in the cases of conferences and journals, but this should be the last thing to be written! Although abstracts may be invited for conferences before the paper is written, the abstract of the final paper ought to be written after the paper is finished, summarizing the paper that has been produced, rather than the paper that was originally intended.

In evaluating a research paper, it must be possible for the evaluator to summarize the paper, indicating an overall assessment of the adequacy of the study for exploring the research problem and an overall assessment of the contribution of the study to this area of research. Authors would do well to bear this in mind when writing their papers, so that they can provide the relevant cues that will lead a reader to conclude exactly what the author has concluded. Sometimes, there is simply not sufficient information in the paper to enable such statements to be articulated. In such cases, the paper should be revised.

References

Hughes, W.P. (1999) Construction research: a field of application. Australian Institute of Building Papers, 9, 51-58.
Johnson, P. and Duberley, J. (2000) Understanding management research: an introduction to epistemology.
London: Sage.
Latour, B. (1987) Science in action: how to follow scientists and engineers through society. Cambridge, Mass.: Harvard University Press.
Rudestam, K.E. and Newton, R.R. (1992) Surviving your dissertation: a comprehensive guide to content and process. London: Sage.
Seymour, D. E. and Rooke, J. (1995) The culture of the industry and the culture of research. Construction Management and Economics, 13(6), 511-523.
Silverman, D. (2000) Doing qualitative research. London: Sage.
Turk, C. and Kirkman, J. (1982) Effective writing: improving scientific, technical and business communication. London: Spon.

Checklist of questions to keep in mind while reviewing a research paper:

1 Conceptualization and theoretical basis of the work

(a) What is the problem or issue being investigated?
(b) What are the major concepts, and how clearly are they defined/explained?
(c) Is the connection to an existing body of knowledge or theory clear?
(d) Is there some practical relevance in this work (research practice or industrial practice)?
(e) What is the theoretical basis of this work, i.e. knowledge domain, for example, where would you expect to find this work in a library (classification number)?

2 Analytical framework and hypotheses

(a) Is there a clearly stated research question? (It might not be a research paper, as such)
(b) Are there hypotheses? Are they clearly stated? If there are not hypotheses, is the paper a review, case study, contribution to theory development or some other type of study?
(c) If there are hypotheses, are the relationships between the main variables explicit and reasonable? If there are not hypotheses, is there adequate development of theory?
(d) If there are hypotheses, are they stated in a way that makes them testable and the results, no matter what they are, interpretable? If there are not hypotheses, are there clear indications as to the significance to theoretical development?

3 Research design

(a) Are the methods of research appropriate to the nature of the question being asked?
(b) What is the type of research design?
(c) Could the design be improved? How?
(d) Is there a clear account of the criteria used for selecting the focus (unit) of analysis and the cases chosen?
(e) Does the research design isolate what is being measured from other effects? Are the variables clearly and reasonably operationalized (what is measured and how)? Are the reliability and validity of the measures discussed?
(f) Is the population appropriate for the research question being studied? Is the sample specified and appropriate? Can the results be reasonably generalized on the basis of this sample?

4 Results and discussion

(a) Are the data appropriate for the study? Was the data collection and record keeping systematic?
(b) Are the statistical techniques appropriate and adequately described? Is reference made to accepted procedures for analysis?
(c) Are the control variables adequately handled in the data analysis? Are there other control variables that were not considered but should have been?
(d) How systematic is the analysis?
(e) Is there adequate discussion of how themes, concepts and categories were derived from the data?

5 Conclusions

(a) Do the conclusions flow from the work that has been reported?
(b) Are the conclusions of the study consistent with the results of the analysis? (If there is no numerical analysis, are the conclusions consistent with the development of the argument in the paper?)
(c) Are alternative conclusions that are consistent with the data discussed and accounted for?
(d) Are the theoretical and practical implications of the results adequately discussed? Are the theoretical implications adequately connected to the literature discussed at the beginning of the paper?
(e) Are the limitations of the study noted (in terms of parameters of the research and applicability of the findings)?
(f) Is there adequate discussion of the evidence for and against the researcher's arguments?
(g) Is a clear distinction made between the data and their interpretation?

6 Summary

(a) What is your overall assessment of the adequacy of the study for exploring the research problem?
(b) What is your overall assessment of the contribution of the study to this area of research?

References

Rudestam, K.E. and Newton, R.R. (2007) Surviving your dissertation: a comprehensive guide to content and process. 3rd ed. London: Sage
Silverman, D. (2009) Doing qualitative research. 3rd ed. London: Sage.