Questions, theories and lenses

I was asked an interesting question today: "In theory-testing research, which is common in our discipline of Construction Management, we try to find a theory from a mainstream discipline, and test it in CM to see whether it works. But, how does theory-building research work in an applied discipline like CM?" 

Immediately, I was struggling with how to answer this. I knew what he meant, but my stumbling was around the notion that CM is an academic discipline! My view is that it is not a discipline in the sense that, say, economics or law are disciplines. University departments are often confused with academic disciplines, but the way a university is organised has little to do with the way that knowledge is organised. Construction management, like any study of management, is essentially an empirically-based study. In writing journal papers, it is not simply a question of the authors casting around for some kind of theory to test. I suggest that we do not start from theory. As an empirical field, we start from practice. Presumably, research in an empirical field begins with a question, not with a theory.

We looked at a random journal paper to help in understanding this. The opening of a paper should explain what the question is and why it is important. That is, after all, the purpose of an introduction. Then the literature review helps us to figure what "kind" of question it is. It is the nature of the question that informs the way that theory is used. The review of previous research will help the author/reader in understanding why the particular question from the introduction is typical of a general group of questions. In a well-written paper, the literature review will lead us to a conclusion that our question is an illustration of a particular theoretical view. Every question implies a theoretical position. The purpose of a literature review is (mostly) to tease out what various theoretical positions are being used in dealing with question of the type we are suggesting in the paper. 

Questions are also connected to what kind of people ask the specific kind of question. This also takes us through specific strands of literature. Considering that questions arise from thought and action leads us to the idea that questions are based on observations of the world and, therefore, are essentially subjective. So the interesting thing is what are the qualifiers we need to put on a question to make clear the subjectivity of the questioner? I feel that it is acceptable if a question arises from a previous paper, especially since most papers cocnlude with limitations of the research and an indication of further research questions.

The key thing about the introduction is to identify whether there is a general phenomenon that the specific question of interest belongs to. That is why we cite literature in the introduction. It is not simply a question of proving that many people agree that it is a question! The random paper that we were looking at established, first, that a search on Google Scholar for one of the keywords provided 1.77 million results. That statement gave me sense of despair. If we insisted on the force of researcher-numbers to justify our questions, then progress would in the field would, perhaps, be both slow and dull. Knowledge is not a popularity contest! It doesn't matter how many people have mentioned a word in documents scanned by Google. If it did matter, then we would need to analyse the metrics, the algorithms and the alternatives for counting the number of incidence of the word on the internet. But that is an entirely different kind of study. It is not unimportant, but it had nothing to do with the question in the paper we looked at.

My advice was that the introduction does not need to justify the question by proving that a large number of people want to know the answer. Rather, it should show that the author's question is interesting and important, with a clear idea about what kind of question it is and what kind of theoretical position is implied by this kind of question. That sets the scene for the literature review that categorises and explores groups of papers that are themed around theoretical positions and types of question. It is not "testing the theory to see whether it works in construction". It is using a theoretical lens to study the specifics of an interesting question.

So, how does theory development work in an applied discipline like CM? The simple answer is that it works the same as in any other body of scientific literature! All theories are provisional and open to challenge. Science is known as a "provisional consensus that is always open to question". This is fundamentally about rationality, rather than quantification.

Susbequently, I was asked whether I could provide examples to show what good practice looks like in relation to the opening sections of a paper. So, I had a quick loook in the ARCOM CM Abstracts database and picked up a couple of papers:

• Emuze, F (2020) Factors causing fatigue and safety-related errors on Construction Sites In Bloemfontein. In: Scott, L and Neilson, C J (Eds.), Proceedings 36th Annual ARCOM Conference, 7-8 September 2020, UK, Association of Researchers in Construction Management, 215-224.
• Henjewele, C, Sun, M and Fewings, P (2008) Uncertainties and the challenge of long-term value for money in the delivery of PFI projects. In: Dainty, A (Ed.), Proceedings 24th Annual ARCOM Conference, 1-3 September 2008, Cardiff, UK. Association of Researchers in Construction Management, Vol. 1, 227–36.

(If you want to follow this, please click on each paper and open them in a separate window.) 

To begin with the paper by Emuze: The opening sentence of the introduction is excellent, Fatigue is a feeling of mental and physical exhaustion that leads to the inability to perform work effectively. The reason that I like this sentence so much is that it clearly locates the research topic and question within the experience of any reader. We all understand this. No references are required for stating something that is widely understood. Wisely, the authors gives none. We immediately know what the papers is going to be about. You might argue that the title tells us this, too. But as someone who has read thousands of journal and conference papers, I am tired of paper titles that do not match the paper! So this opening sentence gives me a great deal of confidence that I am safe hands with this author. The next two sentences each have a reference to research that has been carried out into such issues. This is immediately persuasive that the topic of Emuzi's paper isrecognised as important and relevant. Moreover, these references are to journals like Occupational Medicine and ournal of Occupational and Environmental Medicine. Again, this gives me confidence in the researcher's grasp of the topic. He is not trying to re-invent all of human knowledge from first principles becaause he is clearly aware that what is true for people in general must also be true for specific sub-groups of people, such as those in the construction sector. This is good. The first three paragraphs of the introduction identify the question and its importance, as well as the kind of science being used. This is all that is needed from an introduction. I especially like the use of reasoning at this point of the paper, rather then strongs of authors' names at the end of each sentence. If this were a journal paper, rather than a conference paper, the final two paragraphs of the introduction would be the beginning of the literature review section. I am not complaining about where the headings appear, of course. What matters if the sequence and flow of ideas. With the restrictions on the length of ARCOM Conference papers, reporting new research or research in progress requires the focus to be on methods, data and findings. So the last two paragraphs of the introduction satisfactorily explain the underlying concepts that are to be used in the research. What I like about the beginning of this paper is that we very quickly get the picture about the topic, its importance in the world, the specifics of data collection in relation to geography and industry sector, and the scientific basis on occupational health and its impact on errors and mistakes.

In the paper by Henjewele: Again, the opening sentence does a great job of making absolutely clear what this paper is about: PFI advocates itself as government’s best mechanism through which the public sector gets a high degree of certainty in long-term value for money (vfm) objectives in collaboration with the private sector through sharing of competencies and transfer of risks. It includes a reference to a UK Government document from HM Treasury (although, frustratingly, with an acronym of HMT, and with an incomlpete reference in the list of references). The remainder of the first paragraph uses reasoning, rather than references, to develop the argument that underpins the paper, which is really nicely done. The next three sections each have their own heading and together build up the concepts that are used in the research in way that makes sense and leads directly to a section on methods (although not methodology, as the heading indicates; see "What is the difference between method and methodology").

Both of these papers illustrate how to make clear the linkage between question, concepts and methods. They both show what "kind" of question they are dealing with. They are good examples of how the authors make conscious choices about what kind of question they are dealing with. They could just as easily looked at the phenomenon they discuss with a different theoretical lens. The theoretical lens is not a characteristic of the question, but of the researcher. It is a subjective choice informed by the successes and failures of past research in terms of what may or may not provide a useful explanation for the problems in focus, and useful findings that might improve the situation in some way. Thus, not only is the connection to existing knowledge made clear, they do not assume that all the answers lie in the construction management literature. Rather, they identify the kind of science they are applying, and draw upon that to apply it to an empirical phenomenon that is specifically in construction. This way, their findings may be of relevance to the world outside of construction management, which is useful.

I hope that these two examples help those postgradaute students who are not entirely clear about the expectations they are being confronted with in their work.

Reviewing research papers

There are several posts in this blog about reviewing research papers, and it is a theme that constantly crops up, both in relation to refereeing a journal paper for an editor of a journal, an in relation to building up the elemental parts of a literature review. As I have said before, these are two distinctive activities.

Many students struggle to make critical evaluations of published papers, and there are many reasons why they might find this difficult at first. One way of approaching the task could be to try to focus on whether a paper is making an interesting and useful contribution to our collective understanding. I think that what we are looking for is signs that author of a paper is clever enough to provide a persuasive case. What I mean by this is to look for evidence in a paper that the author is knowledgeable, has a good understanding of the issues, can put together an argument and can make judgements.

In order to be convinced that an author is knowledgeable, I would look for evidence that he/she knows about the generalities and specifics of the topic. To be convinced that the author has an understanding, I would try to find evidence of different views being weighed in the balance and ideas being applied to examples, especially I would look for the use of metaphors and analogies. In thinking about whether the paper has a strong argument, I would look for the use of logic and the sequence of ideas. The language of argument is interesting, and this is closely related to grammar, vocabulary and logic. I do not find the use of jargon or inflated diction particularly compelling. But plain language in straightforward words is often an element of a good argument. It should be clear. Finally, in looking to see whether an author is making judgements, there should be evidence of discrimination in the kind of sources being relied on to form the foundations for the work, and judgement in the nature of the conclusions as well as the all of the choices about what to include and what to exclude in putting the paper together.

Some common errors frequently reveal weak papers and unclear thinking. Papers typically follow this structure:

• Abstract - summarizing the whole paper including the specific findings
• Introduction - explaining what the paper is about
• Literature review - dealing with what we collectively already know, specifically, but not exclusively, in terms of past research
• Possibly some empirical work - not all papers fit into the category of empirically-based research
• Analysis and/or discussion - empirically-based papers would separate the analysis from the data, then all papers would include discussion of what all this leads us to
• Conclusions - explaining what this all means, perhaps with an outline of the parameters of the research and how that restricts the application of the findings, and usually with suggestions about what certain groups of people might do differently as a result of this research, whether these groups are scholars, practitioners or policy-makers

In weak papers authors often get the sections mixed up, probably because a weak paper will not really lead to any conclusions. Thus, I often find that the conclusions only really summarize the paper, the abstract introduces, rather than summarizes, and the introduction jumps straight into a general literature review before the detailed and focused part of the literature. As a result, in my editorial capacity, I often suggest to authors that they should move the introduction section to the literature review, move the abstract to the introduction, move the conclusions to the abstract, and then try to write some proper conclusions that convey something about what all this means. If they cannot do that, then the work is not really good enough to be called research, I would venture to suggest.

It might be an interesting exercise to see if you can find a paper where the sections are mixed up in this way. It would certainly give new researchers the confidence to take a critical stance when it comes to reviewing papers for their literature review!

Unsanctioned authorship

I just sent this email to a scientific editors' email forum. If you are interested in commenting, please add your comments to this blog, and I can include them in a summary of the discussion:

I am interested in the phenomenon of authors adding co-authors to their papers, without first consulting the so-called co-authors. I get the feeling that this is something that is rare, but gradually increasing. I think that people might do it to make their papers look more authoritative, or to drive up citations by adding a highly-cited author. Clearly, it is bad practice and I was wondering what could be done to ensure that it does not happen.

My impression from the publishers is that their aim is to ensure that if it does happen, they are not culpable. They will have had the corresponding author sign a declaration that all of the authors are aware of the paper and are validly listed as co-authors. Yet this does not overcome the problem, it merely moves the liability for the problem, in my own view.

Is this something we should be worried about? Should we seek explicit acknowledgement from each author that they have indeed played an instrumental role in the production of the paper? Or is it sufficient to identify this as the sole responsibility of the corresponding author? I would very much like to hear your thoughts on this issue.

Literature searching

The most essential prerequisite for a successful piece of academic research is the establishment of what is already known about the topic. While it is important to be able to produce a critical review of a research paper (See "Reviewing Research Papers"), it is just as important to be able to string together all the bits of information from the individual readings into a coherent critique of what previous research has shown, and where we are up to, collectively, in our understanding of the issues.

In trying to develop pointers and hints to help students go about this important task, I recently came up the following summary about what is required:

• Catalogue: Keep track of every item you come across, and make a note of its status in your literature searching process, particularly whether it has been acquired, read, reviewed, critiqued, and so on.
• Categorize: Figure out for each item what keywords you would use to index it, in such a way that you can connect papers on similar topics. Include keywords about methods and data, not just about ideas, and develop your own view of the definitions of each term, always seeking to use dictionary definitions when they work, but defining terms more specifically where necessary. But dont re-invent the wheel! If some past researcher has a good definition, use it (and cite the source).
• Characterize: Figure out what each item is like, and explain in your notes what kind of paper it is, and the basic characteristics of the type of research and type of question. Make connections with the literature about research methodology.
• Conceptualize: Develop a framework of the broad concepts that emerge as being the basic buildings blocks of knowledge in your topic, as represented by the literature you have discovered.
  
• Recognize themes, resonances and contradictions: Look for overlaps and clashes, erecting new keywords as you go, and checking each time whether a new keyword should have been used as well for the papers you already looked at. Find out where past researchers are in agreement, and where there is controversy.
  
• Identify areas of interest: As you go through these steps, you will inevitably develop a clearer understanding of not only what has already been done, but what remains to be discovered or challenged.

I hope that these notes help researchers to develop their approach to literature searching and to structuring a logical and rational narrative about the research base that they are building on in their work.

Multilingual editorial guidelines for scientists and translators (Press release)

The European Association of Science Editors (EASE) has recently published simple editorial guidelines for authors and translators of scientific articles (http://www.ease.org.uk/guidelines/index.shtml).This project is aimed to make international scientific communication more efficient and to prevent scientific misconduct. The guidelines advise on how to write complete, concise and clear (=understandable) manuscripts, and explain what is regarded as scientific misconduct. If authors and translators follow the guidelines before submission, then their manuscripts will be more likely to be accepted for publication. The editorial process will probably also tend to be faster, so authors, translators, reviewers and editors will then save time. The guidelines also explain what is regarded as scientific misconduct and provide links to the flowcharts of the Committee on Publication Ethics (COPE).

The document is freely available as downloadable PDFs in English, French, Spanish, Chinese, Japanese, Korean, and we plan to add more translations (made by volunteers) in the near future. Moreover, the English version has appeared also in print, for distribution during some major scientific events, such as ESOF2010 in Turin and the AIDS Conference in Vienna. EASE nominated the guidelines for the ALPSP Award for Publishing Innovation 2010. Our association did not receive this award, but the ALPSP judges stated that the guidelines “meet a very real need and very much hope that EASE will be able to secure sufficient endorsement from editors for the guidelines to become a recognised standard.”

Double-blind reviewing

As previously noted in this blog, the practice of double-blind refereeing is often compromised by authors who cite their own previous work in their draft papers submitted for review.

It is in the author's interest to maintain their anonymity, and the main reason for doing this is to protect unknown authors from being dismissed out of hand. It is easy enough for an author to cite his or her own work as "Anonymous, Year" and to enter "Reference omitted for purposes of refereeing, to be inserted in the event of acceptance", or similar, in the list of references. If authors choose deliberately to reveal their identity, there are many other ways that they can do this, so we do not expend a lot of editorial time on trying to overcome this. It is rarely such an obvious method of revealing an author's identity that surely it cannot be accidental?

Why do CM researchers continue to cite Egan and Latham Reports?

Opening a research paper with citations to Latham and Egan is probably not a good idea. These were government-sponsored reports, not research projects, and they need to be viewed very cautiously as they are now quite dated, and the political and economic landscape in the UK has moved on significantly since they were published. They do not represent the state of the art in terms of knowledge: indeed, they never have. If they were to be cited, they would need to be critiqued, although they have been endlessly, so there is not much point citing them as a reason for carrying out research into how construction work is procured and managed. Rather, researchers should seek out the best of the research that has been carried out (internationally) where this point has been tested and proved. A journal research paper is not the right forum to take up an outdated rallying call for UK industry improvement!

Have a look at them and see if you agree:

• Latham Report 1994
• Egan Report 1998

Cacography

A referee for a paper in the journal that I edit used an unfamiliar word to describe the quality of a particularly problematic paper that we are dealing with: cacography. I was immediately intrigued by this word, appearing in this context, "The paper is not presented well with errors of typographies and cacographies...". A quick bit of internet searching reveals that this is a word with strong pedigree, deriving from the Greek kakos meaning bad and graphos meaning writing. The prefix is not connected with the Old English word cack which means excrement (a word that we used to use in its correct sense when we were kids, I was pleased to learn!) According to Michael Quinion, cacography seems to have emerged at the end of the 16th century in the sense of bad spelling, around the time that personal choices for spelling were becoming less acceptable with the development of standardized approaches due to the new technology of printing. In this sense, cacography was seen as the opposite of orthography which was the term for correct spelling. Subsequently, it came also to refer to bad handwriting, as an opposite of calligraphy, which refers to fine writing. My correspondent was using it to refer to poor sentence composition, as opposed to mere spelling mistakes. What a fine word.

Authorship

Some years ago, when I was modifying the ARCOM model paper layout to deal with issues of authorship, I suggested the following text: "Authorship should respect the rights of those involved in the production of the paper. The person who wrote most of the text should be the first-named author, even if this is a student. The sequence of authors should reflect the magnitude of each person’s contribution to the text of the particular paper. Supervisors, grant-holders and heads of department should not automatically be added as authors unless they took part in the writing of the paper. If a junior person wrote the paper, and a senior person helped with the editing, structuring and drafting, the senior person should be acknowledged in the acknowledgements, but merely helping to guide someone through the writing process does not warrant authorship."

A colleague on the ARCOM committee responded, thus: "I cannot come to terms with your recommendation to add a text in the Model Paper about the Authorship. I think it is too prescriptive and does not recognize the variety of possibilities that might well be legitimate. There are a diversity of efforts that go into undertaking a research and converting the results into papers. To reduce them into "the person who wrote the most of the text" is an understatement. Obviously there are certain practices that are not acceptable: not reading the paper on which your name comes first is on that radar and plagiarism is at the heart of that radar."

My response was: "I see what you mean. I agree that the world is more complex than the monochrome vision that I paint. And, yes, I agree that I am overstating the case. That is what I usually do! My reason for habitually overstating the case is that it makes it easier to explain, and easier to disagree with. Your rejoinder makes me realize that I should precede my suggested words by making it clear that they are only suggested guidelines which may result in a different approach for particular authors at particular times, but which would at least prompt an informed discussion between authors. My motivation is to empower junior authors to actually have this discussion with their senior co-authors. If they decide that this suggested policy is not appropriate in their case, then OK. I know it sounds prescriptive, but I did not mean to produce a set of rules, so, yes, I should re-phrase it. I wanted to provide an indication of what would be equitable, so that authors did not have to wait until they were more senior to become first authors, by which time they were no longer the primary progenitors of the text!"

Interestingly, none of this text is present in the guidelines now, so we must have got distracted by other matters. I still think that it is important to be clear about authorship of papers, and I still find that it is quite common for senior academics to have their names as authors despite sometimes having little or no involvement in the development or drafting of the paper.

The issue is important, and I quite like the Wikipedia entry about authorship

Reviewing research papers

The text below originally appeared in an ARCOM Newsletter. I reproduce a revised version here to make it more accessible to new researchers who might not be combing through back-numbers of ARCOM newsletters at www.arcom.ac.uk

There are many things that contribute to the quality of a research paper. Any report of research begins with a review of the relevant body of literature, and there is excellent guidance for undertaking a literature review in Silverman (2000: 12) and in Rudestam and Newton (1998: 50-51). Based upon this guidance, this paper offers a personal view of what we should be looking for in research papers, with the aim of prompting wider debate.

In all papers, a structure is required and the argument should flow from one section to the next. Obviously, clear English should be used throughout and jargon should be avoided. Good papers will move from the general to the particular and begin with the context of the work, move through the statement of the problem being investigated, deal with the empirical and/or analytical aspects of the work, then develop the discussion and draw conclusions based upon what has been covered in the paper, relating these back to the original context of the work. Generally, papers will either develop theory or test theory. A paper that does neither will not add to the sum of knowledge and therefore will not fall into the category of a research paper. Issues connected with style, structure and presentation are dealt with extensively elsewhere in the literature (for example, Turk and Kirkman 1982) and there is no need to reiterate that guidance here, other than to state that the easiest questions can be the most difficult to answer: what have you done, why is it important and how have you gone about it?.

The focus of this article is on whether the material that is included in a paper is suitable for a research paper, rather than whether it is well-written. Silverman’s (2000) headings form an excellent basis for a discussion of what constitutes a good research paper:

• Conceptualization and theoretical basis of the work
• Analytical framework and hypotheses
• Research design
• Results and discussion
• Conclusions of the paper

Conceptualization and theoretical basis of the work

The first questions to ask about a paper are connected with the problem or issue being investigated. There should be a clear statement near the beginning of a paper explaining what problem the paper seeks to resolve. Authors often leave this until half way through the paper. Indeed, some leave it out altogether, perhaps assuming that it is self-evident or simply not realizing that although it is obvious to the author, a reader with no previous knowledge of the work only has the paper to go on.

Any serious piece of research will involve concepts that are specific to the issue being investigated, or to the investigative approach that has been taken. These need to be summarized at the very least, and explained if they are not common within the field of the target audience. This is not just a case of explaining the concepts related to the particular construction industry phenomena under investigation, but, more importantly, to identify the methodological basis of the work. Thus, a research paper is not place for “text-book” explanations. Of course, the nature of the investigation is inevitably connected to some issue of relevance, but, while it may seem heretical to some, it is not necessary for a piece of construction management research to be practically relevant in industry. A piece of research may hold relevance only for other researchers, but that should not detract from our judgment of its value.

In assessing a report of research, it is always helpful to be able to see clearly how the reported work builds upon previous work. There should be explicit connections to an existing body of knowledge or body of theory, although these may not reside in the literature of construction management. Indeed, it is helpful if there are references to bodies of research and knowledge outside our own “domain”, since ours is not an academic discipline in its own right, with its own research techniques and theories (Hughes, 1999). While there are some emerging strands of theory that are unique to construction management or construction economics, most research in this area builds upon theoretical models developed elsewhere in the social sciences. These connections must be identified in order to make clear where a particular piece of work is rooted and to ensure that we are not simply re-inventing theories and models that are well-known in more mainstream disciplines. Without such connections, we run the risk of consigning our research to an academic backwater. With such connections, we may even be able to influence developments in mainstream thinking. In determining the theoretical basis of a piece of research, it is useful to think about knowledge domains. At the very least, it might be helpful to think about where would you expect to find a particular piece of work in a library so that connections and antecedents are clear.

These issues are important because progress in our understanding often depends upon our ability to generalize from specific examples. One question I often ask research students when they are trying to make these connections is “what is the general class of problem of which your chosen topic is a specific example?”. Understanding this enables some kind of view to be developed about the extent to which findings might be generalized into a wider context. Thus, good papers will begin with what is well-known and move gradually deeper into the less well-known (Latour, 1987: 57).

Analytical framework and hypotheses

All research papers have an analytical framework. Unfortunately, it is not always clearly articulated. The extent to which a particular approach is authoritative is often judged in terms of where it has come from. Thus, further connections to the research literature should be expected in the passages describing the analytical framework. When this is done well, it helps to establish the credibility of the paper by showing the usefulness of the particular approach, or approaches that precede it. The articulation of the analytical framework helps in judging the usefulness of the research question. It explains how the concepts and theories are being applied in this particular case.

One perennial problem with research papers in our field is the question of whether there should be hypotheses. They are certainly not a pre-requisite for a good research paper. In fact, they may not belong at all. The question about whether there should be hypotheses is, perhaps, a wrong question. Their presence or absence depends upon the methodological stance of the research. It is not intended to enter into the methodological debate here, other than to point out the dangers of not understanding the methodological implications of different approaches to research (see, for example, Seymour and Rooke, 1995). Given one methodological stance, hypotheses may be irrelevant. Given another, they may be indispensable.

If there are hypotheses, they should be clearly stated. If there are no hypotheses, then this, of itself, is not a problem, but it should be clear whether the paper is a review, a case study, a contribution to theory development or some other type of study. Without clear articulation, the reader stands no chance of determining the value of the contribution. In the presence of hypotheses, the relationships between the main variables should be explicit and reasonable. They should be stated in a way that makes them testable and the results, no matter what they are, interpretable. If the research is not built on hypotheses, the significance of the paper’s contribution to the development of theory must be explained.

Research design

In undertaking research, there are many methods that can be used to find answers to questions. Some are more suitable than others. In answering certain types of question, one particular method may be very powerful, but the same method may be weak in dealing with other types of question. Therefore, the relevance of the methods of research will be judged in terms of their appropriateness to the nature of the question being asked. Similarly, the sensitivity of the methods must match the needs of the research question. A good paper will make clear the type of research design, perhaps by reference to earlier, similar studies from different regions, different industries or different disciplines.

The research must be focused on an appropriate unit of analysis. It is useful to describe the criteria by which this was chosen, as well as the criteria by which the cases were chosen. For example, the unit of analysis could be a person, a finished building, a project, a firm, an industry or a country. Each would result in an entirely different study from the others. Moreover, cases might be selected from a large number of similar cases, which would imply one kind of approach, or the question might be framed in such a way that there is only one case, implying an entirely different approach. Neither, of itself, is more or less valid than the other. Indeed no judgement can be made about the validity of a piece of research simply by counting the cases or referring to the unit of analysis. Each characteristic depends on the other.

It is always important to address whether the research design isolates what is being measured from other effects, or, at the very least, identifies the inter-relationships between the effect under scrutiny and other effects. If the research design involves the identification of variables, they need to be clearly and reasonably operationalized (i.e. translated into simple descriptions of what is measured and how it is to be measured) and the reliability and validity of the measures should be discussed. Similarly, there will be issues related to the appropriateness of the population for the research question being studied, the sample size used and the extent to which the results can reasonably be generalized on the basis of this particular sample.

Again, not all research is as deterministic as this, but there are traditions in different types of work and if a phenomenological or ethnographic approach is being adopted, then the author should take this stance clearly and confidently and not try to dress it up in hypothetico-deductive clothes! These issues are well-articulated by Johnson and Duberley (2000), who warn against the dangers of not dealing with the epistemological positions that are implicit in different approaches to empirical research. It is important to emphasize that none of us wishes be prescriptive about the kind of research that is encouraged in ARCOM. We welcome research outputs that add to our understanding of construction management issues. But, the relevance and appropriateness of research outputs can only be evaluated in the light of the epistemological background to the research and this is why it is important to discuss the design of the research.

Results and discussion

Within the research paper, the data or evidence of the field-work must be present in some guise. But there are limits on the length of papers, whether for conferences or for journals. It is inevitable that the data will not be reported in their entirety through these outlets. Thus, one technique is to describe what the data is like, rather than what it is. Perhaps sample sections of interview transcripts can be given; examples of diaries or other source documents can be used to illustrate the approach taken. The full record of the data can be maintained elsewhere, perhaps in a departmental library or on the internet, so that the interested reader can interrogate the data further.

In any event, there must be sufficient information within the paper itself for the reader to evaluate whether the data were appropriate for the study and whether the data collection and record keeping were systematic. Similarly, the validity and robustness of the results of the study will depend upon whether the analytical techniques were appropriate and adequately described. Most importantly, there should be reference to accepted procedures for analysis. This helps the reader to understand what kind of tradition there is in the particular kind of analysis and how such research is generally reported.

In assessing how systematic the analysis has been, one of the main ideas is to persuade the reader that if he or she were to have done the same things, then the same conclusions would have been reached (Latour, 1987). Again, it is important that this very statement implies a certain epistemological stance, so the researcher and the reader need to be clear about whether they are working from the same basis in coming to their views about the results and their discussion.

Conclusions

Conclusions can be the most difficult part of a paper to write, particularly if the context and research design have not been addressed properly in the first place. It is often the case that those who have the greatest difficulty writing conclusions, can trace their difficulties to poor research planning. When research is well planned, the conclusions become obvious from the work that has been reported.

No new facts should be introduced in the conclusions. The conclusions of the study should be consistent with the results of the analysis. Where there is no numerical analysis, the conclusions should be consistent with, and follow from, the development of the argument in the paper. I am not quite sure why, but many authors commence their conclusions with a summary of the paper. This is usually not required, if there is already a summary at the beginning.

The conclusions may be expected to reconsider the purpose of the research, summarize what was discovered and provide a discussion about the implications of the findings. In many cases, conclusions can be bolstered by considering whether there are alternative conclusions that are consistent with the data or arguments that have been presented. Also, it is useful to consider both theoretical and practical implications of the results. If the research has been properly contextualized at the beginning of the paper, the theoretical implications of the reported research can be adequately connected to the literature discussed there. It is usual to include limitations of the study and future research needs, to the extent that the research has revealed further gaps in our collective knowledge. The limitations of the study should be noted in terms of the parameters of the research and applicability of the findings. Authors sometimes misinterpret the purpose of a section on limitations of the work and attempt to indulge in soul-searching self-criticism, identifying faults in the execution and reporting of their own work. This is simply not required. The section on limitations should make clear that, for example, the conclusions do not apply to all construction activity in all places at all times. The approach taken in the research enables certain generalizations to be made, but what are they?

Conclusions can also be bolstered by including discussion of the evidence for and against the researcher's arguments and making a clear distinction between the data and their interpretation.

Finally…

Any research paper is capable of being summarized succinctly. Papers are expected to include an abstract or summary at the beginning, especially in the cases of conferences and journals, but this should be the last thing to be written! Although abstracts may be invited for conferences before the paper is written, the abstract of the final paper ought to be written after the paper is finished, summarizing the paper that has been produced, rather than the paper that was originally intended.

In evaluating a research paper, it must be possible for the evaluator to summarize the paper, indicating an overall assessment of the adequacy of the study for exploring the research problem and an overall assessment of the contribution of the study to this area of research. Authors would do well to bear this in mind when writing their papers, so that they can provide the relevant cues that will lead a reader to conclude exactly what the author has concluded. Sometimes, there is simply not sufficient information in the paper to enable such statements to be articulated. In such cases, the paper should be revised.

References

Hughes, W.P. (1999) Construction research: a field of application. Australian Institute of Building Papers, 9, 51-58.
Johnson, P. and Duberley, J. (2000) Understanding management research: an introduction to epistemology.
London: Sage.
Latour, B. (1987) Science in action: how to follow scientists and engineers through society. Cambridge, Mass.: Harvard University Press.
Rudestam, K.E. and Newton, R.R. (1992) Surviving your dissertation: a comprehensive guide to content and process. London: Sage.
Seymour, D. E. and Rooke, J. (1995) The culture of the industry and the culture of research. Construction Management and Economics, 13(6), 511-523.
Silverman, D. (2000) Doing qualitative research. London: Sage.
Turk, C. and Kirkman, J. (1982) Effective writing: improving scientific, technical and business communication. London: Spon.

Checklist of questions to keep in mind while reviewing a research paper:

1 Conceptualization and theoretical basis of the work

(a) What is the problem or issue being investigated?
(b) What are the major concepts, and how clearly are they defined/explained?
(c) Is the connection to an existing body of knowledge or theory clear?
(d) Is there some practical relevance in this work (research practice or industrial practice)?
(e) What is the theoretical basis of this work, i.e. knowledge domain, for example, where would you expect to find this work in a library (classification number)?

2 Analytical framework and hypotheses

(a) Is there a clearly stated research question? (It might not be a research paper, as such)
(b) Are there hypotheses? Are they clearly stated? If there are not hypotheses, is the paper a review, case study, contribution to theory development or some other type of study?
(c) If there are hypotheses, are the relationships between the main variables explicit and reasonable? If there are not hypotheses, is there adequate development of theory?
(d) If there are hypotheses, are they stated in a way that makes them testable and the results, no matter what they are, interpretable? If there are not hypotheses, are there clear indications as to the significance to theoretical development?

3 Research design

(a) Are the methods of research appropriate to the nature of the question being asked?
(b) What is the type of research design?
(c) Could the design be improved? How?
(d) Is there a clear account of the criteria used for selecting the focus (unit) of analysis and the cases chosen?
(e) Does the research design isolate what is being measured from other effects? Are the variables clearly and reasonably operationalized (what is measured and how)? Are the reliability and validity of the measures discussed?
(f) Is the population appropriate for the research question being studied? Is the sample specified and appropriate? Can the results be reasonably generalized on the basis of this sample?

4 Results and discussion

(a) Are the data appropriate for the study? Was the data collection and record keeping systematic?
(b) Are the statistical techniques appropriate and adequately described? Is reference made to accepted procedures for analysis?
(c) Are the control variables adequately handled in the data analysis? Are there other control variables that were not considered but should have been?
(d) How systematic is the analysis?
(e) Is there adequate discussion of how themes, concepts and categories were derived from the data?

5 Conclusions

(a) Do the conclusions flow from the work that has been reported?
(b) Are the conclusions of the study consistent with the results of the analysis? (If there is no numerical analysis, are the conclusions consistent with the development of the argument in the paper?)
(c) Are alternative conclusions that are consistent with the data discussed and accounted for?
(d) Are the theoretical and practical implications of the results adequately discussed? Are the theoretical implications adequately connected to the literature discussed at the beginning of the paper?
(e) Are the limitations of the study noted (in terms of parameters of the research and applicability of the findings)?
(f) Is there adequate discussion of the evidence for and against the researcher's arguments?
(g) Is a clear distinction made between the data and their interpretation?

6 Summary

(a) What is your overall assessment of the adequacy of the study for exploring the research problem?
(b) What is your overall assessment of the contribution of the study to this area of research?

References

Rudestam, K.E. and Newton, R.R. (2007) Surviving your dissertation: a comprehensive guide to content and process. 3rd ed. London: Sage
Silverman, D. (2009) Doing qualitative research. 3rd ed. London: Sage.

Remembering things

I wish I could remember things to write into the blog. There is so much going on, I don't know what to write about! Very strange.

Three papers

Friday morning was interesting. My first appointment was with Sam Laryea and we spent the best part of an hour discussing a draft of our paper for the Dubrovnik conference in the autumn. The paper, which is about standardization of procurement in construction, has to be submitted at the end of this month. We went through the draft together, and agreed some changes to the literature review and context, then looked at the table comparing various tendering methods, finally agreeing the basis of the conclusions. We will bounce the next draft off the British Standard committee with whom we are working. We meet next week, so we have to move quickly.

The second meeting was with Professor Said Boukendour, who is spending his sabbatical with us, from University of Quebec. We are working on a paper about a new way of arriving at a price for a Guaranteed Maximum Price (GMP) contract. We spent our time talking about the similarity that Said had noticed between these contracts and call options. In traditional GMP contract, the bidder is incentivized to push the GMP up. This will increase the contractor's profit because of the way that the difference between the outturn and the GMP is shared with the contractor. Competition between contractors would overcome this, except that the evidence is that GMP is usually a negotiation with a sole contractor. So, it the GMP is artifically high, then this sets up a low-risk, high-compensation deal which goes against the rational, economic approach. Perhaps this is an unintended consequence. Without competition, this situation can be overcome through open-book accounting, by enabling transparency. But this seems to be saying that even though GMP incentivizes the behaviours we wuold like to see, we still need to check the detail, being unable to trust the contractor to behave. Does this mean that GMP inherently fails to incentivise the contractor adequately? This was the issue that Said had dealt with in an earlier paper. I was thinking that if a contractor offered open-book, then this is a gesture of goodwill, whereas if a client insisted upon it, then it implies that GMP is not a sufficient incentive and is being reinforced with intrusive monitoring. Said's point is that if there was an effective financial incentive, then there would be no need for all the detailed analysis of open books. This was the thrust of his 2001 paper. Simplifying the construction situation, a GMP contract based on one lump-sum payment at the end of the contract is exactly the same as a call option in other types of market. This means that the GMP can be seen as a cost-plus contract with an option to switch to a lump-sum. This is interesting because there has already been a lot of work on call options. Using this understanding as basis, we are developing an approach to incentivizing construction contracts in a very effective way.

The third meeting was with Jan Hillig who had finished a very detailed edit of our chapter in a forthcoming book about procuring complex performance. Wisdom Kwawu had developed the initial draft from my outlinem and I'd thought the chapter was just about complete until Jan worked his magic and added some detailed sections on legal aspects as well as editing the rest of the chapter in considerable detail.

By the time I was walking over to a seminar with 20-odd industry people to launch our new Technology for Sustainable Built Environments, I was feeling that we'd had a very productive and enjoyable morning. What a team!

Multiple authorship

New Scientist, in its Feedback column of 18 July 1998, wondered how may scientists it takes to write a research paper. Their readers discovered some remarkable papers. One with 562 authors in Physics Letters B (vol 231, p 539). Then another with 596 authors in Nuclear Instruments and Methods in Physics Research (vol 289, p 35). 718 authors were found on one paper in The Canadian Journal of Cardiology (vol 12, p 127). But the clear winner appears to be something identified by the judges of the 1993 Ig Nobel Prize for literature. They deemed E. Topol, R. Califf, F. Van de Werf, P. W. Armstrong and their 972 coauthors (The New England Journal of Medicine, vol 329, p 673) worthy of the award for their achievement in "publishing a medical research paper which has one hundred times as many authors as pages". Can that record be beaten?

They also thought about which paper cited the largest number of institutions. One reader found 143 institutions listed in a single paper in The Lancet (vol 342, p 821).

And we thought multiple authorship was getting excessive!

Active vs passive voice

Some authors struggle with the problem of whether to use active or passive voice. I get the sense that they have a vague notion that the tradition in scientific writing is to write in the third person. This is manifest in their fear of using first person, or an active voice. I think of it as fear for two reasons. First, it's irrational. Second, they go to great lengths to avoid active voice or first person. I say it is irrational because when I ask them about it, they don't even understand what active or passive voice is. Just to clarify: if the verb relates to the person or thing, then we are using active voice, whereas if the verb is not connected to the doer, then passive voice is used. For example, in the preceding sentence, "use" is used in both voices, first actively then passively. The differences between first and third person is, perhaps, a little more straightforward.

I think that the fundamental problem is about using passive or active voice, and a lot of people seem to get very confused about this. Active voice is good, if you want to engage the reader. Passive voice is good if you want to stand to one side and look at the data, analysis and conclusions dispassionately. The confusion arises when authors pretend to change active to passive by changing "I" to "the writer" or "the author", which is just a clumsy way of revealing that they do not know what this active/passive thing is all about!