Answers to Exercise

4.2 The Hawthorne experiment

Q1 What are the physical working conditions that contribute to increasing productivity of workers?

Q2 Research design includes incremental changes to supposed key variables; continual comparisons against predicted variable; and continuous observation.

Q3 The selection of workers is not random; there is no parallel control group; open observation takes place throughout, including interaction with the subjects.

4.3 Designing a call centre

Q1 From the list of general factors, is there a best combination? You would need to include cost in the equation – the best design might also be impossibly expensive.

Q2 You would have to start with any constraints on design and layout that are imposed by the proposed building itself. Then it makes sense to prioritize factors, perhaps on the basis of impact or cost. You might consider a sequence of studies which varies one factor at a time, in order to identify whether there is an optimal combination of design features. Some factors (such as density and layout) would be the same for everyone taking part in a particular study, while you could vary the ergonomics of the workstation for different workers within the same study.

4.4 Knowing about probability samples

Activity 22: Knowing about Probability – Catherine Dawson’s 100 Activities for Teaching Research Methods.

Example 1

This is a simple random sample that gives each member of the population an equal and known chance of being chosen. Using this procedure, a number is assigned to each element/individual in the study population. Random numbers are then generated (e.g. using a calculator, online random number generator, spreadsheet, or a printed table of random numbers) to select the required sample. This method requires an accurate list of the study population and is ideal for generating statistics.

An advantage is that each selection is independent of other selections and has an equal and independent chance of being selected. Problems can arise if it is difficult or impossible to identify every member of the population, or if members of the population are geographically widely dispersed (costly and time-consuming). Also, care must be taken when generating random numbers as some methods are open to bias, human error or software/hardware malfunction, for example.

Example 2

This is a cluster sample. This technique is used when it is impossible or impractical to compile an exhaustive list of all elements within the study population. Instead, the elements are grouped into subpopulations (already existing or created by the researcher) and then elements from each subpopulation are chosen using a simple random sample (used in this example) or a systematic random sample (described below).

A problem with this method is an overrepresented or underrepresented cluster in terms of certain characteristics. This can skew the results of the study. The technique is less precise than other types of probability sampling, but if a larger sample size is available (due to the practicality of travel, for example) this could offset the loss in precision.

Example 3

This is a systematic random sample (or quasi-random sample or interval random sample). This is a statistical method involving the selection of elements from an ordered study population. A starting point is chosen at random, with each subsequent selection made at regular intervals.

An advantage to this method is its simplicity; if carried out carefully, the population will be evenly sampled. However, a problem with this method is that it depends on how the list has been organized (alphabetically, for example). The researcher must ensure that a pattern, or periodic trait, is not hidden in the list as this will have an influence on randomness. Also, it is only the first selection of where to start the list that is a probability selection: there may be some units/elements that have a zero chance of being selected.

Example 4

This is a stratified random sample, which is a method of sampling that involves the division of a study population into smaller groups known as ‘strata’. These groups can differ in behaviour or the attribute under study. Once the different groups have been identified, members can be selected using a simple random sample or a systematic random sample, for example.

Advantages include convenience and cost, and stratified samples tend to be more representative than other probability samples, ensuring that elements from each stratum are represented. Also, different sampling techniques can be used within the strata (mixed methods approaches can overcome problems inherent in certain methods). Problems can arise in the identification of appropriate strata (timely and costly) and in analysing results. Misrepresentation of elements into the chosen strata can increase variability.

Example 5

This is a disproportionate stratified sample. With this type of sampling the sample size of each stratum does not have to be in proportion to the population size of the stratum. When disproportionate allocation is used, the data that have been gathered must be weighted, and this can lower precision. Other strengths and weakness of this method are similar to those described in Example 4.

4.5 Non-probability sampling

Discuss the reasons underlying the decisions to adopt a sampling strategy based on a) quota sampling, b) snowball sampling and c) purposive sampling. How do the authors of the studies you selected justify their choice?

Quota sampling is a type of non-probability sampling technique that allows researchers to target individuals they wish to sample based on certain known criteria or characteristics such as gender or educational level. The population is divided into groups (also called strata) and samples are taken from each group to meet a quota. This can ensure a proportionate representation of particular strata (groups) and facilitates comparisons between groups. Quota sampling is inexpensive, fast and easy to carry out and it tends to be used when probability sampling techniques are not a viable option. A major problem with quota sampling is the introduction of unknown sampling biases.

Snowball sampling is another type of non-probability sampling technique. It involves research participants recruiting other participants and is often use when potential participants are hard to find, for example when one seeks to sample concealed populations, or when higher levels of trust are required to initiate contact. It is called snowball sampling because once ‘you have the ball rolling’ (participants start recommending/recruiting other participants) your ball (i.e. project) will pick up more ‘snow’ and become larger and larger.

A purposive sample is a non-probability sample. The main goal of purposive sampling is to focus on particular characteristics of a population according to the needs of research question and emerging theory. For example, a researcher may look for critical cases, for maximum variation or for a very homogenous sample. Such sample is not representative of the population, but this may not be considered a weakness when conducting qualitative research. Researchers rely on their judgement when choosing research participants. Usually, the sample being investigated is quite small.

4.6 A longitudinal case study

Q1 The study involved gathering data from multiple sources over a period of time, and this was appropriate to studying the complex inter-relationships and processes within the company. But also the feedback to key actors in the company also added to the validity of the conclusions we drew.

Q2 Because this is a unique study, the findings can only be generalized theoretically. That is, the case study needs to be able to demonstrate something about dynamic capabilities, knowledge management, or the relationship between the two of them, which supports, contradicts, challenges or adds to current theories in the literature.

Q3 They do make sense if you want to publish the results of the study in an academic journal. However, there are plenty of stories that came out of the study which can provide useful anecdotes to illustrate decision-making processes, group dynamics and so on – which we had not expected in the first place, but which made good stories in their own right.

4.7 How grounded is this? An email from a doctoral student

Q1 This depends on whose version of grounded theory Suzanne follows. She uses open coding, which leads to a central category (see Chapter 7), but there is no theoretical sampling, nor saturation. The interplay between external theory/literature and data is more complex than as outlined in Glaser and Strauss.

Q2 We think this deviation is quite appropriate, provided Suzanne is able to articulate what and why. Reasons include the constraints of the research setting/access, the nature of the theory she is looking at, and the conscious combination of theory development and deduction.

4.9 Research design template

 

Epistemology

Strong positivist

Positivist

Constructionist

Strong constructionist

1

Background

What is the theoretical problem and what studies have been conducted to date?

What is the theoretical problem and what studies have been conducted to date?

What are the ongoing discussions among researchers and practitioners?

What are the ongoing discussions among researchers and practitioners?

2

Rationale

What is the main gap in existing knowledge?

What are the main variables, and how are they related to one another?

What perspectives have been covered and what are missing?

What are the limitations in the discussions so far?

3

Research aims

Specify testable hypotheses.

List main propositions or questions.

Identify the focal issue or question.

Explain how the research will add to the existing discussion.

4

Setting

Determine the wider population from which you will draw your sample.

Determine the research setting and the population from which you will draw your sample.

Identify an appropriate research setting and justify your choice.

Describe your research setting and justify that the methods you intend to use are appropriate to the setting.

5

Data

(see Chapters 6,7 and 9)

Define variables and determine measures.

Define dependent and independent variables and determine measures.

Explain and justify a range of data collection methods.

Identify main sources of data. How will interviews be recorded/transcribed, etc.?

6

Sampling

(see Chapters 4 and 9)

Explain how group selection and comparison will eliminate alternative explanations.

Justify sample size and explain how it reflects the wider population.

How will the sample enable different perspectives to be included?

Explain sampling strategy. Will it be opportunistic, emergent, comparative, etc.?

7

Access

(see Chapter 5)

How are experimental subjects to be recruited?

How can responses to questionnaires, etc., be assured?

What is the strategy for gaining access to individuals, organizations?

How will insights from co-researchers be combined?

8

Ethics

(see Chapter 5)

Is participation voluntary?

Could results be used to harm any participants?

Will the interests of individuals and organizations be protected?

How ‘open’ is the research? Will there be any deception?

9

Unit of analysis

Differentiate between control, experimental groups, etc.

Specify whether individuals, groups, events or organizations.

How will units/cases be compared with each other?

What are the entities that are to be compared with each other?

10

Analysis

(see Chapters 8, 10 and 11)

Statistical procedures for examining differences between groups.

Statistical procedures for examining relationships between variables.

Arrangements for coding, interpreting and making sense of data.

How will co-researchers be involved in sense-making?

11

Process

Explain stages in the research process.

Explain stages in the research process.

Explain what can be pre-planned and what can be open-ended.

Provide realistic timing including adequate provision for contingencies.

12

Practicalities

(see Chapter 5)

How will groups be recruited? Where will experiments take place?

Who will gather data? How will it be recorded/ stored? Who will analyse it?

How will researchers share observations? Who will do transcriptions, etc.?

How will co-researchers be engaged?

13

Theory

How will hypotheses be tested?

In what ways will the results add to existing theories?

Will the research build on existing theory or develop new concepts?

Will the research build on existing theory or develop new concepts?

14

Outputs (see Chapter 12)

Where will the research results be published?

What is the dissemination strategy?

What is the dissemination strategy?

How will insights be shared with colleagues and collaborators?

 

Consider the case of researchers conducting applied research or evaluation research. In order for them to use the template, would you add or omit any questions? If yes, which ones? Can you explain why?

Broadly, where the template presented is as useful for applied research and evaluation research as it is for basic research. However there are some differences in relation to emphasis and the relevance of some of the questions. Background, rationale and aims are clearly still very important although the objectives and deliverables and the value of the research to an organization come to the fore in applied and evaluation research. Also important is the setting or context in which the research takes place but here, it is because applied research and evaluation research places emphasis on practical outcomes and therefore relevance becomes closely related to particular contexts and these need to be additionally recognized. Many process consultants are keen to understand in detail the cultural historical context in which the research or evaluation is to take place so that the findings they present to clients have the best chance of gaining acceptance.

In terms of data, applied or evaluation research needs to take into account just how the evaluation (of success) will take place. For example what measures or surrogates or benchmarks will be used? This gives rise to the possibility that measurement might be so important in the context of the data question that it requires a separate category of its own.

In terms of sampling two additional considerations might well be cost and time. Although in basic research time is never limitless in applied and evaluation research the constraints are often that much greater.

Access, might be easier to gain for applied and evaluation research as the research it is likely to have been commissioned. As a consequence the organization has already demonstrated a commitment to engage.

Ethics applies in the same way to any research and ethical standards need to be viewed as a given.

Unit of analysis – here research designs might differ to the extent that applied and evaluation research needs to ensure it focuses on the utility that it offers. The processes and practicalities will very much depend on the type of research that is to be conducted whatever type of research is taking place. This said a feature of process consultancy (a popular type of applied research) and evaluation research often require longitudinal studies to take place. This requires a commitment to collect information regularly over time and therefore for the relationship to remain strong between the researcher and the organization over time.

Finally, in relation to theory, consultants are keen to understand not simply ‘expoused theory’ but also ‘theories in use’. The latter includes practitioner knowledge (knowing) that is enacted and forms practice. This presents a challenge for researchers and requires particular skills. This often involves understanding managers’ implicit as well is explicit understandings, and how these were formed in order that they might be changed.

4.10 Classification

 

Ontology

Epistemology

Methodology

Method

Grounded theory

*

***

**

*

Unobtrusive measures

 

*

 

***

Narrative

*

*

**

*

Case method

 

*

***

*

Ethnography

*

**

**

 

Critical realism

***

**

*

 

Participant observation

   

**

*

Experimental design

 

***

   

Falsification

 

***

   

Theoretical saturation

 

*

**

**