Unit Author: Professor Nick J Fox

Learning objectives

Having successfully completed the work in this unit, you will be able to:

  • define what is meant by research design
  • distinguish between:
    • qualitative vs. quantitative
    • experimental vs. non-experimental
    • cross-sectional vs. longitudinal
    • prospective vs. retrospective
  • choose an appropriate design for your own research question.

Introduction

People sometimes get confused between research design and research methods. In a nutshell, research design is the structure within which we do our research. A research design will provide the framework for the methods used to collect and analyse data. It is also known as a methodology.

Research methods, by contrast, are the techniques or practical steps we need to take in order to actually turn a design/methodology into a piece of research (for instance, to collect data or to analyse it (Bryman, 2012: 45).

For example, a researcher may wish to discover the effectiveness of a new government policy. He or she selects a longitudinal survey as the research design, in order to track any impacts of the policy over a period of time. Once the design has been selected, the next step is to choose the most appropriate research methods to collect the data. (Simply choosing a research design will not provide you with the data.) There are a number of different research methods that could provide the data required. For instance, face-to-face interviews, postal questionnaires, or telephone interviews.

We will cover the different research methods that are available to collect and analyse data in later units. In this unit we will simply look at designs.

1. Choosing a research design

There are a wide range of research designs used in social research. Research designs do not fall into a hierarchical order, with one design being superior to another. It is rather that the different designs are appropriate for answering different types of research questions. The design that is chosen is therefore largely dependent upon what kind of understanding is being sought from the research question. This is why we spent time in the last unit looking at research questions. Only with a clear question, can you choose a research design! The clear definition of the research question should help direct you towards an appropriate research design.

Designs can be differentiated in various ways:

  • quantitative vs. qualitative designs
  • experimental, quasi-experimental or non-experimental designs
  • cross-sectional vs. longitudinal designs
  • prospective vs. retrospective designs

It is possible to answer a research question with a number of different research designs. Thus for example, we could have a quantitative, quasi-experimental, longitudinal, prospective design. (However, not all combinations of these four typologies are possible, as we will see below.) There may be other factors influencing your choice of research design including ethics, resources, funding, time available and so on. This unit will help you move from research question to research design confidently.

2. Qualitative versus quantitative approaches

At a very general level, one of the first things that you need to decide upon is whether your research question requires a qualitative or quantitative approach.

Essentially, quantitative designs enable you to answer ‘what’ type questions, or explain causal relationships. They use a large number of respondents or cases, aggregating these by means of statistics, to provide findings are both rigorous and generalisable to the wider population.

Qualitative approaches, on the other hand, are usually based on a fairly small number of individuals and can be useful in exploring ‘why’ type questions. They search for reasons, motives or explanations, often investigating events through human accounts. They are useful to explore the beliefs and attitudes of individuals, in a way that is not possible with quantitative approaches.

For example if we wanted to know exactly how many people eat their ‘five-a-day’ fruit and vegetables, then a quantitative approach would be required. However if we then wanted to know why some people do not eat their ‘five-a-day’, then a qualitative approach would help to identify the reasons. Qualitative approaches are often very useful in enabling the researcher to get some insight into the respondents’ world from their point of view and hence to get an understanding of what might be viewed by some as ‘irrational’ behaviour.

There have been numerous debates and arguments regarding the nature and value of qualitative and quantitative methods in social research. Quantitative and qualitative approaches differ markedly in terms of the fundamental principles concerning how we know the world, or epistemology. Quantitative research designs tend to operate within a realist framework, and use general theory to test specific hypotheses by deduction. Qualitative approaches, by contrast, often adopt and interpretivist and naturalistic approach, generating theoretical propositions from data inductively. We examine these issues further in the following units, as we consider validity and reliability (Unit 4), qualitative and quantitative methods, and the philosophy of science (Unit 1).

Despite these differences, it is important to remember that qualitative and quantitative approaches are not mutually exclusive. On occasions, a good research design can productively combine these approaches to give breadth and depth to data. For example, qualitative research may be used in the preliminary stages of a research programme to generate hypotheses; to supplement quantitative research by gathering in-depth interview data; or in the later stages to seek explanations for new or unexpected quantitative findings.

SAQ 3.1 Quantitative and qualitative designs

Read the following research studies: would a quantitative or qualitative research design be more appropriate in each?

[table id=21 /]

In the next sections, we will consider social research designs in terms of the other dichotomies noted earlier. Many of these are quantitative designs, but some also apply to qualitative research designs. The latter research designs are covered in greater detail in Unit 8.

3. Experimental, quasi-experimental and non-experimental designs

True experiments are extremely rare in social research: they are more common in the natural sciences, where non-human objects can be manipulated under artificial laboratory conditions. However, you may come across research that applies experimental methodology in psychology, nutrition and health services research.

3.1 Experimental designs

An experimental design allows the researcher to quantify the effect of an intervention. For example, an intervention could be the establishment of a city farm or the introduction of traffic restrictions that separate motorised vehicles from bikes (where the effects might be on diet, exercise or safety), a social media campaign promoting a product or a cause, or a new social policy (for instance, an awareness campaign countering homophobia in schools).

In simple terms, a classic experimental design enables the researcher to measure the impact of a cause ‘A’ (independent variable) on an outcome ‘B’ (dependent variable), by measuring some indicator of B both before and after the intervention. Thus the researcher will measure the baseline before the intervention and the outcome after the intervention. In a classic experimental design, to ensure that the difference identified is not due to some other external factor, an experimental design will contain a control group for comparison with the intervention group. A control group is a group of subjects who receive no intervention or sometimes a different intervention. For example, to test the effect of the anti-homophobia campaigns, a comparison could be made with a school where there was no campaign.

To enable you to associate A and B, the experimental design has to rule out the likelihood that variables other than A could lead to a change in B. For instance, if the intervention A was the introduction of a city farm, and outcome B was weight loss among its users, then other variables that could lead to a change in B could be dieting, exercise, or availability of fresh fruit and vegetables from other sources. These other factors are known as extraneous or confounding variables.

The randomised controlled trial (RCT) is claimed to be the most powerful way of demonstrating the causal effect of an intervention because of randomisation. Randomisation within an experimental design is a way of ensuring control over confounding variables as it involves the random assignment of subjects to either group. This process should result in an equal distribution of confounding characteristics in each of the two groups, for example, in terms of age, sex, and ethnicity. As such, randomisation allows the researcher to have a greater confidence in identifying real associations between an independent variable (the cause) and a dependent variable (the effect or outcome measure). This is shown in the following figure which represents a trial of change of diet to reduce blood cholesterol.

Experimental designs can demonstrate the causal effect of an intervention, but it has been argued that it is best suited to the laboratory, where confounding variables can be fully controlled. It is often difficult to apply in the real world where outside influences abound and may not even be recognised. RCTs have therefore been criticised as lacking direct applicability to real life (external validity: see Unit 4).

3.2. Quasi-experimental designs

A quasi-experimental design is one that mimics an experiment, but in which the researcher has no control over who receives the intervention and who does not, usually because it is using real-life situations to gather data. For instance, we could explore the impact of a city farm on health without having to set one up for the study. Instead we could compare two neighbourhoods, one with an existing city farm and one without, and explore the impact various outcomes such as diet, health and well-being.

In a quasi-experiment, the respondents self-select, so this diverges from a true experimental situation in which participants are carefully selected in terms of attributes and also randomly allocated to trial and control groups. However, it is still possible to control for extraneous variables that derive from participants’ characteristics by matching (as opposed to randomisation). Matching can be done by selecting for certain characteristics such as age, socioeconomic status (SES) and gender. In the example of the city farm study it would be necessary to match the people studied for these and perhaps other attributes such as body-mass index (BMI). In this way, a researcher can control for these variables that may have effects on the dependent variables being studied.

3.3 Non-experimental designs

The majority of social research conducted is non-experimental, with no formal intervention. Non-experimental designs include all qualitative studies, and many quantitative studies. Some non-experimental studies are simply descriptive, as they do not test a hypothesis.

The main non-experimental designs that you will come across are:

  • surveys using quantitative questionnaires or structured interviews;
  • qualitative interviewing;
  • ethnography (observational studies);
  • case-control studies;
  • cohort studies.

We will look at these in greater detail in terms of cross-sectional and longitudinal design.

4. Cross-sectional and longitudinal designs

The distinction between cross-sectional and longitudinal design is an important one, and is a basic distinction used in both quantitative and qualitative research. A cross-sectional study provides a measure of phenomena as they are at one point in time, equivalent to a snapshot.

A longitudinal study in contrast is more equivalent to a film, in that information is collected from individuals over a period of time, sometimes more than twice. Longitudinal studies therefore can demonstrate change over time.

(Note: all experimental and quasi-experimental studies are longitudinal: they measure before and after an intervention.)

We will focus here on non-experimental cross-sectional and longitudinal designs, as these are most common in social research, and we have already considered experiments.

4.1 Cross-sectional designs

A frequently used cross-sectional quantitative design is the social survey. Cross-sectional studies are usually fixed in time and data is collected from two or more cases, though it is worth noting that the cases in a cross-sectional design are not always individuals. Cities, households and schools are just some other examples of cases that are studied in a cross-sectional design.

Surveys can take many forms, but in social research they are particularly useful in the following areas:

  • Gathering information on the attitudes, beliefs or values of the individuals that comprise a population. For example, a survey might assess responses to levels of crime in a neighbourhood, or the extent to which religious values affect daily life.
  • Comparing different groups in a wider population. For instance, I either of the above examples, surveys might be taken in different cities, or the sample might be stratified to compare men and women.

In a cross-sectional quantitative survey, research is usually carried out by administering a questionnaire or carrying out a structured interview at one point in time from which the data can be quantified and analysed using inferential statistics. (We will look at survey approaches in detail in Unit 6.)

Cross-sectional research designs are also employed in qualitative research. Data is usually collected using focus groups or qualitative interviews at one single time point and analysed using different qualitative data analysis techniques such as content or discourse analysis. (We will discuss these methods in more depth in Units 8 and 9.)

4.2 Longitudinal designs

As mentioned above, the longitudinal design is more equivalent to a film, in that information is collected from individuals over a period of time, sometimes more than twice. Longitudinal studies can therefore demonstrate change over time.

Some of the best-known longitudinal studies are the various cohort studies that follow a sample of the population (for instance, all children born on specified dates) from birth forwards. While many of these focused principally on health issues, later studies such sexuality-assemblage the ‘Children of the 90s’ study run by University of Bristol (http://www.bristol.ac.uk/alspac/) also explore many social factors surrounding development across the life-course.

In social research, data for quantitative longitudinal research is typically collected using structured interviews and questionnaires, the only difference from cross-sectional research designs being that it is collected from the sample on at least two occasions.

For a longitudinal qualitative research design, data is typically collected using qualitative interviews, which are carried out on more than one occasion. Ethnographic research which is carried out over a long period of time would also be a method of collecting qualitative data using a longitudinal research design.

5. Retrospective and prospective designs

All longitudinal designs (experimental, quasi-experimental and non-experimental) can be categorised as either prospective or retrospective, depending on whether they look back from the moment of research, or forward from the launch of a research study (as in an experimental study). Surveys and most interview designs are neither retrospective nor prospective.

5.1 Retrospective designs

Retrospective studies depend upon access to data from earlier time points, often making comparisons from an early date to a later date or to the present, and will depend upon access to good quality historical data. Epidemiological studies have exploited routine gathering of data (for instance, air quality or other weather events) to explore subsequent health outcomes. So for example, Anderson et al (1996) looked back at both daily variations in air pollution in London between 1987 and 1992, findings an association with daily levels of mortality.

Case control studies are also used in health research, as an alternative to a prospective experimental study. In this design, people with a disease and those without it are compared in terms of some past events that may have led to the illness. The most dramatic examples are the retrospective studies that found the link between smoking tobacco and lung cancer (Doll et al, 2005).

Social research does not tend to use such designs to a great extend, mainly because access to high-quality quantitative data on social factors is limited. Exceptions are use of electoral rolls and studies such as the UK General Household Survey (Integrated Household Survey since 2008), both of which have been gathered at regular intervals in the past. So, for example, Machin et al (2011) used a range of historical data sources (such as the British Crime Survey and Labour Force Survey) to reveal the inverse association between criminal activity and perpetrators’ level of education.

In qualitative research, documentary analysis is an example of a retrospective design, while interviews can also be used to look back in time, asking respondents to describe events from their past. Secondary analysis of qualitative datasets collected in the past may also be used to make comparisons with the present, though of course the respondents in the past and present studies will be different.

The advantages of retrospective designs are that they provide access to large data sets and can be cheaper to run than a prospective study. The disadvantages are that there is no way to assure the validity of data and the information available may not be wholly relevant to the research question being asked in the present.

5.2 Prospective designs

Prospective designs are forward looking, in that the researcher can have complete control over who receives the intervention, if any, and over the data collected. This means that the researcher can define the outcome measures and some level of quality can be applied to the data collection process. A prospective study begins at zero time and any interventions or observations are made in the following months/years.

The classic examples of prospective designs are experiments, including randomised controlled trials, and the quasi-experimental designs which have been discussed earlier. In social research, as has been noted, most studies are non-experimental, and the main types of prospective studies will be survey, interview designs and ethnographies that collect over a period of time, for example, by interviewing respondents at a number of points to compare responses.

In quantitative prospective research, longitudinal surveys are described either as cohort studies (which identify a specified group or cohort of people drawn from a population and sample them repeatedly) or as panel studies, which differ from cohort studies only in that they typically sample over a far longer period across the life-course of respondents, and may select a more heterogeneous group than a cohort study.

Cohort and panel studies may use qualitative methods, while ethnography is often regarded as a prospective design, as it entails observation over an extended period of time.

Prospective designs have the advantages that the researcher has greater control over data collected; can specify precisely what variables and indicators are to be studied; and can assure the validity and reliability of measures. The disadvantages are that they are often very costly to set up and run, and as a consequence may be based on a small sample that may be further decreases due to both real and experimental mortality (drop outs).

We have now explored social research designs in terms of the four typologies we set out at the beginning of the unit. To ensure you have a grasp of these, read each of the following descriptions of a research study and select an appropriate design, and indicate if it is qualitative or quantitative, experimental or non-experimental, cross-sectional or longitudinal, prospective or retrospective.

SAQ 3.2 Research design

What is the most appropriate research design for each study?

[table id=22 /]

Summary

In this unit we have examined the main research designs that are used in social research. We have considered the different aims and outcomes of experimental, quasi-experimental and non-experimental design, as well as longitudinal and cross-sectional studies. We have also introduced you to the terms retrospective and prospective, referring to whether the study is looking backwards or forwards in time, respectively.

However, the main message to be gleaned from this review is that a research design should be appropriate to the research question and adequate to deliver an answer. Furthermore, it is essential that an appropriate and adequate research design is decided before you decide what method(s) you are going to use.

Many of the designs discussed here are not mutually exclusive and sometimes a research question can be answered by several different designs, or combinations.

You should by now have an understanding of some of the possibilities of these designs for social research. Please complete your study of this topic with the following exercise for your log book.

Reflective exercise 3.1

You should now have an understanding of some of the possibilities of research design. We would like you to start thinking about how you can apply this to your own research. Write down a research question that interests you.

Think about potential research designs, and answer the following questions.

[table id=23 /]

References

Anderson, H.R., Ponce de Leon, A., Bland, J.M., Bower, J.S. and Strachan, D.P. (1996). Air pollution and daily mortality in London: 1987-92. British Medical Journal, 312 (7032), 665-669.

Bryman, A. (2012) Social Research Methods (4th edition). Oxford: Oxford University Press.

Doll, R., Peto, R., Boreham, J., and Sutherland, I. (2005). Mortality from cancer in relation to smoking: 50 years observations on British doctors. British Journal of Cancer, 92(3), 426-429.

Machin, S., Marie, O. and Vujić, S. (2011) The crime-reducing effect of education. The Economic Journal, 121: 463–484.

Further reading

May, T. (2011) Social Research: Issues, Methods and Research (4th edition). Buckingham: Open University Press.

Answers to SAQ 3.1

  1. Qualitative
  2. Quantitative
  3. Qualitative
  4. Quantitative
  5. Either or Both

Answers to SAQ 3.2

There are other possible research designs that can answer all of these questions, but the following will provide appropriate and adequate data.

  1. Survey using questionnaire or structured interviews (quantitative, non-experimental, cross-sectional).
  2. Case Control study to look at the effects of those flooded and those not flooded (quantitative, non-experimental, longitudinal retrospective).
  3. Ethnography with unstructured interviews (qualitative, non-experimental, longitudinal, prospective).
  4. Documentary analysis of relevant literatures (qualitative, non-experimental, longitudinal, retrospective).
  5. Cohort study of a representative sample of the population (quantitative, non-experimental, longitudinal, prospective) (Quasi-experiment not possible as programme is nation-wide).

[box type=”info”]This unit is part of our course on Social Research Methods. You must be registered and logged in to access course content. Back to courses Welcome page.[/box]