Unit 2 The Research Question

Unit Author: Professor Nick J Fox

Learning objectives

Having successfully completed the work in this unit, you will be able to:

select a research question
refine a research question
distinguish between dependent and independent variables
define concepts and indicators
construct an index

1. Choosing a research question

Defining the research question is the essential first step in the research process.

A clear research question will enable you to:

establish a hypothesis, aims and objectives
generate coherent data, and analyse that data to test your hypothesis
offer conclusions that ‘answer’ your question.

Without an appropriate research question, your research will become a ‘moving target’ and what you wish your study to uncover will remain hidden.

There are multiple reasons why a particular research topic might be chosen. For students, one of the most important reasons is personal interest, a question that arises from work, personal or professional experience or just something you feel passionate about and want to explore. This is an excellent basis for research, as you will need that kind of passion to keep you inspired for what can be prolonged periods of study.

If you are having difficulty in selecting a research question, then there are a number of possible ways to find inspiration. Firstly, consider your own experiences:

Can you think of any particular activities or issues that cause you concern or a problem at work?
Are there particular activities which are carried out routinely, but which you feel are not necessary or could be changed in some way?
What is the evidence for some of your current activities?
What activities do you do purely because they have always been done in that way?
What areas of your practice could be challenged as ineffective or even dangerous?

You will need to consider your current practice and examine it critically.

If you are still struggling for a research topic, look outside your work to external sources. Check out the research agendas and priorities of potential funding bodies. For example, try to establish the research interests of charitable organisations working in an area that interests you. What are their priorities?

Identify the research priorities of your company, organisation or professional body. They may well have a written programme of their research and development priorities.

In other cases, a research area (or even the specific question) may already have been identified by employers or research sponsors. Although this has the advantage of giving you early focus, you still need to explore the topic for yourself before you begin the study, in order to ‘make it your own’. There might be a number of interested parties, each with their own perspective on the subject. It is vital that you pin down the research question, as outlined below, in order to make sure that it is answerable and you are able to complete the project.

One way of making the research your own is to review the literature. This will help to understand and clarify your research question. Reviewing the literature before you start can be a difficult task. One needs some of the skills of critical appraisal, so as not to become lost in the ‘sea of literature’. For guidance on this please see Unit 12.

2. Refining the research question

Once you have a research topic – whether it has been identified by you, or by someone else – you will need to refine it into a workable question. Occasionally it is possible to come up with a research question ‘off the top of your head’. However, many of these research questions are unanswerable in their initial form. It is often necessary to go through a number of revisions to produce a clear answerable question.

For example, if my area of interest is obesity, my initial attempt at defining a research question might be:

‘Why do people become overweight?’

Although this is a very pertinent question to ask, in its current form t is unanswerable in any specific or meaningful way, mainly because it is far too broad in scope, and does not articulate if we are trying to answer the question at the level of genetics, physiology, psychology or culture. The terms of reference need to be more precisely specified, in order to make the question answerable.

So I might try to refine the question, limiting what it asks and making it a more clearly defined query. Based on reading around the topic of obesity, I might pose this question:

‘Have changes in the food families eat led to the current obesity epidemic?’

Here I have chosen to focus upon possible changes to family diet (for instance, fewer ‘sit-down’ meals and greater consumption of highly processed foods) and the impact on patterns of obesity.

However, this second question still poses some issues that need to be resolved, such as:

Precisely which time periods am I comparing?
Should my study look back in time (retrospective) or study people into the future (prospective)?
What data exists from earlier periods about food consumption?
How do I define processed and unprocessed foods?
How do I measure consumption of processed foods?
How exactly do I define obesity?
How do I define ‘epidemic’?

It is important to understand clearly the process by which an idea is turned into a workable research question that you can answer. Importantly, when refining a research question, we have to bear in mind what we are going to measure, and what tools are available to do that measuring. It is all very well having ideas about studying a concept, but if no methods exist for measuring it, we cannot hope to answer the research question.

All these practical issues mean we need to refine our research question even further. It is going to be very tricky and time consuming to do a detailed study over the past 60 years, so maybe we need to be more focused? So the following, fully refined research question may seem less ambitious, and also a bit long and convoluted, but at least it is now answerable:

‘How have changes in the formality and content of family meals influenced the prevalence of obesity and overweight in adults and children?’

3. How to turn a research question into a hypothesis

Most but not all research questions enable the formulation of an explicit hypothesis that the research will test, and an associated null hypothesis

As an example, consider a short, relatively unrefined research question:

‘Does the educational status of parents affect incidence of obesity among teenagers?’

The associated hypothesis will be:

‘The educational status of parents affects incidence of obesity among teenagers.’

This will then be tested in the study. Our null hypothesis would then be:

‘The educational status of parents does NOT affect incidence of obesity among teenagers.’

Having established the hypothesis and the null hypothesis, we can go on to establish the aims and objectives of the research, which will help us plan the project. For instance, taking the example above, the aim and objectives of my research project could be:

Aim:

To investigate the effect of parental education on obesity in their teenage children.

The objectives describe the practical steps needed to achieve this aim:

Objectives:

Choose a sample (e.g. inner-city families)
Identify which educational levels to compare.
Decide how to obtain data on teen children’s weights.
Control for other variables (e.g. age of children, age of parents, family income level) when sampling.

Note that descriptive studies will not have a hypothesis. However, all studies have a research question and an aim. The objective in a descriptive study will simply be to describe a phenomenon (for example, describing the ways in which obesity is managed and experienced).

4. Independent and dependent variables

When developing your research question, it is important that you have a clear idea of the independent variable(s) and the dependent variable.

An independent (or intervention) variable can be understood as a causal factor. In this example, our independent variable is ‘parental education’.

A dependent (or effect) variable is an outcome measure; in this case, our dependent variable is ‘obesity’.

Sometimes it can be difficult to decide which is the dependent variable and which is the independent variable, because it is unclear which is the causal factor. For instance, does low educational achievement lead to poverty, or does poverty lead to low educational achievement? It is usually important that you decide on these definitions before you embark upon the research design stage.

It is possible to have more than one independent variable. There may be many factors influencing a particular outcome. For instance, adopting a vegetarian diet may be a matter of conscience, financial necessity, cultural norms, health choice, and so on. If you have a research question with many independent factors, this will in turn influence the research design and the analysis stages of your project.

Some descriptive studies do not have explicit independent variables, as the purpose of the research is to identify what these may be. For example, the following question does not have an independent variable:

‘What proportions of adults in the UK are vegetarian for health, ethical or environmental reasons?’

SAQ 2.1: Independent and dependent variables

Read the list of research questions shown below and underline the independent variables and circle the dependent variables.

[table id=12 /]

5. Descriptive, analytical and experimental designs

The research question is an important part of the research process and may determine the type of research design that is chosen to answer it. Some examples of how the research question can drive the design are shown below.

A researcher for example might wish to pose the research question:

‘What is the pattern of obesity-related illness in Sheffield?’

This is an open question that seeks to observe a phenomenon. It requires an observational or descriptive design based upon an examination of primary or secondary data (data already collected such as hospital records). There is no intervention and the design would be described as non-experimental or descriptive.

Another research question might be:

‘How has meat consumption in Manchester changed since 1920?’

Clearly, answering this question would involve a degree of analysis and so an analytical design is suggested by the question. You would look at historical archives and data for different periods, say every decade since 1920 to the present.

A question which implies an intervention might be, for example:

‘Did obesity-related illness reduce after a healthy eating project was established in a particular Glasgow neighbourhood?’

This would suggest a quasi-experimental design (for instance, a cohort study) to answer this particular question.

SAQ 2.2 Types of research question

For each of the following research questions given below please indicate whether these could be answered following an experimental, analytical or descriptive design.

[table id=13 /]

6. Concepts and indicators

Having identified the question that encapsulates your research, we can move on to the next step. But in fact, we have to turn back immediately to our research question and pull it apart! But never fear, it’s not back to the drawing board, this is a creative pulling-apart, which is very necessary if we are to move from an abstract question to a research design and eventually a study that will enable the question to be answered.

Two stages are involved in this procedure. The first concerns the ‘concepts’, the second the related ‘indicators’. So what exactly do we mean when we use these terms?

6.1 Concepts

A concept is an abstraction from some event, object or aspect of the world around us. It is something created in the mind, which is intended to represent that aspect of the world outside. You may read books that talk about constructs, and this is what a concept is – a mental construct, so-called because it is constructed by human thought. We will treat concepts and constructs as equivalent.

A concept is not the same as the thing it is intended to represent. To give an example, most of us have come across dogs during our lives. Most of us will also, when we encounter a dog, have a mental construct to meet the occasion: we will think to ourselves “dog”. We have learnt to recognize the reality, and use a mental construct to identify it, to compare it with other dogs we have come across, and make appropriate responses (which may be to put out a hand to pat it, or to run for the nearest tree, depending on our experiences of dogs in the past).

Concepts are the building blocks of thoughts and of reason, and are usually (but not always) linguistic. In this unit we will be concerned only with linguistic constructs, as these are what we must use in developing our research question.

Concepts may be simple, like ‘dog’, or ‘table’. They may be in everyday usage, or they may be technical. Each profession, or branch of academia, has its own concepts that may be unintelligible to those outside. A social scientist may use the concept of ‘health’, ‘gender’ or ‘social class’. These concepts may be relatively straightforward, but often may be abstract, and not directly observable in the way that a dog or a table is observable. Researchers may offer indicators which they will argue can demonstrate the presence or absence of, for example, ‘deprivation’. (We will come to indicators in just a moment).

We can see that concepts are indeed at the heart of any research question.

SAQ 2.3 Concepts

Underline the concepts in this question. If you are still not sure what constitutes a concept, think about what things are referred to in the question.

[table id=14 /]

But there is more to this business of concepts than simply listing those that appear in our research question. As we noted earlier, sometimes concepts can be very technical, and abstract. We may be able to assume that everyone can identify a dog when they see one (although we will consider what is involved in this recognition later in this unit), but we cannot always assume that everyone will know precisely what is meant by a concept such as social class, or even that very well-worn word ‘health’. So having identified our concepts, the next stage is to define them. For example, we will need to define five concepts in the second research question.

[table id=15 /]

You may not agree with these definitions, and in fact some are quite arbitrary. We can choose to define parenthood in a number of ways, or debate what kind of attitudes to eating are considered ‘abnormal’. What is important is that by defining these concepts precisely, we all know what is meant. Think back to the previous question. It matters very much whether ‘processed food’ means simply ‘cooked’, or subject to multiple industrial processes and the research would give different results dependent on what degree of processing is being defined for the study.

At the start of a piece of research it is very important to define the concepts, to ensure everyone knows what you are researching. Rarely can we just assume everyone ‘knows’ what your concepts mean precisely. In many cases concepts can be very vague and need further defining.

6.2 Indicators

Having considered the concepts that make up a research question, and offered some precise definitions, we are now able to move to the next stage, to begin to consider how to actually answer the question. Defining concepts is not sufficient to enable us to do this, for the simple reason that concepts, as was stated right at the start of the unit, are ‘all in the mind’. To be able to test how one concept may influence another, we need to find real world ways to measure them. The things that represent concepts in the real world are indicators. You may also hear people speak of measures or operations, and we will use these three terms interchangeably.

Indicators are simply ‘the things that indicate the concepts’. The term ‘measures’ can be used because we want to impose some kind of value on the things that indicate the concepts, although the term may be a little misleading when what we are looking at is more to do with a quality rather than a quantity. People may be male or female, and we could have various indicators of that concept of sex or gender.

Perhaps the term ‘operations’ is the most incisive, although we may find it a strange word to use at first. But this is a good definition of indicators:

‘The operations that generate the data that the researcher is satisfied reflect the concept.’

Operations are concrete activities: they may be measurements using a thermometer or some other piece of equipment, they may be some kind of research tool like a check-list or a questionnaire, they may simply be observations by someone trained to differentiate between objects, be they normal or cancerous cells, male and female subjects, or kinds of body language. But the important thing is that they are concrete rather than abstract, they will generate empirical (based on observations) data, which will make it possible to transform the abstractions that are concepts into results which can test a hypothesis or answer a research question.

From this word operation, we also get the word operationalisation, which refers to the process of turning concepts into indicators.

For every concept there needs to be at least one indicator, so in addition to definitions of concepts we also need to list the indicators that we need for our research. For example, consider the simple research question:

‘Is the consumption of processed food linked to levels of teenage obesity?’

We can list the concepts and possible indicators:

[table id=16 /]

The abstract concepts have been turned into operations which can be applied in a research setting: all we need to do is sample teenagers (rather than adults), and obtain a figure for the volume of processed food consumed per week, and a BMI. Easy isn’t it?

Well actually it’s not that easy, and the rest of the unit will consider issues about choice of indicators at greater length. The importance of what follows should not be underestimated: choice of indicators determines research design and the methods to be used, and the validity and generalisability of the study.

6.3 How do we decide which indicators to use?

Let’s go back a step or two and think again about that dog that I mentioned earlier. Few of us will have trouble visualising a dog when we use this concept. But imagine that a dog has just run across the room you’re in. How would you identify it as a dog? To put it another way: what are the indicators of ‘dog-ness’ that enable you to make that differentiation from other animals? List the indicators you think are relevant for this test of ‘dog-ness’.

How did you get on? You probably have a long list now – when we’ve done this exercise in class, people come up with fifteen or twenty indicators/attributes of dogs. But how useful are the indicators that you have listed? How many are really going to be able to differentiate dogs from cats, or more seriously from foxes or wolves or even pigs (which can look very much like dogs and can even be trained to walk on a lead)?

‘Four legs’ isn’t much good, nor is ‘carnivore’ or ‘hairy’; they can refer to other animals. Better are ‘wet nose’, ‘responds to commands’, ‘barks’ and maybe even ‘chases cats’. But are these really sufficient to confirm ‘dog-ness’? Even these are questionable, after all, seals bark and monkeys can learn to obey verbal commands.

What is happening here? We are virtually certain we can identify a dog, yet when it comes to identifying how we do it, it doesn’t seem so easy. The point of this exercise is to illustrate the problems with indicators. Firstly, all indicators fall short of perfection when it comes to trying to provide concrete actions to reflect a concept. It is as if the concept is always elusive and difficult to pin down: no indicator fully reflects its character. (Admittedly, some indicators are closely identified with their concepts, so that temperature and the reading on a thermometer are considered synonymous – we will encounter this phenomenon in Unit 4 in relation to ‘face validity’.) ‘Everyone’ accepts a thermometer reading as an indicator of temperature. The certainty with which one can claim that an indicator represents the concept we are suggesting it reflects, is a measure of the validity of that indicator, and in social research that will never be 100 per cent.

Secondly, one indicator may not be sufficient to reflect a concept adequately on its own. We need a range of indicators to pin down the concept; thus ‘dog-ness’ clearly needs a whole range of indicators which taken together will reflect our concept. What we take for granted on an everyday level, needs to be that much more rigorous when it comes to research. Finding the right indicators to reflect a concept can be very difficult and sometimes impossible.

What strategies can we use to choose the right indicators? There are four possibilities:

The concept is so straightforward that choice of indicator can go ahead with relatively little debate. For example, ‘voting intention’ is usually indicated by the well-established question ‘how would you vote if there were an election tomorrow?’
Although not straightforward, there is an indicator that has been well used previously, so it can be applied without prolonged debate. For example, social class has traditionally been indicated by a classification of occupations into various groupings.
A number of indicators are combined to create an ‘index’, which reflects all aspects of the concept even though it is diffuse and difficult to summarise. For example, more recent attempts to define class have incorporated income, educational level, personal tastes and preferences, friends’ and associates’ occupations (Savage et al, 2013).
A concept is so abstract that it is very hard to justify any operations that will reflect it. In the 1970s, a social scientist called Selltiz came up with a concept of ‘a sense of loss of identity among people who have moved to a foreign country’. He was the first person to want to explore this, but what indicators could he use? In such a situation, the researcher has to argue the logic of the construct. Logically, it may well be related to self-esteem and to alienation, so measures of these concepts could reflect Selltiz’s construct. What about nationalism? What about stress? The trouble is, these may or may not be associated – that’s what needs to be researched.

In such situations, indicators may be questioned by other scientists, with the consequence that the validity of the research is also questioned. Only over time will such concepts come to be accepted.

To summarise, indicators provide the concrete actions we can take to gather the data that will allow the concepts in a research question to be asked. But the choice of indicators is not simple or straightforward. We need to be as certain as we are able that an indicator reflects what we say it represents. Sometimes we find that – however clever our research question – there simply are not indicators which will adequately reflect the concepts. In such circumstances, the question is unanswerable, and we have to change the research accordingly.

7. Use of an index

An index is nothing more than the collection of indicators that has been agreed by the research community together to operationalise the concept satisfactorily. For example, take the concept ‘health’. What indicators would you use to operationalise this concept? List them – you will be well on the way to creating an index.
You will probably have written things like ‘presence or absence of pain’, ‘visits to a doctor’, ‘number of prescriptions issued’, ‘subjective experience of well-being’. The list is potentially endless, as we can think up many different aspects of this very amorphous concept ‘health’.

Often, one can adopt a ready-made index, one that has been used by other researchers in the past. If there is no ‘off-the-shelf’ index, you will have to construct your own.

Indices (plural of index) are used whenever a single indicator is insufficient to cover all aspects of a concept, and as such it is related to content validity, an approach to validity that attempts to incorporate all the main features of the phenomenon we which to operationalise or measure. We will look at content validity in more detail in Unit 4.

Summary

In this unit we have looked at the building blocks of research: the research question and how to frame it, the concepts that underlie a research question, and the indicators or operations that you need to use to actually do the research. We have looked at how to choose an indicator, including issues to do with indices, and problems of choosing the wrong indicators.

Many issues raised in this unit will become back into focus in Unit 4, which concerns validity.

It is also relevant to design of questionnaires and other survey instruments, and to data collection and analysis. And when we look at critical appraisal, a key element of assessing research concerns the choice of indicators – did the researchers really measure what they claim to be measuring?

This unit may have seemed straightforward, common-sense even. However it underpins the entire edifice of research and this is the message that has been running through it. It is essential to think very carefully about the research question, concepts in your research question, and how you are going to develop the right indicators. Some indicators will be better than others, and some will be absolutely dreadful. Don’t wait until you have started collecting data before you decide your indicators are dreadful rather than satisfactory.

Reflective Exercise 2.1

Refine the following research question: Do men and women have different attitudes towards vegetarianism?

[table id=17 /]

References

Savage, M.et al (2013) A New Model of Social Class? Findings from the BBC’s Great British Class Survey Experiment. Sociology, 47: 219-250

Answers to Self-Assessment Questions

SAQ 2.1

[table id=18 /]

SAQ 2.2

[table id=19 /]

SAQ 2.3

[table id=20 /]

[box type=”info”]This unit is part of our course on Social Research Methods. You must be registered and logged in to access course content. Back to courses Welcome page.[/box]