For today’s June Blog – the 8th so far – I wanted to write about something that I think is often quite hard for students writing up a final project dissertation or thesis. It also seems especially relevant now, in the midst of dramatic changes to research plans dsrupted by Covid-19. Namely…
…is it ok to change your research questions?
I think this has always been hard for people – I found it hard when I wrote my thesis – but it’s even harder now in the context of psychology’s replication crisis and concerns about p-hacking and other phenomena.
My fairly firm position now, is that yes, it is OK to change your research questions, but not your hypotheses. To understand this, we need to first define those two, and I’ll use a fairly simple (totally made-up) example for that.
Your research questions is the question you are trying to answer in your research. For example, does drinking a cup of coffee beforehand improve performance on a maths test?
Your hypothesis is what you expect to be the answer to that question. It should be grounded in the literature – what people have discovered before. For example, if previous authors have found that coffee drinkers get better scores on maths tests you might have the hypothesis that yes, drinking a cup of coffee beforehand will improve performance on the test. This is your hypothesis.
It’s important to note that in this example, you might also have the opposite hypothesis. After all, the previous research merely shows that coffee drinkers to better on maths tests. But that might be some other kind of relationship, other than coffee causing good test results. Maybe coffee is expensive, so coffee drinkers have more money, and they have more money because they have previously excelled at maths tests – giving them higher paid employment opportunities. This would account for a relation between being a coffee drinker and being good at maths tests, without the former actually causing the latter.
So, why would you change your research question?
Keeping up with the example above, imagine you go to the shops on the day of your experiment and they’ve completely run out of coffee. So instead you buy tea, reasoning that at least it is caffienated, and that’s good enough. Now, when you writ eup your study, you can’t stick with your original research question: “does drinking a cup of coffee beforehand improve performance on a maths test?” Instead you need to adapt your research question. It might become: Does drinking a cup of tea beforehand improve performance on a maths test? or perhaps: Does ingestion of caffiene beforehand improve performance on a maths test?
The important thing to note here is that the research question you state in your final report has to be something you can actually answer with your data. Even if that it is different – subtly or dramatically – from what you set out to ask.
Does this ever actually happen though?
It might seem implausible that this would ever happen – and indeed the example above is a bit absurd. But there are lots of ways that people find themselves actually doing something a bit different from what they had planned, in a way that means the original question they set out to address is just not the one they are answering.
In our lab, we use a lot of video coding methods. This means watching video footage – for example of two children playing – and systematically counting or defining what’s happening in a the video. Let’s imagine you set out to code how well two children get on while they’re playing (I’ll call this “rapport”), and whether different types of toys infuence this. Your research question might be something like Does toy type influence rapport between children? You are going to answer your research question by giving some children representational toys to play with – like dolls and a tea set. Other children get cause-and-effect toys, like building blocks, toy instruments and a jack-in-the-box. Your hypothesis, based on the previous literature is that Representational toys lead to better rapport during play than cause-and-effect toys.
You decide a good way to code rapport is to count how often the children each smile (and in fact, all the different types of facial expressions they do) and how often they look at each other. So you collect all your video samples and you code them dilligently – and this could take months or even years. But you start noticing that the children playing with the representational toys and making all sort of facial expressions – frowning, surprise – as they act out what the dolls are doing or feeling. Meanwhile, the kids with the cause and effect toys are making hardly any eye-contact because they all have fiddly buttons and component pieces, which means the kids have to look at them very closely. You start to realise that what you have coded doesn’t capture rapport, but instead captures other phenomena related to the play setting. This suspicion is confirmed when you compare your video codes with a measure that directly asked children whether they liked the person they played with. Your video codes and the child’s own ratings don’t go together at all.
This is, to emphasise, an entirely made-up example and I’m not sure it would hold much water scientifically. but I think it does serve to illustrate how it is possible – even quite easy – to set out to answer one research question but end up answering another. In this case, you might decide that your codes can tell you about the range fo facial expressions used in the two play settings, and that would still be interesting even if the question is different.
But what about your hypotheses?
Well, what we should never do is change our hypotheses after we’ve done an analysis. We shouldn’t pretend that whatever we found is what we expected all along. but of course, if you research qestion changes, your hypothesis might change with it. In the play example above, having decided that your data no longer mapped onto the question you intended to ask, you woud want to refine your research question and then examine the literature to come up with a reasonable hypothesis for that new question. How would you expect toy type to influence children’s facial expressions? But the key thing is that this change is driven by the data you have available but not by the results of your analysis.
For students battling with adaptations ot their research due to Covid-19, or just because of the usual challenges of research, I hope this helps you feel it is OK to look again at your research questions, and make sure that whatever question you ask, that’s the one you’re answering.
