By far the most popular post I wrote this year was this one in April about dealing with rejection as an academic. Likewise, one of my favourite posts which I read was this ‘backwards CV‘ – a phenomenon I could easily replicate with my own list of rejections. Everyone I know who works in academia knows that it is essential to press on even when it seems like no-one else thinks your work is any good – but it is hard and we need to share these stories as well as the achievements.
Another reason to blog about this aspect of research life is that I think it provides a really valuable insight for participants, practitioners, patient groups, and other stakeholders. Why is that when you participate in a project it can take so long to hear about the outcome? Why is it that, when someone approaches your organisation to ask if you’d support a new research proposal, it takes months or even years for the project to get started (or it never starts at all)? Why is that when you call for researchers to do more relevant work it seems to fall on deaf ears?
In an attempt to further showcase the reality of research I’m going to review my year, not focusing on the success particularly but instead revealing the struggles. Since I’m largely a researcher who uses quantitative methods, here are some stats to get you started:
- papers and book chapters submitted (all of these had a first drafts submitted before the start of June) = 7
- papers now ‘in press’ (i.e. accepted to be published) = 3
- total number of resubmissions of papers not yet published (this is the number of times I had to either submit a paper to a new journal because it was rejected outright, or revise it in response to comments from reviewers) = 16
So you can see that of outputs from seven different, completed projects, only three have made it into the light while four are still struggling for life. One of those four was first submitted in May 2010 and has been through multiple review processes at three different journals. It is a particularly agonising case, as frequently reviewers have requested changes which negate those made in response to the previous set of comments, and amount to reverting to a previous version. We are quietly hopeful that it will at last be accepted for publication in the new year, but nothing is guaranteed. The delays were exacerbated by my taking maternity leave in 2011 but, while this example is fairly extreme, I would say most academics have similar tales to tell.
What about getting new projects off the ground? Here are some more stats, this time focusing on getting new income:
- grants applied for = 11
- grants won = 3
- grants rejected = 4
- grants pending (no decision yet) = 4
Of these 11 grant applications, 5 of them require more than one stage – normally a “short” outline and then another longer proposal in the second stage. In each case, as well as writing a proposal describing what we plan to do, I need to produce a budget, get this approved by the University, and provide evidence of my qualifications and support from my head of department. There are normally a few form sections or attachments too – for example a ‘lay abstract’, a description of the expected impact or stakeholder input, or a detailed justification of the costs.
The amounts applied for ranged from £3000 to £5,000,000. That big one is a bit of an outlier though – most are between £50,000 and £300,000 with a couple hovering near the £1m mark. My role on the grants varies too – of course I’m not the lead applicant on all these, especially not the big ones. It’s a reflection of the fact that I work as a psychologist embedded in first Education and then Medical schools – I get asked to be the lone psychologist as a co-applicant on projects which fall on disciplinary boundaries.
To complement these quantitative reports of my progress this year (or lack of – at best it is two steps forward, one step back), let’s also have a look at a case study of a specific project. I think it is reasonable to imagine that I can devote something like one day per fortnight to this project – I have other projects, student supervision, public engagement, admin responsibilities and all those grants and papers to write and re-write after all. So that means I can achieve two to four tasks per month. The timeline also involves a lot of consultation with collaborators all over Europe, which results in important enhancements, but also slow progress.
The project, in a nutshell, involved holding focus groups, designing a survey, circulating the survey in 11 countries, compiling the data, analysing the data and (eventually, hopefully) publishing the data – you can read more about it here. This is roughly what I did over the past 12 months:
- December: hold a focus group with autistic adults in Glasgow. Write up notes. Conference call to discuss focus group results from different countries.
- January: design the survey including 2 or 3 meetings with student volunteers. Update ethics for the study by sharing the outline survey script.
- February: Circulate survey to stakeholders for comments and test runs. Circulate survey to collaborators.
- March: Finalise survey following collaborator feedback. Send information (ethics, study outline) to various groups who we are requesting help share the survey online. Submit a conference abstract reporting on the survey (yes I know, we hadn’t even started collecting data!)
- April: Start sharing the survey in the UK (twitter, blog, networks, databases). Set up collaborators with Survey Monkey accounts. Nudge collaborators to complete their translations and start sharing too.
- May: share, nudge, share, troubleshoot, nudge, share. I sent 28 recruitment emails in April and May.
- June: The UK, Spanish and Italian surveys closed but lots of others were still open. We started compiling the database which meant converting text into number codes (e.g. male = 1, female = 2), removing data where people didn’t answer all the questions, and generally labelling and tidying the excel file. This is incredibly time consuming and boring but also incredibly important to get right. I had three volunteers helping me at different stages. Also answers to open questions (e.g. “Is there anything else you want to tell us?”) had to be sent back to their country of origin for accurate translation into English, so all the data could be analysed together.
- July: We closed some more surveys, did some more data downloading and tidying, and collaborators did more translating. One of the organisations that helped us to share the survey in the UK in April asked me to provide a summary of what we had found – you can imagine I wasn’t really ready for this, though it had been 3 months, so from their perspective it probably seemed reasonable to ask for results.
- August: Analysis. This single word covers a huge job, combining all the data files into one massive file (surprisingly tricky). Turning answers to individual questions into useful variables for analysis. Transferring the whole lot into specialist statistical software called SPSS – again time consuming as each variable needs to be defined (e.g. is it nominal or continuous), named and labelled.
- September: I hadn’t finished the analysis by any stretch, but that conference we applied to back in March had rolled around so I wrote up some slides with the outline of our findings and went to Toulouse to present them. People were very interested.
- October: sharing the conference presentation and a first draft of the findings with everyone. Discussions about plans for more analysis – you can’t just go trying every statistical test you fancy until you find something ‘important’. We had to agree a plan of action and a suitable way of filtering the vast amounts of data we had collected.
- November: we submitted another abstract, this time reporting on more complex findings, to a bigger conference. And I did more analysis. And some more. And finally got around to writing a report for the participants (I’ll post it up here very soon). That too needed to be checked and approved by my collaborators.
- December: so now, after twelve months of hard work, we have a pretty decent analysis of the data and an idea of what we might like to write. And in January, I should be able to get the first draft of a paper done. But as you can see from the stats above, that’s only a fraction of the process. Frankly, I’ll be impressed if this gets published by the end of 2015!
So a busy busy year, lots of projects, lots of work. And in terms of something to show for it? – well, only a fraction of that work has really come to fruition. This is part of why I think we need to remember that research takes a long view. Imagine you wanted a researcher to come and evaluate the autism support in your son’s secondary school. It could easily take more than the five years he is a pupil there to get the funding, complete the project and share the information. Research needs to focus on real issues, yes, but in a way that acknowledges the inevitable long timelines and looks to create a foundation on which future practice can be built. Otherwise the work risks being out of date before it is published, and valuable investment will have been wasted.
But I want to end on a high note – so check out the DART Christmas card if you haven’t already, thanks for reading, and Happy Christmas!
