What if the results of years of PhD research are disappointing?
You put your heart and soul into research for several years and in the end that promising intervention turns out not to work at all. Or worse: the results do not answer your research question. What does that do to PhD students and their careers? ‘I have terabytes of data, but it is not very useful for the question I had in mind.’
Every year, hundreds of PhD students start their PhD programme at the UvA full of high expectations, enthusiasm and good spirits. Hanne Duindam was one of them about four years ago. She did research on detainees who train hard-to-place shelter dogs. The expectation was that the welfare and behaviour of detainees would improve by training dogs, but a few years later her research showed that this was not the case.
‘It is disappointing when results do not match expectations,’ she says now. ‘Not only for me as a researcher, but also for those involved in the field, such as the people in prison who have been supervising the programme for years and the programme makers who designed the training. If you believe in something and the science doesn't prove it, it is difficult.’
‘Together with my colleagues I did my best to translate the research results into practice. My hope is that despite the initial disappointment, someone will use my research.’
Continuing your research
Scientific journals do not always publish studies with results that differ from those expected, Duindam notices. Fortunately, she does not feel that this was the case with her. ‘I don't think I was disadvantaged, but I don't know how easy publishing is for someone who finds large effects.
In the meantime, she has been able to defend her thesis and started working as a university lecturer at Utrecht University. She would have liked to do more research into dog training programmes during her PhD, but the money is now gone. She hopes other people will take up her research. ‘I would like my research to make people think, for example about who and under what circumstances the programme may lead to positive changes.’
Ambitious start
Pieter Goltstein also remained a researcher after his somewhat difficult PhD project at the UvA. He now works at the Max Planck Institute in Germany. More than ten years ago, he started his PhD research on the reward system in the brain. He wanted to use two-photon microscopy. This technique allows you to map living cells, for example brain cells of mice. Now many researchers use this technology, but in those days it was a different story.
‘I was ambitious when I started,’ he says. ‘There was little literature available. People said to me: “Don't start. You need experience for that. You're not going to find anything.” It took me a year and a half to set up the experiment. In the end, I had gigabytes of data and I had the task to find something useful.’
Goltstein’s results showed the opposite of what an earlier study described. He found that certain brain cells reacted strongly when a mouse saw something on a screen that was linked to a reward. Earlier research in monkeys had seen a weaker response.
Refused
‘It didn’t feel right; I was afraid that the data was not good and I couldn't interpret it properly. The scientific journal Neuron did not accept the article that my colleagues and I had written on this research.’ Even after Goltstein and his colleagues conducted additional experiments, the journal refused the article.
Finally, Goltstein and his team sent it to another scientific journal, the Journal of Neuroscience. The reviewers of this journal wanted to see the analyses in the article in a different way. ‘It took four or five rounds before the publication was completed. YTou get desperate and you don't quite see the point any more of doing this research. My research never seemed to end, but my appointment did. My contract was not extended, so I had to continue in my own time. That was a disappointment, but it worked out in the end.’
Jorrit Montijn started his PhD project ten years ago, studying the brain activity of mice while they saw stimuli and had to respond to them. At first, he had trouble getting the project off the ground. ‘I was trying to teach mice a behavioural task that was actually too complicated. It took a while before I could persuade the professor to change the experimental design. After eighteen months I switched to a simpler experiment. I had to put it back together again; that takes some time. I don't want to say it was a waste of time. Sometimes you just have to try if something works. In the end I got very interesting results and even obtained my doctorate cum laude.’
Publish or perish
‘When the study was accepted, I felt confident that the results were probably correct. Besides, it turned out later that other papers described the same results.’ Goltstein was relieved. But he did need a lot of patience for his publication. ‘Science has become too much of a business environment,’ he says. ‘It is publish or perish.’
UvA graduates who are currently pursuing a PhD still notice this. ‘The harsh reality is that you have to publish to obtain a PhD,’ Julien Fiorilli says. He is at the end of the fifth year of his PhD programme in which he researches how memory works. His results are quite far from what he expected to see: a correlation between brain activity in the perirhinal cortex, a brain region involved in memory, and the recall of objects.
Great expectations
‘As a student you often hear: “If you don't have results, it’s also a result.” But journals are less inclined to publish negative results anyway.’ Fiorilli is currently writing an article, but for this he had to delve into the literature again and perform new analyses. As a result, his entire PhD project has been delayed.
‘It is a mental switch. If after a couple of years you find out that your results are disappointing, it's very intense. How do you carry on then? The first reaction is that you feel everything has been for nothing. Your PhD research feels a bit like your baby, with a lot of emotion attached to it. You have high expectations. Now I try to put that away.’
Fortunately, Fiorilli's work is valuable for further research; others will continue to work with it and publications will follow. ‘I have several terabytes of data; you can look at it in endless ways. It's just not very useful for the question I had in mind.’
After the end of his contract, he will have to continue with the completion of his PhD project. He does not know yet how he will do that. In any case, he is not finished with science for the time being. ‘I am open to continuing research. But I would be a little less naive about it. I wouldn’t be so quick to embark on a high-risk project or a project that puts more emphasis on originality and innovation. I would rather do something that has a greater chance of success, for example, on a topic on which literature is already available.’
