On the occasion of Open Access Week 2020, the University of Groningen Library (UB) and the Open Science Community Groningen (OSCG) organized the online event Celebrating Openness. Keynote speakers and UG researchers explored the benefits and challenges of opening up research. A panel discussion was dedicated to the modified lottery system as a way to assign research funding or prizes. 

The benefits of opening up research

Two keynote speakers discussed the importance of transparency in the research process. Simine Vazire (Professor of Psychology, Ethics and Wellbeing, University of Melbourne) called in from Australia to discuss What is transparency for? Being more transparent – e.g. by preregistering research plans, opening up data and code and publishing preprints – is, according to Vazire, a necessary condition for robust science but not a sufficient one. If you buy a car, lifting the hood does not mean that the car is in great shape. Good quality control is also needed to make sure that the errors that come to light by being more open and transparent are corrected. This can be done by fostering critical appraisal, which is an important part of research that is often discredited. Examples are incentivizing replication and reproducibility, publishing null results or ‘red teaming’: placing a bounty on your research by which you challenge fellow researchers to disprove your conclusions.

The second keynote speaker, Ineke Wessel (Associate professor Experimental Psychopathology, University of Groningen), discussed the Benefits and challenges of adopting open research practices. A case of p-hacking from 2016 made Wessel realize that the academic system tends to reward only positive research results. These help researchers win grants and prizes, journals gain prestige, and universities rank high in top 100-lists. This has led, however, to an increase of false positives and, as a consequence, of flawed publications. According to Wessel, we should instead focus more on ‘the file drawer’ – studies that are usually not published because they show inconclusive results. The journal Memory for instance recently dedicated a special issue to “failures to reject the null hypothesis,” to which also Wessel and her co-authors contributed. Wessel concluded that open science practices have many benefits: diversity in research teams introduces you to new tools, opening up your materials and data allows your research to be corrected by the world, and a multiverse analysis of your data means that you do not need to rely on single p-values. The main challenge is that these practices do take a lot of time.

Open Research Awards

The second part of the event was dedicated to the first annual Open Research Awards at the University of Groningen, to celebrate the many ways in which academics make their research more accessible, transparent or reproducible. The three winners were unconventionally determined by means of a modified lottery . After seventeen submitted entries were screened by a jury , the winners - Merle-Marie Pittelkow, Yoram Kunkels and Marieke Helmich - were randomly selected from of the fourteen eligible cases using a bingo mill. The organization hereby acknowledged the impossibility of ranking the applicants in a competitive manner. 

Besides receiving a prize of 500 euro, the three winners were given the opportunity to present their case studies as lightning talks during the event. Merle-Marie Pittelkow described how a project that addressed the replication crisis in psychology provided an incentive for her to open up her own research as well. She published it on the Open Science Framework platform, shared her data and code, and published a preprint on Psyarxiv . Her message: practicing open science is as much reward as it is work, but you do not need to do this on your own. Read more about Merle-Marie’s case .

The second winner, Yoram Kunkels, has developed a repository for ESM items with a team of Dutch and Belgian researchers. ESM stands for Experience Sampling Methods, which are used to investigate psychological phenomena such as moods in daily life settings. The idea for this repository arose during a 2018 Open Science hackathon and has helped to improve transparency, methods reproducibility and validity in ESM research. Read more about Yoram’s case .  

‘Could you be more specific?’ is what Marieke Helmich thought when she started working on her PhD project on transitions in depression recovery. In her lightning talk she described how open research practices such as preregistration and opening up materials have helped her to translate the broad research question of a research proposal into testable hypotheses, and to navigate the challenge of combining both exploratory and explanatory work. 

Panel discussion: Luck of the draw

The final part of the event was dedicated to a panel discussion about whether modified lotteries are a useful and just way to assign research funding or prizes. The discussion was moderated by Rink Hoekstra and featured Pauline Kleingeld (UG, Faculty of Philosophy), Marie-José van Tol (UMCG) and Marco Bieri (Swiss National Science Foundation, SNSF).

Marco Bieri explained how the SNSF has experimented with a random selection process since 2019. Its selection panels can use this tool when they find that there are multiple, equally qualified candidates for a grant. The SNSF hereby wants to acknowledge that there is a certain randomness in the allocation of funds. Good communication about this has been crucial: applicants were informed beforehand and received a letter of acknowledgement if their proposal was rejected. In the first year 4% of decisions were reached by means of random selection. 

Although Pauline Kleingeld – herself recently awarded with the  prestigious Spinoza Prize – recognized the arbitrariness in the allocation of grants, she found that the real problem of the current system of research funding is its inefficiency: researchers are forced to dedicate a lot of time to grant writing, while their chances of actually getting funded are very low. A modified lottery will not change this, because proposal writing and peer review are still necessary. According to Kleingeld a lottery system would be beneficial if it would help to change the criteria for research evaluation. Early career researchers experience a lot of pressure to acquire grants, because getting tenure depends on it. 

Would it then be a solution to distribute research funding among all qualified researchers? Marie-José van Tol finds this a tempting idea. This would end the costly and time-consuming competition for funding among researchers. She pointed to a study from 2017 which suggests that an “egalitarian distribution of funds” is a worthy solution for every researcher. We should also not forget, Kleingeld added, that the current system is the result of political decision-making by the Dutch government, that years ago has shifted funds from university budgets to NWO, with the result that grant applications have become far more important than they used to be.  

Veracity Award

During the closing remarks Maurits Masselink (OSCG) took the opportunity to announce another annual award, namely the Veracity Award, which the OSCG will hand out to someone who exceptionally contributes to the promotion of Open Science and the improvement of the veracity of research findings. The first winner: Vera Heininga, who, according to Masselink, has been the driving force behind all OSCG-activities. Congratulations, Vera!

Links

All 14 eligible case studies have been published on the Open Science Blog .