Open peer review. An interview with Casper Albers

Peer review plays an important role in academic publishing. A particular type of peer review is emerging: open peer review. This reflects a growing trend towards more transparent scholarship. Here everyone knows the identity of the author and the reviewers. Some journals also publish reviews together with the final accepted article, so readers see both the identity of the reviewers and their comments. PeerJ is a journal that offers this possibility. Casper Albers , Associate Professor of Statistics at the Faculty of Behavioural and Social Sciences, published an article in PeerJ and chose to make the accompanying review history public. We asked him some questions about peer review.

What is the most common form of peer review in the social sciences?
The most common form is the classic form of peer review: single-blind peer review. Here the reviewers know who the author is, but the author doesn’t know who the reviewers are. PeerJ is the first journal that I’ve published in to make the peer-review process open to the extent that it does. I always sign my reviews; that’s my own choice. There are also journals that only let you review for them if you sign the review. But PeerJ goes a step further. Not only do they do that, but they also let you publish all reviews and replies to reviews online, so that everyone really can see every step of the peer-review process.

What is the process at PeerJ?
As a reviewer you can choose whether you want your name published or not. As an author you can choose to have everything published online. Both author and reviewers must opt for the peer-review process to be published online. It’s not compulsory but PeerJ does encourage it. I see no reason not to have everything published online. I like to be able to work like this with reviewers. I’ve also been lucky enough to have really good constructive reviews. I’m even planning to work with one of my reviewers on a follow-up paper.

The reviews of your paper on PeerJ were detailed; is this the norm or were these the exception?
How detailed a review is can differ. Sometimes you get really superficial reviews, but sometimes they’re detailed, right down to the finest detail. And reviews of fundamental points, which means it can take a good deal of time to revise the article.

One alternative to open peer review is double-blind peer review. Here the reviewers don’t know the identity of the authors and vice versa. This is supposed to counter reviewer prejudice against the author (sex, race, affiliation), making for a more honest review process. What’s your opinion of this?
I don’t understand double blind at all, particularly if you’re in a niche discipline. It’s then fairly easy to guess who the authors are from the paper itself and the papers that it cites. I do understand that blind reviews counter prejudice, but I think there’s a whole load of disciplines in which blind reviews simply don’t work.

Yet another variant of peer review is ‘post-publication review’. Here a paper is, or continues to be, reviewed and revised after publication. For instance, in the form of a comment page or a discussion forum alongside the published article. Do you have any experience of this with journal articles?
No, not with journal articles. At least, not that a journal provided such space itself. I did recently publish a preprint of a paper of mine online and got feedback on it: about why I’d chosen to do it in the way that I had and whether that had led me to make a mistake. But this type of review is behind the scenes or on Twitter, and you then need to chance upon it. It would indeed be great if journals did provide somewhere for post-publication reviews, for instance space for comments on a paper. Because it’s only when a paper is published that it’s brought to the attention of a wider audience.

Wouldn’t you worry about trolls or self-proclaimed experts saying odd things?
Everyone knows that odd people say odd things on the internet. And it will undoubtedly be the case that if you have such an option all sorts of spam will be posted there. But the best thing to do is to appoint a kind of moderator who looks at whether a response is to the article itself. There are websites that provide the option of post-publication comments and there isn’t much ‘vandalism’ there. The academic community as a whole is always trying to progress and that simply means working together.

Back to the open peer review history at PeerJ: for students, and PhD candidates in particular, it’s also very interesting to see how such a peer-review process works. This means that it also has an educational role.
That’s right. We see that with the PhD candidates in our group. At a certain point they submit their first paper and receive reviews. It’s nice if they can then see how to write a response letter. And I think that an open peer review history can also prevent an old-boys network of people who keep reviewing each others’ papers and almost have an agreement with each other along the lines of, ‘I’ll be nice to yours if you’ll be nice to mine.’

Do you see any drawbacks to this type of peer review?
Personally I don’t, but I can imagine that others do. If, for instance, you’re reasonably negative about a paper and this paper is by more renowned members of your field, I can imagine you might rather that they didn’t know that the criticism came from you. Particularly if you’re still at the beginning of your career and have a temporary job and need to look for a new one, perhaps at the university where these people work. Then it can be risky for you. I can certainly imagine that these are real worries for early-career researchers.

And finally, do you think that open peer review should become the norm?
As I just mentioned, I can see reasons why a reviewer, particularly an early-career researcher, may sometimes prefer not to express public criticism. I therefore don’t think open peer review should be compulsory. But PeerJ doesn’t do that anyway: the author and the reviewer can both say ‘no’ and that’s enough to halt the process. But I would be in favour of this system being the norm (open peer review, unless...). In principle nothing changes – you decide for yourself whether it goes ahead. It’s the approach that changes: the open peer-review process becomes the norm.