An important part of the work we do as researchers is reviewing. But reviewing conference abstracts is different from reviewing full papers since you have to judge a researcher’s work based solely on a summary of about 1000 words. That’s quite a challenge! So how can we do this effectively? In this article, I put together five practical recommendations for your next review of a conference abstract – in particular of course with the RENT conference in mind.
I am sharing with you my expertise and experience as a reviewer, who was selected as one of the best reviewers at the RENT in 2019. Additionally, I invited three of my colleagues who were awarded as best reviewers at the RENT as well to share their insights. So now you are getting five practical recommendations from four best reviewers – to step up your reviewing game for the RENT and other conferences!
- Be constructive and respectful
Some people believe that a good review needs to be as critical as possible. While I am all in favour of critical perspectives, each review should offer exactly that: perspectives. My colleague Dagmar sums up our role as reviewers nicely:
I feel that as a reviewer, it is your role to help the authors write the best version of their paper, by providing them with insights that you may have that they themselves have maybe not yet realized. It is thus your responsibility to help them forward [and] to be constructive: your goal is to aid people, not to tear them down. They have put in effort in their work and thus you are responsible for returning this favour. (Dagmar Ylva Hattenberg)
In a small research community such as the ECSB you are likely to meet the person whose work you have reviewed again. Hence, we recommend that we should all definitely return this favour. What I always do is I read the review again after some time has passed, and then ask myself: If I met the author(s) at RENT and had to deliver my review directly to them, would it feel right? If so, you have probably written a constructive review.
- Be specific and transparent
It’s sometimes the small details that make a review really helpful for the authors. If you give references, provide the full references so that people can actually find it. Give the authors some insight into why you think that a specific point is problematic or what kind of flaw there is in the theoretical framework. And, of course, be specific about the paper that you have in front of you not what you would write yourself instead
This last point is especially important for reviewing conference abstracts, because abstracts typically leave the space to imagine a variety of different storylines for full papers. If you feel that it is helpful for the authors, point them towards the various possibilities they have to further develop their paper. However, most of your comments should clearly relate to the abstract right in front of you.
- Structure your process
Let’s see how people are actually doing this and how it works for them when they have the usual 4-6 abstracts to review for the RENT. My colleague Margo takes about an hour for each abstract and structures her process like this:
- Read the abstract
- Read the abstract again, and make notes
- Collect the notes into comprehensive themes for feedback
- Write a few thematic feedback points
- Read the written review again and check the tone and understandability
Now, the actual process can of course look a bit different for everyone. Most of us agreed that we read all the abstracts first, and then each one individually in more depth. There is no perfect process, but having a structure makes your process, and hence your reviews, better.
- Focus on key points
There are always a lot of things you can comment on in conference abstracts, because a lot of information is, of course, still missing. However, avoid writing reviews that are longer than the actual abstract. For me, a good rule of thumb is to focus on the three key points that I want the authors to get from my review. These are typically points that relate to the overall storyline, the expected contribution and the possible further directions for a paper. These major points are what I focus on in my reviews. Your review does not need to cover every aspect of the abstract, but it should open up developmental perspectives for the authors and help them to realize the potential of their work.
- Judge potential in a fair way
Regarding a fair judgement, one of the most challenging things about reviewing abstracts is that you are not judging the complete work, but you mostly judge potential. My colleague Sanna describes it like this:
The papers are in different stages. Sometimes, particularly in early-stage papers, it can be difficult to understand the idea and overall scientific value of the paper. Then it simply requires time to "digest" the paper. It might be that the paper cannot be accepted for the conference but I still try to provide good comments for the future development. (Sanna Ilonen)
As a reviewer, I very much agree that it takes time to digest certain papers, especially if they feel a bit unfamiliar or are not so closely related to what you normally work on. Now to actually judge potential, a key point is to reflect: Will this paper be developed into a full paper by the expected deadline? If it is presented at RENT: will my fellow colleagues be excited about its presentation? Of course, sometimes the answer to this is “no”. Hence, in order to keep up the quality of a conference, you will sometimes recommend that an abstract won’t be accepted. That might feel difficult, but it is just as important for a research community than an enthusiastic acceptance.
Now back to the second part of what Sanna said: even if an abstract cannot be accepted, you can help the authors with the future development of the work. Your judgement call will not always be considered fair or be shared by the other reviewers, but if the authors understand why their work has been declined and they have some guidance how to proceed, they will experience this as a fair process – one that shows respect for your fellow researchers’ work.
These are 5 simple recommendations to guide you through your reviewing process. On behalf of the ECSB: thank you for your dedication as reviewers to developing our supportive community. If you want to discuss further, feel free to contact me on LinkedIn or Twitter!
Author: Verena Meyer, Leuphana University of Lüneburg; ECSB Country Vice-President for Germany