When you review a conference or journal paper submission, you are providing an important service to the research community. After all, the quality of one’s research is often judged by the resulting peer-reviewed publications, and the moment you start your research with me, you have already become a peer of our community.
As a student, reviewing papers is also an excellent opportunity to learn how to write a good paper—which is the main reason why I ask you to review papers for me. Providing a review is an intensive thought process, from which you may be inspired with new ideas, learn how to better shape your own ideas, and learn how to better present your results. Putting your thoughts down into a review is always a good writing exercise. Finally, for international students, the experience of providing peer reviews may help you get your green card sooner.
If you are a first-time reviewer, you should read this article on “The Task of a Referee”. Before you start your review, remember the following key points:
- Take your review seriously. Otherwise, your review is of no use to the authors and no use to yourself.
- Become knowledgeable on the subject. If you need more knowledge, read more background materials—look for pointers in the background and related work sections of the paper, and search on Google (Scholar) for additional materials.
- Be thorough. Consider all of the following aspects when examining the paper:
- Novelty: is the paper’s idea new compared to the existing works (first, you need to know what are the relevant existing works)?
- Significance: does it solve an important problem?
- Soundness: is the paper technically correct and convincing in its design, implementation, and evaluation?
- Presentation: is the paper well organized and well written?
- Related work: has the paper provided sufficient (qualitative and quantitative) comparisons to the closely related works—search on Google Scholar for any missed related work
- Be reasonable and constructive. No paper is perfect. Compare this paper to those published works you have read. And, don’t just criticize—try to provide some suggestions for the authors to improve their paper.
- Turn in your review on time. Nothing good comes out of procrastination, and in this case, you hurt not only yourself but also the authors if you procrastinate. Writing a good review takes a good amount of time. Manage your time well.
- Read my final reviews. I personally review every paper myself, and I will incorporate your review into mine. Take a look of my final review (ask me if you don’t have access to it) and see what you missed and what I skipped. Discuss with me if you don’t get why I made those changes. This goes back to the first point—you learn fast only when you take things seriously. Eventually, you’ll be ready to review papers independently, when you are called up by a program committee chair or editor to offer your service to the community.
Here’s review I did a while back and you can use it as an example.
A few additional sources of advice on writing reviews:
- Timothy Roscoe. https://people.inf.ethz.ch/troscoe/pubs/review-writing.pdf
- John Ousterhout. https://people.eecs.berkeley.edu/~fox/paper_writing.html#rev
– Prof. Zhao