Research Paper Evaluations

Here are the guidelines for your paper evaluations. These should be short (about 1 page, 10pt font, 1-inch margins). Remember that the idea is to think critically about the work: insight is more important than quantity.

When you write your review, imagine that you are a journal editor and the paper was just submitted to your very own journal. You don’t want to blindly accept all papers, lest you decrease the quality of your journal’s content, but at the same time if you are too harsh/picky on the small details then you won’t have anything to publish.

Your Name, Paper Title, Paper Authors

A very concise (Twitter style) summary of the work. I won’t check for 280 characters — but aim for a line or so. This is your chance to make a quick, insightful point.

Summary

A few lines, in your own words, that discuss the goals of the paper or specific problems that solves. What are aspects of the system/approach that make it unique?

Critical Analysis

This is the main point of the paper evaluation: thinking critically about the work. This should be the majority of your report content. Here are a few ideas for things you may discuss:

What are the trade-offs being made in this work? What are some situations where the trade-offs do not make sense?
Construct situations which cause the system to fail, or assumptions to break down. In other words, what are the weaknesses of the system?
If any benchmarks were run, did they convey the intended point and make sense? Could you propose a better way to test the system?
What would you do differently if you tackled the same problem?

Accept/Reject Decision

After your critical analysis, you must decide whether this paper is worthy of your hypothetical journal. Your decision can be:

Accept (+1)
Neutral (0)
Reject (-1)

Obviously since we are reading the paper it was accepted somewhere, but that doesn’t mean you have to agree with that decision!

Question(s)

Include a thoughtful question you would ask the author, or points you will bring up in the class discussion. This may relate to your critical analysis or accept/reject decision above.

References

If you used any external references while writing your review, list them here. Representing someone else’s work as your own violates the university honor code.

Grading

For these submissions, grading is based on a simplified scale:

10 – excellent analysis of the work with insightful comments, good grammar, formatting.
9 – good analysis but with some minor issues (inaccuracies, grammar, poor formatting, etc.).
8 – good analysis, but might be somewhat obvious.
≤ 5 – substantially missing the mark, not including all elements, inaccuracies, etc.

In Class

Some of your tweets and accept/reject decisions will be featured in our class discussions.

Submission

Submit your write-up in PDF format via canvas. In general, write-ups will be due on Mondays to help you prepare for discussion on Tuesday.