How to Read a Paper

This is a summary of this article by S. Keshav, "How to read a paper,"

Goal

• Minimize wasted effort
• Avoid drowning in the details
• Estimate the amount of time required to review a set of papers

THE THREE-PASS APPROACH

I pass: general idea about the paper

Take five to ten minutes for following steps:

1. Carefully read the title, abstract, and introduction
2. Read the section and sub-section headings, but ignore everything else
3. Glance at the mathematical content (if any) to deter- mine the underlying theoretical foundations
4. Read the conclusions
5. Glance over the references, mentally ticking off the ones you’ve already read

Then, answer 5 C's:

1. Category: What type of paper is this? A measure- ment paper? An analysis of an existing system? A description of a research prototype?
2. Context: Which other papers is it related to? Which theoretical bases were used to analyze the problem?
3. Correctness: Do the assumptions appear to be valid?
4. Contributions: What are the paper’s main contributions?
5. Clarity: Is the paper well written?

Notes

• Adequate for papers that aren’t in your research area, but may someday prove relevant.
• Carefully choose coherent section and sub-section titles and to write concise and comprehensive abstracts
• A ‘graphical abstract’ that summarizes a paper with a single well-chosen figure is an ex- cellent idea

II pass: paper’s content, but not its details.

Read the paper with greater care , but ignore details. Jot down the key points, or to make comments in the margins.

Take an hour (for experienced reader) to do:

1. Look carefully at the figures, diagrams and other illustrations in the paper. Pay special attention to graphs. Are the axes properly labeled? Are results shown with error bars, so that conclusions are statistically significant? Common mistakes like these will separate rushed, shoddy work from the truly excellent.
2. Remember to mark relevant unread references for further reading (this is a good way to learn more about the background of the paper).

If still don't understand the paper:

1. set the paper aside, hoping you don’t need to understand the material to be successful in your career,
2. return to the paper later, perhaps after reading background material
3. persevere and go on to the third pass.

Notes

• Appropriate for a paper in which you are interested, but does not lie in your research speciality.
• Able to summarize the main thrust of the paper, with supporting evidence, to someone else.

III pass: understand the paper in depth.

Four or five hours for beginners, about an hour for an experienced reader to do this pass:

1. Virtually re-implement the paper. Making the same assumptions as the authors, re-create the work, then compare.
2. Identify and challenge every assumption in every statement.
3. Think about how you yourself would present a particular idea.

Then you are able to:

1. Reconstruct the entire structure of the paper from memory
2. Identify its strong and weak points
3. Pinpoint implicit assumptions, missing citations to relevant work, and potential issues with experimental or analytical techniques.

Timing between passes

After first pass, weeks later for a second pass. Weeks or months later to extract the final useful insight.

DOING A LITERATURE SURVEY

Step 1

Use an academic search engine such as Google Scholar or CiteSeer and some well-chosen keywords to find three to five recent highly-cited papers in the area. Do one pass and read their related-work and find a recent survey paper.

Step 2

Find shared citations and repeated author names in the bibliography. Download the key papers and set them aside. Then go to the websites of the key researchers and identify the top conferences in that field they’ve published recently.

Step 3

Scan through the proceedings of top conferences will usually identify recent high-quality related work. These papers, along with the ones you set aside earlier, con- stitute the first version of your survey. Make two passes through these papers.

References

[1] I.H. McLean, “Literature Review Matrix,” http://psychologyinc.blogspot.com/

[2] S. Peyton Jones, “Research Skills,” http://research.microsoft.com/en-us/um/people/simonpj/papers/giving-a-talk/giving-a-talk.htm

[3] T. Roscoe, “Writing Reviews for Systems Conferences,” http://people.inf.ethz.ch/troscoe/pubs/review-writing.pdf

[4] H. Schulzrinne, “Writing Technical Articles,” http://www.cs.columbia.edu/∼hgs/etc/writing-style.html

[5] G.M. Whitesides, “Whitesides’ Group: Writing a Paper,” http://www.ee.ucr.edu/∼rlake/Whitesides_writing_res_paper.pdf