8.1.6 Keshav’s third pass
It may be that, for Keshav’s paper at least, you’ve already gained enough understanding to make a third pass unnecessary. However, for most academic papers this won’t be true – moreover, for some of the most complex research articles a third, or even fourth, fifth or sixth pass still will not suffice. Academics have been known to spend their whole working lives trying to understand (and build on) the research covered by a single paper.
Keshav offers some very useful observations:
Sometimes you won’t understand a paper even at the end of the second pass. This may be because the subject matter is new to you, with unfamiliar terminology and acronyms. Or the authors may use a proof or experimental technique that you don’t understand, so that the bulk of the paper is incomprehensible. The paper may be poorly written with unsubstantiated assertions and numerous forward references. Or it could just be that it’s late at night and you’re tired.
You can now choose to: (a) set the paper aside, hoping you don’t need to understand the material to be successful in your career, (b) return to the paper later, perhaps after reading background material or (c) persevere and go on to the third pass.
Decision criteria for a third pass
As Keshav argues, not all papers require a third pass and you should take care in deciding whether a particular paper should have one at this particular point. We say this because the importance of understanding a paper may only become apparent later – it may turn out to be a highly cited paper that needs to be understood so that other papers become accessible; or, it may be that a career change on your part means you need to come back to it. Here we see the motivation for annotating a paper and recording your present understanding: if you find you do need to return to the paper at some point, you won’t start from scratch if you’ve made good notes. This also makes clear why you should keep a master version of a paper, with your earlier notes stored with it.
If you decide that investing time in a third pass of a paper is indeed necessary, Keshav’s key skill is to attempt to ‘virtually re-implement the paper’ – that is, making the same assumptions as the authors: recreate their work. This means being able to restate their assumptions, and to recreate the reasoning they use to reach their conclusions. Essentially, you are asking whether, starting at the same point, you could have made the same contribution to knowledge or practice as the authors. As an aside, that’s a good definition of truly understanding an academic paper.
There are many reasons why you might not have been able to make the same contribution, by the way – and not all of them point to a lack of understanding (or creative genius) on your part:
- perhaps the paper has failings, makes inappropriate assumptions, and/or fails to cite other work that you know of
- perhaps the arguments the paper puts forward are circular and/or nonsensical
- perhaps the research method used does not make sense.
Keshav adds: ‘During this pass, you should also jot down ideas for future work’. This is an excellent idea, should you be a researcher (as Keshav is), or a practitioner, and you may wish to identify projects or the like to which the understanding you have gained can be applied.
According to Keshav, the third pass will take about four or five hours for a beginner, and about an hour for an experienced reader. However, longer periods are quite possible: indeed, whether even more passes are required will very much depend on how critical to you the paper is.
Activity 20
Carry out a third pass on Keshav’s paper, and then try to construct a virtual re-implementation of the paper.