Editors, Reviewers, Graders, Readers

This came up on Twitter the other day. A researcher had been desk-rejected by a major journal less than four hours after submitting. An efficient editorial process is normally a good thing, of course, but this seemed a little too fast in some quarters of academic Twitter. Is it really possible, asked Paul Hünermund, to evaluate the quality of a paper in such a short time? I think that’s an excellent question, not merely a rhetorical one, and I’d like to take a few moments to offer an answer to it, mostly in the affirmative.

To be fair to the journal, we’re not here talking about the full review process, just the editor’s decision about whether to send the article our for review. A journal with many submissions (as a top journal will be) will need to make short work of this decision. Someone probably has to look at several papers every day (at least at times) and decide whether the paper is on the face of it a good candidate for review. A positive assessment here will result in work for (usually) three other scholars (the reviewers) and waiting time for the researchers who submitted the paper, so this decision is important. But it should also take considerably less effort than the full review process.

Also, we should keep in mind that underlying our response to the time it takes to review a paper is our intuition about how long it takes to simply read and understand one. We expect reviewers and (post-publication) critics of our work to actually read it, usually the whole thing, before passing any sort of judgment on it. We might perhaps discuss whether the editors in this case owe the writers a full read of the paper, but let’s try to get a sense of what a reasonable effort at understanding a paper involves.

As always, we will try to appreciate the finitude of the problem. A standard journal article in the social sciences might consist of roughly 40 paragraphs. Each paragraph, I usually say, should be written so that it can be read and understood by a peer reader (a scholar in the same discipline as the writer) in about one minute. So let’s say it takes about 45 minutes, and no more than an hour, to read a paper all the way through once, with comprehension. That is, a good paper presents about 40 claims of various kinds along with the support, elaboration, or defense that the reader needs in order to believe, understand, or agree with them. After an hour or so with your paper it should be clear to the reader what you are trying to tell them and why you think you’re right about it.

If that is not possible — that is, if after an hour the point of the paper is not yet clear — that strikes me in itself as a reason to desk-reject the paper. But if the paper is good, so that that first hour of effort on the part of the editor produced an understanding of the paper’s purpose and, not incidentally, a favorable assessment of the quality of the prose, then it should not take the editor another three hours to decide whether the paper is within the scope of the journal, or whether the conclusions are interesting enough, or whether the data is substantial enough to support them.

I guess I’m trying to say that a good paper is easy to read, given the right reader. That also means that readers can quickly decide that either the paper is not good or they are not the right reader. If the reader is an editor, this can guide the desk decision. If the reader is a teacher, this can guide the grade. If the reader is a scholar, it can simply guide the decision about whether or not to keep reading, i.e., whether they have an obligation, as a peer, to see what a fellow scholar has to say here. Giving an hour to a 20- or 30-page paper to this end seems reasonable to me, even somewhat charitable.

Being kind to your reader is the best way to get your paper through the various hoops that academic life presents us with. Whether you’re trying to get published or to pass a course, write your paper with the aim of making it easy, not hard, to evaluate. Write it in the spirit of exposing your ideas to the criticism of a qualified, competent peer, whose opinion you respect. Don’t expect your reader to struggle for hours to decide what you are trying to say and whether or not it is correct. Give your reader an occasion to respond naturally and intuitively to your prose. (It should, as Orwell said, be like window on your mind. Your ideas, not your words, should be at the center of the reader’s attention.)

The quality of a good paper is immediately apparent. If it takes a long time to assess the quality or relevance of a paper — when, e.g., applying a journal’s editorial standards — then that assessment has actually already been made. And this should also help us enjoy the work of reading more, whether as scholars, editors, reviewers, or teachers. Don’t get frustrated by bad writing. Just put it down for a moment and take stock of the situation. Your frustration is already a sign that there is something wrong with the paper in your hand. It may be that you’re not the intended reader or it may be that it’s not very good. It’s certainly not obviously your obligation to spend the rest of the day struggling with it. Decide how much time to give to it, make the relevant decision or assign the appropriate grade, and move on to the next paper.

Most of us are able to distinguish necessarily difficult ideas from needlessly difficult writing. Read charitably. Write generously. But remember that there must be limits to our kindness, and to the kindness our readers show us. Like being good, reading well means “finding ourselves correctly attuned in the apportionment of the moment,” as Heidegger put it. Much of this is just about planning your time and following your plan. Give every paper the attention it deserves.

Leave a Reply

Your email address will not be published.