The Ethics of Peer Review in the Age of Adjunctification

Academics inhabit a world in which the difference between having an article published or not can mean the difference between landing a tenure track job or not. Later in an academic’s career, the difference between one or two articles and a few might mean the difference in earning tenure or not. Peer reviewers are often in the position to make decisions that can change people’s lives. So why does peer review often take so long — months and sometimes years? Committing to timely peer review is a vital ethical resolution that might significantly change the academic landscape.

dkng-hitchcock_big

Recently, peer review has become the subject of some discussion in the academic blogosphere. Some academics have argued for the ability to track versions of an article after publication, so that corrections could be made to online versions after publication, thereby leading to commenters providing positive rather than negative reviews. Others have suggested that a quid-pro-quo approach might lead to more timely and careful reviews. Regardless of the overall structure of how academic publishing happens — and I do think online, easily amended articles is a great idea and might significantly change citational practices — every peer reviewer could make a change for the better by committing to turning in a peer review within two weeks of being asked to review a manuscript.

Two weeks might seem arbitrary, but here’s my reasoning: if it takes longer than two weeks to get around to doing something, it usually takes a very long time. That is, most of us are pretty good at scheduling in the short term — a week or two — but when it comes to scheduling beyond the next month, things get nebulous. When an article manuscript falls into that nebulous beyond-the-next-month period, it’s probably going to get lost in the shuffle. And when it comes time to read it, it’ll probably be because an editorial assistant is hounding you and not because you scheduled to read it in two months’ time. This means I’m always scheduling a peer review, even if I don’t have a manuscript on hand. If I don’t get asked to do a peer review, then it’s no big deal. But I’m ready if I am asked and don’t feel put out by the work.

Elsewhere, I’ve suggested the qualities that make for a productive peer review — generally, it boils down to helping the author make the most of the manuscript at hand. It might not be suitable for the journal that you’re reviewing for, but that’s usually up to the editor to decide. For the reviewers, the question is: what would it take for this manuscript to be published as an article in this journal? Answering that question might take a couple of hours of work — reading the article and writing up comments — and I would guess most of us spend two hours a day reading the news, checking social media, playing video games, or otherwise distracting ourselves from work. That can all wait; people’s careers can’t. Why not just commit to using that time for one’s peers, and when taking a break from one’s work, working for someone else?

We’ve all had long waits for peer reviews to come in, confusing editorial recommendations, and egregious publishing experiences, which has led me to develop these peer review practices, which might work for you too:

1) I always turn a review around in 2 weeks or less. If I don’t know the journal, I’ll take a few minutes to scan a couple articles to see if there are particular conventions in the journal’s published articles, just so I’m on the right page. I usually read a manuscript one day, taking notes while I do so, and then write the review the next day. If particular concerns nag me over the day, I’ll go back and read specific sections of the article again, just to make sure I read it right. My reviews tend to be 1-2 single-spaced pages, and focus on what it will take to make the article publishable. No snark, no random free association. Even if a manuscript is publishable as is, I still take the time to write up a review of what the author has done right, just so if some other reviewer has a different opinion, the editor and author have a sense that at least some readers are on the author’s side.

2) I never agree to review more than one manuscript at a time. If something comes in that I really want to review, I quickly review the manuscript already in my peer review queue and then agree to review the new manuscript.

3) If I can’t turn a review around in 2 weeks, I just say no to the invitation to review. Similarly, if the manuscript is way outside of my wheelhouse, I’ll also say no. But whenever I say no, I try and send the editorial assistant 3-4 names of other people who might be tapped for a review (no need to thank me, friends!); often, junior faculty aren’t really on the peer review map until they have a few publications under their belt, so it can be a benefit to both the reviewer and reviewee to send a manuscript to an untapped junior scholar (doing peer review makes people better writers… trust me on this).

4) If I’ve agreed to review the manuscript and find that I can’t be a kind peer reviewer for some reason, I get in touch with the editor and ask him or her to find a different reviewer. If this happens in the first two weeks that an article is out for review, it’s no big deal for the editor to turn around and find a new reviewer. But if it’s three months into the review process, it’s very harmful to the author of the manuscript, since now they have to begin the waiting process all over again…

Even if your final assessment is that a total overhaul is necessary, knowing that sooner rather than later will allow the author to get on with the necessary work — which might mean finding another journal. In a work context where people have very little time to focus on their own research and writing, being able to schedule necessary revisions is critical.

You might be the fastest of a set of reviewers, and so things will slow down while the editor waits for another review or two to come in. But if everyone starts reviewing more quickly, the whole machine of peer review should speed up noticeably for everyone. Not only will the academically precarious benefit, but so should scholars throughout the academic life course. If you’ve ever experienced a slow review process, commit to making it better for others by being a timely reader. Or at least refrain from agreeing to read something you don’t have the time for.

Three Pathways to Publication (Excluding Deals with the Devil)

In this post, I discuss three of my articles and what the experience was of getting them through peer review and their ultimate publication. Some of the details are a little foggy — in some cases, it’s been upwards of seven years since I started work on the articles mentioned here — but most of what I want to convey is that getting a manuscript to publication depends on two things: 1) don’t take criticisms and rejection too seriously, and 2) make sure you’re writing for the right audience. If you’re doing the second, the first should be minimized anyway…

Faust_und_Mephisto,_Stich_von_Tony_Johannot

I should say at the outset that I’m leaving names of journals out, since editorial shifts happen frequently and my experience of any of the journals I’ve interacted with in the past isn’t predictive of your experience with the same journal or editors. I’ve also kept all of my peer reviews, editorial decisions, and other paperwork from all my publication efforts, and I draw on them below.

So, first off is ‘Natural Hegemonies.’ I had written it as a job talk back in 2006, and presented it in a few of different versions as colloquium talks and workshop presentations. It’s sprawling, and covers a lot of the content from my dissertation — and later, book — in summary format. It was usually well received by audiences, and so it seemed like the good basis for an article to send to a flagship journal. After some fine tuning, I sent it to an anthropology journal generally seen as one of the top of the field. It spent a long time in peer review, and when it returned to me it was returned as a revise and resubmit with five reviews. The reviews were generally positive, with a couple very glowing reviews. The editors had a series of revisions they were seeking, and I set to work on them; they also, in retrospect, had deep philosophical disagreements with me, and if I was reading clearly at the time, I would probably have pulled up stakes after the first round of reviews…

After revising it, I sent it back, and this time it was under review for even longer than the first time. When it was returned to me, it was sent back as a rejection, despite having generally positive reviews. Reviewers said things like ‘I found just about every aspect of this text compelling and thoughtful. I urge publication of the manuscript in its current form’ and ‘This is a great paper, which I enjoyed reading immensely. The paper is engaging, flows, and connects together a series of seemingly unrelated themes together in such a way that they appear in retrospect to be inevitable and one wonders why one never thought about it before. This is the hallmark of a very engaging and persuasive argument, one that appears impossible at the outset and inevitable in retrospect. I’d certainly publish it.’ But the editors disagreed. Which, ultimately, was fine. I worked on the manuscript for a couple of hours — addressing only the big things pointed out by reviewers from the last round of review — and sent it to Current Anthropology the same day it was rejected from the other journal. After a round of peer review, it was accepted with minor revisions. I set to work on those revisions, and it ended up coming out in 2011.

What I realized in working on ‘Natural Hegemonies’ was that the kind of article it was — sweeping in its empirical scope and making a pretty theoretical argument — just wasn’t right for the first journal I sent it to. Despite the peer reviewers, the editors saw the mission of the journal as promoting a different form of anthropological scholarship, which my manuscript didn’t quite fit. But Current Anthropology offered a more ecumenical approach to the discipline, and the article fit right in there.

A more straightforward path was that of ‘Therapy, Remedy, Cure.’ I originally wrote it as a colloquium talk around the publication of my book, and presented it a couple of times over the course of a year. Because it was largely ethnographic and developed an argument I saw as of potentially broad appeal — about time, capitalism and medical treatment — I decided to take another stab at publishing in a flagship anthropology journal. Based on what I had seen in the journal recently, I thought the editor would be interested in the piece, and work with me on revising it as needed. But the editorial reigns had just been handed over, and the new editor seemed to have different tastes, including kinds of reviewers to send things to for review.

Of the three reviews that were returned to me, the first was principally concerned that my methods section was a footnote rather than in the body of the article, which would seem easily remedied (and was an artifact of the paper as a presentation). The second reviewer had a number of suggestions, and asked for revisions prior to publication. The third reviewer wrote a three sentence — and extremely positive — review. Despite the reviews, the editor ‘definitively’ rejected the piece. Not to be dissuaded, I sent it out the same day to Medical Anthropology, a journal I had published in before and always had excellent experiences with.

At Medical Anthropology, the piece received favorable reviews, and was accepted pending revisions, most of which were minor. In my experience, sometimes minor revisions are the hardest to make, since they’re usually just to satisfy specific peer reviewer concerns, and they always stick out to me as being just that. But I set down to work on the manuscript, addressed what I needed to, and sent the article back for review within a couple of months, at which point it was accepted for publication. Again, although I thought the argument would be of broad disciplinary interest — maybe it is? — the best home for it was a subfield journal where the kind of evidence and argumentation that was the basis of the manuscript was easily recognized and supported.

Maybe the easiest — yet longest — experience I had was with ‘Where Have All Our Naps Gone?‘ The meat of this paper was in my dissertation (which focuses on various experiments with sleep over the 20th century), but it didn’t really fit into the book version. Being largely historical in its focus, I decided to send it to history journals. Over five years, I sent it to five journals. Maybe it was even more than that. Of those journals, a couple times it was rejected without peer reviews — the editors simply thought it wasn’t interesting and right for their audience. A couple other times it underwent peer reviews, and was ultimately rejected for one reason or another. I was asked to present something at a workshop on sleep in the 20th century, so I dusted the manuscript off and presented it there, where it received a favorable response — which reaffirmed my sense that there was something to the argument, but that maybe it just wasn’t right for historians.

I was asked by Peter Benson and Rebecca Lester to guest edit an issue of Anthropology of Consciousness on sleep, and it dawned on me that maybe this would be the right venue for the piece. Instead of writing a lengthy introduction to the issue, I asked for the article to go through peer review for consideration in the journal. The review process was relatively painless, which may have been because it was a special issue, or because I had been working on the manuscript on and off for so long — or maybe because it finally found the right audience. At Anthropology of Consciousness, the article ended up winning the annual ‘outstanding article’ award — something I didn’t even know existed, but it made it evident to me that the article had finally found its audience. Maybe historians will get turned onto it sooner or later…

Over the years, I’ve learned to not take rejection as an indictment of my abilities, my research or my writing. The biggest help on this front was peer reviewing other people’s work, often that of people I knew and respected. Critically reading through other people’s work with the goal of helping them publish also helped me see some of the mistakes that I was making. Going through peer review and doing peer review significantly changed my thinking about writing and how finished something needed to be before it could be sent out — a three-quarters finished manuscript might risk rejection from an editor, but it also gives reviewers a lot to respond to and help you work through.

My general rule is to write a new article manuscript every year. Part of my reason for doing so is that I don’t like presenting the same material as colloquium talks more than a few times, so I’m always looking towards the next presentation. The other reason for doing so is that it takes the burden off any one manuscript to get published. If something gets hung up in peer review for a year or two, like ‘Natural Hegemonies’ did, I know some other stuff will make it through to publication in the meantime.

But, generally, rejection is no big deal. This isn’t to say that you should ignore why things are getting rejected, but that rather than over think why things are being rejected, you should take the criticisms seriously, address them as succinctly as you can, and move on. Targeting the right audience will reduce your overall rejection rate, but it’s absolutely normal for an article manuscript to be rejected once or more on its way to eventual publication…

How to Write a Journal Article (in 6 Steps): Step 6 — Fine Tuning

After writing your conclusion, literature review, empirical evidence and introduction, you have a full article manuscript in hand, and it’s time for some fine tuning before you send it off for peer review to your journal of choice. Fine tuning is really about being deliberate and making sure that the whole manuscript works as a piece of sustained argumentation. More than anything else, you want to make sure that your manuscript is consistent. It doesn’t need to be perfect — peer review is there to get it as close to perfect as it can get — so just make sure it’s 85% complete and that there aren’t any huge gaps.

sand-patterns

The biggest challenge in sending a manuscript out for review is coming to terms with it not being 100% complete. But, fundamentally, a manuscript is never done — and it’s up to the peer review process to help you finish it (at least enough for it to be published). An article isn’t a definitive statement, but rather part of an ongoing conversation (or maybe a conversation starter). As such, the burden is just to carry the conversation forward — not to bring it to a conclusion. Accept that an article is never complete, and get ready to send it out for peer review.

With that in mind, here’s the checklist:

1) Make sure that your argument is well articulated and flows throughout the manuscript. Along with that, make sure that keywords that appear in one part of the text appear throughout (e.g. if you’re talking about biopolitics in the conclusion, make sure that it’s in your introduction, lit review and cases). Read it through once on paper or in some other not-easily modified way and take notes on what to fix (editing at your computer can descend into lots of new writing, and you should avoid that at this point). Maybe take a day or two off and read it again. And then sit down to work through the corrections on the manuscript. I tend to find that having someone else read it during this time to be helpful, as I get a little myopic in my reading of my own work after working on it closely for a while. So a fresh reader can be a great asset, especially when it comes to seeing the inner workings of an argument.

2) Verify that there aren’t any non sequitors or holdovers. Hopefully you haven’t done a lot of copying and pasting into the manuscript, which usually increases the number of these kinds of artifacts. In any case, read through the manuscript and make sure that everything you say will be done is actually attended to, and that you don’t make any presumptions of what happened early in the manuscript at late points (look for those telltale ‘as mentioned above…’ and ‘below’). And make sure that you don’t refer to any evidence that isn’t in the manuscript.

3) Check your citations and bibliography. Make sure that everything that should be cited is cited, and that the citations appear in the bibliography. It’s always a headscratcher as a peer reviewer to check a citation that’s unfamiliar to find that it doesn’t appear in the bibliography…

4) Ensure that the manuscript meets the journal’s formatting guidelines. Every journal should have this information posted on their ‘For Authors’ or similar page, including their bibliographic style preference and other style concerns. Make sure you follow these as closely as you can (although sometimes things slip through the cracks — which isn’t anything to worry about), and know that the more closely you can follow them, the more clearly you demonstrate to the editor that you’re serious about publishing in his or her journal and have done your homework. The most important thing here is to make sure that you meet the word limit requirements, usually a little short of target so you have room to revise when it comes time for that.

5) Write your abstract, pick keywords, and write a cover letter. The length of abstracts can vary quite a bit, so make sure you know what you’re shooting for — they tend to be anywhere between 150-250 words. I usually find it helpful to take a summary paragraph from the conclusion of an article manuscript and whittle it down into an abstract. Such a technique ensures that you’re talking about all the things you need to: the argument, the evidence, and the structure of the article. Remember to pick keywords that aren’t in your title (which would be redundant). And prepare a cover letter that briefly states the source (e.g. your dissertation research), intent and word length of the article manuscript. (This all might be worth an additional post…)

If you can, try and do a peer review swap with a friend before you send your article out for review. Make sure that your prospective peer reader is aware of the journal that you’re sending it to and the subdisciplinary or regional debates you’re entering into — you don’t want them to read an article as a ‘general’ reader, since that’s not exactly who you’re writing for. Instead, make sure they’re reading like a specialist. You want to make sure the comments they’re giving you are relevant to your immediate needs, and although a general perspective can be helpful, when you’re targeting a specific journal, such comments can often be a distraction.

So that’s it. Get to work (or keep working), and know that it can be anywhere from 3-12 months to hear back from a journal’s editor. Don’t sit on your hands and wait though, get to work on the next article manuscript

How to Write a Journal Article (in 6 steps): Step 1 — Analyze a Model

Writing an academic article isn’t as difficult as it might seem from the outset — as long as you have enough evidence, a clear sense of the existing literature, and a good model to work from. In this first of several guides to writing a journal article, I want to work through this last element: a good example from an appropriate journal. (This is intended for anthropologists, but it might apply more generally.)

1The first thing to do is identify a journal you’re interested in publishing in. If this is your first article, target a journal that focuses on your subfield or geographical region of interest. Generally speaking, these smaller journals have word lengths of approximately 6,000-8,000 words. These short articles tend to focus on one key idea from your research, and mobilize 2-3 cases to support it. Once you have a significant number of dissertation chapters written, it should be relatively easy to weave together a first draft. But putting something together and making it relevant to the journal you’re interested in publishing in are two different projects. So the best thing to do is to find a good article published in the journal you’re targeting and work from that to get a sense of what the journal is looking for.

A good model isn’t one that comes from a senior academic. Instead, find one published in the last 12-18 months by an assistant professor or someone recently graduated from their Ph.D. program — someone roughly like you. Your model author will also be working from dissertation material, which is significantly different than the kinds of evidence later-career academics work from. Moreover, because junior people are working to position themselves in the field, the burden of their articles is significantly different.

If the editorship of the journal has changed hands since the publication of the article you’re working from, be sure to look at any introductions to the journal that the new editor(s) has written. If the journal is changing its focus or generic form, the new editor(s) will generally make that known early in their leadership.

Once you have the model in hand, read it once all the way through. Then, circle back with a highlighter and read it again. You’re going to need a few different colored highlighters for what’s about to come, and each time you read the article, it will get shorter and shorter…

On this first pass, highlight all of the primary evidence in the article — all of the actual empirical content generated by the author’s research. So for most anthropology articles, this means descriptions of spaces, people and events; it also includes quotes from interviews and other qualitative (and sometimes quantitative) data.

On the second pass, highlight (in another color) all of the secondary evidence in the article, but focus your attention on the content outside of the literature review. This often includes paraphrasing other authors, as well as historical or other anthropological work on the same topic. This does not include theoretical citations.

On the third pass, highlight (in yet another color) all of the argumentative content — the thesis, the topic sentences, and wherever else claims are being made (there shouldn’t be too many in the introduction or conclusion). This may include theoretical citations, especially if the author’s purpose is to argue with a set of theories or theorists.

On the fourth and final pass, highlight (in still another color) all of citations in the literature review section of the paper. Most journal articles should have 1-2 pages where the author is positioning the research alongside other work in the same subfield, other approaches to the same topic, and other research on the topic and subfield in the same geographic region. In some cases these sections can be quite long, but in most subfield and area journals, they’re relatively short.

Once you have this set of tasks completed, you should have a well marked up document. It will provide you with a few things: 1) a sense of how much evidence you need for an article of the same length, 2) a feeling for how much secondary literature you need to engage with, and 3) a scaffold of an argument and its relation to the empirical content that supports it.

With this evidence in hand, you should begin to think about your work and what might be successful at the same scale. Short articles in the 6-8,000 word range usually only have one substantial argument, and use a few cases to argue it. (For an example, you can look at an early article of mine, here.) Generally, your argument can’t get too complicated — you need a well defined problem and interpretation of it for a short article — so it will often be less than a dissertation chapter, or might borrow content from a number of chapters.

Like I discuss in relation to developing a publishing strategy for your early career, often when you’re writing for subfield or area journals, you’re making an argument with the existing literature in that field. So, what does your dissertation research add to dominant approaches to your subfield, area or topic? Just tackling that question is enough for a first or second article, because, in the beginning, you’re trying to do two things: First, you need to get people to pay attention to you, and, second, you need to start putting out articles that you can cite to support later, more complicated arguments in longer, more complex articles… But early on you need tight, short articles that make it clear to your readers what your interests are and what debates you’re contributing to at this stage in your career.

And here is Step 2.

The Elements of Productive Peer Review

One of the most important academic practices to learn is that of peer review — both producing peer reviews of other people’s work, and learning how to read peer reviews productively. And yet, we’re rarely taught how to write generative peer reviews, and even more rarely are we taught how to read peer reviews to improve our work. So, here’s a first stab.

Preparing Peer Reviews

At the outset, I should say that many journals have peer review expectations, and these are embedded in forms that they ask peer reviewers to fill out at the time they submit their peer reviews to the editors. At the most basic, they ask reviewers to evaluate the submission in terms of its ability to be published (e.g. publish without revisions, publish with revisions, revise and resubmit, do not publish). Beyond that, though, most peer reviews are rather free form and are left up to peer reviewers to do with it what they will. That being said, useful peer reviews tend to share some qualities:

Make sure that throughout your review to flag what the author needs to address and what she or he may choose to address. More than anything else, this can really help make the difference between a useful review and one that’s not so useful…

Start with a summary of the author’s argument, as you understand it. This is a pretty good index to the author as to whether or not he or she is conveying their argument, and what, if any, interpretive problems are happening on the reviewer’s end. If the reviewers just aren’t getting it, then there’s a big problem. Or if the reviewers are missing a nuance in the argument, that’s important to see too. This summary needn’t be long, but it should paraphrase the thesis and then cover the argumentation (i.e. the argument is X, which is followed in the paper by discussions of A, B and C). If there are logical inconsistencies in the argumentation, this is the place to point that out. If there are argumentative problems, this section can be much longer — upwards of two pages. But if it stretches beyond that, it needs to be broken into subsections to help the author interpret what to do and where specific problems lie.

The middle section of a peer review usually falls into the realm of free association, which is where the reviewer can spend some time discussing the relative merits and shortfalls of the paper, the paper’s linkage to other existing scholarship, and the overall consistency of the paper. Throughout, it’s important to flag which comments need to be addressed by the author and which are not so critical: it’s fine to go on for a paragraph or two about a pet peeve or some flight of fancy, but if it has little bearing on how the author should be rewriting his or her paper, make sure to flag that (e.g. ‘The author might be interested in…’ rather than ‘The author needs to address…’).

In the middle section of the peer review, it’s worth taking the time to discuss each of the sections of the paper (as the author has it broken down), including a brief description of the section along with some evaluative language (does it work or doesn’t it?). If there are parts of the paper that just don’t work or parts of the paper that really do work, this is the place to point them out. Praising things about the paper is just as important as saying critical things: if one reviewer really likes a section and others do not, it’s helpful to see why that’s the case.

Connecting the article to existing literature that hasn’t been discussed by the author also usually falls into this middle section. Again, it’s important to flag what needs to be addressed and what might be addressed. If an author is making vague references to a body of theoretical literature and would really do well to spell out her or his assumptions more directly, point that out; if an author has a huge blind spot in his or her discussion of relevant literature, point that out too. But if there’s some tangentially related literature that you know but the author doesn’t — and it won’t really have any bearing on the article anyway — make sure to mention that citing that work isn’t consequential, but might be interesting to the author.

The last section of peer reviews usually focuses much more on very specific things that need to be addressed by the author: are there specific awkward or vague sentences, missing citations, bibliographic errors, etc.? This isn’t to say that a peer review’s job is to find syntactical or grammatical problems, but if there are writing issues that interfere with the ability of the paper to be read, these problems need to be pointed out to the author.

The final paragraph of the review should succinctly restate your overall assessment of the paper and outline the major things (if any) that need attention on the part of the author.

It may seem paradoxical, but a really good review can actually be a bad review for an author. I’ve received reviews that say things along the lines of ‘This paper is ready to be published; the author shouldn’t change anything.’ (Usually only after already having passed through peer review multiple times, mind you…) But if the other reviewers are highly critical of parts of the paper, such a blanket positive peer review doesn’t help much. Instead, working through the positives of the paper is more helpful for the author, so that she or he can see why you like sections or the paper as much as you do — if you like things that others do not, it’s vital to see why that is. And, if there’s an editor involved, they’re more likely to be swayed by articulate, negative appraisals than a blanket and vague positive one.

Interpreting Peer Reviews

If you’re an author and have peer reviews that look roughly like what I’ve outline here, there should be very little problem in interpreting them. But, more likely, you’ll have peer reviews that don’t strictly (or even closely) follow this format. The best way to tackle reviews is to read through them and identify those things that reviewers think need to be changed (and thereby isolating them from less pertinent critiques). Often, the best way to do this is to see if there is convergence between readers: are people having similar problems with the paper? If they are, that’s fairly easy to see and address. If they’re having wildly different kinds of problems, it’s worth writing down the criticisms and seeing what the problem lies: is it that they really understand your argument differently? Are they coming from very different interpretive traditions? Sometimes you can address these problems; sometimes, it’s just the luck of the draw. And the important thing here is to isolate what you actually need to address and what you don’t. In not addressing some concerns, it’s important to point out why — sometimes in the body of the paper itself (which may be a way to get your argument more precise).

If you’re preparing to resubmit a paper after peer review, editors will often ask that you enumerate the changes that you’ve made to the text — and which changes you haven’t made and explanations as to why. If you have a list of requested changes, this is a fairly easy document to prepare, as you can list which changes you’ve made, how the criticism has been addressed, and where the change appears in the paper. Doing this can be helpful both for the editor and for peer reviewers, who often want to see that you’ve made the changes that they’ve suggested.