Matthew Wolf-Meyer Professionalization

The Elements of Productive Peer Review

One of the most important academic practices to learn is that of peer review — both producing peer reviews of other people’s work, and learning how to read peer reviews productively. And yet, we’re rarely taught how to write generative peer reviews, and even more rarely are we taught how to read peer reviews to improve our work. So, here’s a first stab.

Preparing Peer Reviews

At the outset, I should say that many journals have peer review expectations, and these are embedded in forms that they ask peer reviewers to fill out at the time they submit their peer reviews to the editors. At the most basic, they ask reviewers to evaluate the submission in terms of its ability to be published (e.g. publish without revisions, publish with revisions, revise and resubmit, do not publish). Beyond that, though, most peer reviews are rather free form and are left up to peer reviewers to do with it what they will. That being said, useful peer reviews tend to share some qualities:

Make sure that throughout your review to flag what the author needs to address and what she or he may choose to address. More than anything else, this can really help make the difference between a useful review and one that’s not so useful…

Start with a summary of the author’s argument, as you understand it. This is a pretty good index to the author as to whether or not he or she is conveying their argument, and what, if any, interpretive problems are happening on the reviewer’s end. If the reviewers just aren’t getting it, then there’s a big problem. Or if the reviewers are missing a nuance in the argument, that’s important to see too. This summary needn’t be long, but it should paraphrase the thesis and then cover the argumentation (i.e. the argument is X, which is followed in the paper by discussions of A, B and C). If there are logical inconsistencies in the argumentation, this is the place to point that out. If there are argumentative problems, this section can be much longer — upwards of two pages. But if it stretches beyond that, it needs to be broken into subsections to help the author interpret what to do and where specific problems lie.

The middle section of a peer review usually falls into the realm of free association, which is where the reviewer can spend some time discussing the relative merits and shortfalls of the paper, the paper’s linkage to other existing scholarship, and the overall consistency of the paper. Throughout, it’s important to flag which comments need to be addressed by the author and which are not so critical: it’s fine to go on for a paragraph or two about a pet peeve or some flight of fancy, but if it has little bearing on how the author should be rewriting his or her paper, make sure to flag that (e.g. ‘The author might be interested in…’ rather than ‘The author needs to address…’).

In the middle section of the peer review, it’s worth taking the time to discuss each of the sections of the paper (as the author has it broken down), including a brief description of the section along with some evaluative language (does it work or doesn’t it?). If there are parts of the paper that just don’t work or parts of the paper that really do work, this is the place to point them out. Praising things about the paper is just as important as saying critical things: if one reviewer really likes a section and others do not, it’s helpful to see why that’s the case.

Connecting the article to existing literature that hasn’t been discussed by the author also usually falls into this middle section. Again, it’s important to flag what needs to be addressed and what might be addressed. If an author is making vague references to a body of theoretical literature and would really do well to spell out her or his assumptions more directly, point that out; if an author has a huge blind spot in his or her discussion of relevant literature, point that out too. But if there’s some tangentially related literature that you know but the author doesn’t — and it won’t really have any bearing on the article anyway — make sure to mention that citing that work isn’t consequential, but might be interesting to the author.

The last section of peer reviews usually focuses much more on very specific things that need to be addressed by the author: are there specific awkward or vague sentences, missing citations, bibliographic errors, etc.? This isn’t to say that a peer review’s job is to find syntactical or grammatical problems, but if there are writing issues that interfere with the ability of the paper to be read, these problems need to be pointed out to the author.

The final paragraph of the review should succinctly restate your overall assessment of the paper and outline the major things (if any) that need attention on the part of the author.

It may seem paradoxical, but a really good review can actually be a bad review for an author. I’ve received reviews that say things along the lines of ‘This paper is ready to be published; the author shouldn’t change anything.’ (Usually only after already having passed through peer review multiple times, mind you…) But if the other reviewers are highly critical of parts of the paper, such a blanket positive peer review doesn’t help much. Instead, working through the positives of the paper is more helpful for the author, so that she or he can see why you like sections or the paper as much as you do — if you like things that others do not, it’s vital to see why that is. And, if there’s an editor involved, they’re more likely to be swayed by articulate, negative appraisals than a blanket and vague positive one.

Interpreting Peer Reviews

If you’re an author and have peer reviews that look roughly like what I’ve outline here, there should be very little problem in interpreting them. But, more likely, you’ll have peer reviews that don’t strictly (or even closely) follow this format. The best way to tackle reviews is to read through them and identify those things that reviewers think need to be changed (and thereby isolating them from less pertinent critiques). Often, the best way to do this is to see if there is convergence between readers: are people having similar problems with the paper? If they are, that’s fairly easy to see and address. If they’re having wildly different kinds of problems, it’s worth writing down the criticisms and seeing what the problem lies: is it that they really understand your argument differently? Are they coming from very different interpretive traditions? Sometimes you can address these problems; sometimes, it’s just the luck of the draw. And the important thing here is to isolate what you actually need to address and what you don’t. In not addressing some concerns, it’s important to point out why — sometimes in the body of the paper itself (which may be a way to get your argument more precise).

If you’re preparing to resubmit a paper after peer review, editors will often ask that you enumerate the changes that you’ve made to the text — and which changes you haven’t made and explanations as to why. If you have a list of requested changes, this is a fairly easy document to prepare, as you can list which changes you’ve made, how the criticism has been addressed, and where the change appears in the paper. Doing this can be helpful both for the editor and for peer reviewers, who often want to see that you’ve made the changes that they’ve suggested.

Matthew Wolf-Meyer Meta-Academia, Professionalization

The Future of Anthropology (According to the Bureau of Labor Statistics)

ANTHARCH_jobgrowth

My partner forwarded me a short blog post about job prospects in anthropology looking better than they have been (started by The Atlantic), and we collectively began wondering whether there’s really anything to look forward to. And, the short of it is this: there isn’t. The number of Ph.D.s awarded annually has increased dramatically over the last 20 years (from 341 in 1991 to 555 in 2011, up from 472 in 2006 — a 17% growth) and the job projections at the Bureau of Labor Statistics (BLS) show only a 21% projected growth in anthropology-related jobs. On the face of it, this might look okay, but when we look closer at the BLS numbers there are some things to be concerned about — and to prepare strategies for:

First of all is the actual number of jobs. The BLS lists the number of anthropology jobs in 2010 as 6100, with a total of 7400 by 2020. If the job growth is gradual, with ~160 jobs being created each year, that’s still ~300 anthropology Ph.D.s out of work in their field each year for the foreseeable future. And this assumes that the normal number of retirements continue, providing for the regular entry-level jobs that account for ~100 academic jobs annually (which is about the number of jobs posted annually on the AAA job center).

Second, when you look at where the jobs are projected to be, the BLS shows that most of the jobs will be in ‘professional, scientific and technical services’ (not education, but consulting), and that there will be shrinkage in jobs associated with ‘research and development in the humanities and social sciences’ on the order of 2.5%). The jobs associated with education (at all levels of higher ed — adjuncting and tenure track) only account for about 200 new jobs (if I read the table correctly). There’s also significant contraction in the number of jobs associated with the federal government (maybe related to human terrain work drying up?).

Most of the growth that’s projected is in ‘scientific research and development services’ — which, I’m guessing, means a lot of contract archaeology. Which is pretty much what the BLS says: ‘Archeologists should have the best job prospects in cultural resource management (CRM) firms.’ But, they warn, ‘due to the large number of qualified graduates and relatively few positions available, jobseekers may face very strong competition.’ (But this, I’m going to guess, depends on growth in the U.S. housing market, where archaeologists will oversee holes being dug…) They also shore up my summary, saying ‘job opportunities for anthropologists will expand in businesses, consulting firms, and other non-traditional settings,’ but that these jobs are going to be very competitive.

So, if that’s where the jobs will be, what can we do proactively to prepare for those employment possibilities?

We should probably slow down the number of Ph.D.s programs grant each year. If there’s continued ~20% growth in Ph.D.s granted, the growth projected by the BLS will only keep pace with the number of new Ph.D.s. And unless job growth continues at the same rate, it will quickly be surpassed by Ph.D.s granted (although there must be a ceiling to the number granted, I’m just not sure what that might be — especially since there seem to be more and more new Ph.D. programs… — who doesn’t want teaching assistants?). I’m not sure how Ph.D.s might be limited in the future, but it seems that at the level of the AAA some kind of ‘best practices’ could be developed — which would both ensure a diversity of institutions granting Ph.D.s (so it’s not all elite institutions) and rewards actual success (programs that only graduate 5 students each year, but most of them are employed should be favored over programs that graduate 20 each year and only get 5 employed). But that’s a pretty farfetched possibility… More realistically, students might take to heart the success of programs when selecting where they’ll go, which might lead to the same results, i.e. unsuccessful programs will wither while more successful ones grow.

More pragmatically, programs need to start training graduate students across the subfields to be able to do contract and consulting work, which requires a firm commitment to the professionalization of students that extends past preparing a pretty CV.

Mixed field programs might find ways for graduate students across the subfields to develop collaborative projects, both as term projects, but also as dissertations. Historical archaeologists and cultural anthropologists share many concerns, and teaming graduate students up to conduct collaborative work would prepare them for a future of working on research teams in consulting situations. And cultural anthropologists might think about ways to participate in and contribute to biological fieldwork, with both living and dead populations — which training in science and technology studies might prepare students for. Preparing multi-author papers will help prepare students for the possibility of team-written reports.

Programs might usefully invite local non-academically employed anthropologists for brown bag discussions with graduate students about locating and preparing for jobs in non-academic situations. Most academically employed anthropologists are fairly alienated from these realities, so finding mechanisms to make sure that graduate students aren’t is clearly vital. (If you’re a non-academically employed anthropologist and reading this, you should offer to visit your local Ph.D. granting departments for a brown bag lunch.)

With contacts in local non-academic organizations, students might look towards the possibility of developing local projects (maybe alongside their dissertations) with organizations that already employ anthropologists (Rachel Wright has a nice piece about this in a recent Anthropology News). Whenever I teach our graduate methods class, I have my students volunteer at local organizations — a trick I learned from Penny Edgell at the University of Minnesota. It helps students get their fieldwork feet wet, can result in an early publication, and it also helps them get a sense of how their professional career might involve working in a non-academic setting.

I’m sure there’s more to be done, but the low hanging fruit seems to be finding ways to prepare students for collaborative research possibilities (which cultural anthropology tends not to do, and that’s compounded by the meager funding opportunities that favor solo research) and to introduce them to non-academic employment possibilities. There must be more though — any ideas?

Matthew Wolf-Meyer Slumbering Masses

Reading The Slumbering Masses (for non-academics)

You may have come across an advertisement for my book, The Slumbering Masses, in your favorite magazine or heard me on the radio — or maybe you just stumbled across it in your local bookstore or received it as a gift. It’s an odd beast: it was written as an academic book, but my editor and press wanted it to have the potential to ‘cross over’ — to have a more widespread appeal. And I did too, which is why I spend a fair amount of time doing radio interviews, blogging for Psychology Today, and working to publicize the book. As dense as it can be in parts, it also has a lot of material that most readers can read with little furrowing of brows or scratching of head. So, if you have picked up a copy of The Slumbering Masses and are finding it sleep-inducing, try this reading strategy:

Cover of The Slumbering Masses.
Cover of The Slumbering Masses.

Read the Preface, skip the Introduction and Chapters 1-3, and then really start with Chapter 4. The Preface will give you a general sense of the book’s scope, and Chapter 4 will get you right to the heart of the matter with a series of case studies of people and families struggling with disorderly sleep. The chapters that pick up from there in Part 2 are all pretty quick — usually just about 15 pages, and mostly case focused. You can read them in any order. From there you can either move forward into Part 3 or go back to Part 1.

In Part 1, Chapter 3 is probably the most reader friendly — it’s based in a sleep clinic and shows how doctors go about diagnosing sleep problems. Chapter 2 looks at the history of sleep and sleep medicine, but unless you really like historical detail, you can skip the last section of it. And the Introduction and Chapter 1 can generally be ignored. The Introduction is really just for academics and does a lot of theoretical work, which, for most readers, is obtuse; but you might read the last few pages of it for overviews of all the chapters. Chapter 1 does some stage dressing, but, again, it isn’t strictly necessary for the general reader.

Part 3 is more case oriented, and has some pretty compelling stuff — military experiments, extreme sports, and sleepwalking murders. The conclusions of the first two chapters might be skipped, but the heart of them should be relatively straightforward. The Conclusion to the book is a big overview of it and makes an argument about ‘multibiologism’ — or thinking about human variation without recourse to pathology. It works through a number of cases, and is pretty readable.

This is all pretty much to say: skip the Introduction and maybe Chapter 1. They’re really written for academics, and don’t add much for the general reader. You can always go back to them later, and for non-academics, reading the Introduction as the conclusion might make the most sense (however paradoxical that may sound)…

If you have questions about the book or other ideas on how people might approach reading it, feel free to post them in the comments below.