Running the peer review gauntlet (from GenomicEnterprise.com)

I have learnt a lot about academic publishing in the past year, particularly about publishing journal articles. This is partly because I have been writing up my PhD thesis into journal articles and in part because of my work for PhD2Published as Managing Editor. In todays post I share some of my recent experiences of peer reviewing to provide advice about what not to send to reviewers.

In the past four months I have gained experience of peer reviewing journal articles. I have reviewed 3 in total; two I was asked to do by a known colleague and mentor who also happens to be a journal editor, and one came out of the blue but related well to my research interests (suggesting I am getting known for my research interests –yay!). I have found peer reviewing a really interesting and fruitful experience. Seeing less than perfect academic writing has been a real eye-opener and has given me a new, more confident perspective on my own writing.

Today’s post is based on my experience of peer reviewing one of these journal articles. Of the three I have reviewed, I have awarded two ‘major revisions’ and one I actually rejected. I found it very difficult to reject the article because as an early career researcher sending out my own work for review, I know how nerve-wracking it can be to put your writing out there. I also thought that the paper had real potential but unfortunately the more I read, the more disappointing it became. I was very careful to give constructive feedback and to fully explain my decision so as not to discourage the author or to offend them. As it turned out, it appeared my decision was justified. The other two reviewers also rejected the paper (which shows I know what I’m doing. Double yay! ;)).

If I’m being honest, I was quite surprised that the paper was sent for review in the state it is was in. Perhaps the looming REF meant it was sent under pressure, or perhaps the author just wanted some feedback at an earlier stage? Perhaps there is a genuine issue in academia that not all of us know how to write a strong first draft of a journal article, an issue that PhD2Published at least has tried to remedy (e.g. Inger Mewburn’s series on Writing Journal Articles). Either way, there were some significant problems identified by all three reviewers. And having gained this insight, I wanted to write this post to share some of these significant issues (without revealing the author of course) so that others who are new to publishing journal articles or asking for early feedback know what to avoid. While some of these examples may seem far-fetched, they are based on my experience. There are other things to be aware of, of course, but for me, these are key and are what I look out for when reviewing.

So here are my tips about what not to send if you want to be accepted:

1. If your paper is based on empirical findings don’t send a paper that doesn’t explain what methods were conducted, who the sample was or how the data was analysed. The readers need to know that your findings and conclusions are based on a well-designed research project.
2. Don’t make grand claims that cannot be substantiated with evidence from data or literature. Your points and claims need to be believable and convince the readers that your arguments are well-grounded throughout.
3. Don’t write a conclusion that does not explicitly state the take-away point from your article. Likewise don’t send a paper that does not explicitly state its aim and purpose in the introduction. Tell the reader what you are going to tell them, tell them and then tell them what you told them.
4. Avoid making assumptions about the prior knowledge of the readers, particularly if you discuss an event or situation that is only well known in your own context. Make use of foot or endnotes for explaining things that aren’t pertinent to the text but need explanation.
5. Avoid over-using jargon and complex academic language that you haven’t explained or referenced, especially if this affects the clarity of the paper.

And here’s a bonus tip from a review of one of my own papers:

Don’t send a paper that hasn’t been edited thoroughly before submission. Irritating grammar errors annoy reviewers and may detract from the quality of your paper.

Kate is a Lecturer in Sociology in the Department of Social and Policy Sciences at  the University of Bath. She completed her PhD in 2007 and details about her publishing, research and teaching can be found here. In this post she shares her top tips for getting journal articles published post PhD.

1. Try to get a paper published on methods. This is good for contributing to your discipline in terms of how you ‘do’ the empirical part, and is also good for developing your teaching profile. I’ve found it useful to teach methods courses as you are involved in the ‘core’ teaching and having a few papers on methods is evidence of your interest in it.

2. Publish in a journal that you know your contemporaries will read (even if not high impact). They will be the ones that come to you for inclusion in research bids, book chapters, general advice etc,

3. Publish in a journal that is important to your discipline so it is clear that you are making a contribution to wider disciplinary debates
(easier said than done!),

4. Edit a book if you can – it is so interesting to see different styles of writing,

5. Get into the habit of reviewing journal papers – so you can see some of the stuff that gets sent in (and therefore breaking the illusion of
perfection). It is, as my supervisor once said, also a free education!

Image by James Yang http://www.jamesyang.com

PhD2Published has several informative posts about writing journal articles, and more recently has featured a post outlining a potentially revolutionary collaborative peer review process for this kind of publishing. Todays post offers an alternative perspective; that of the journal article peer reviewer. Doing peer reviews provides important experience for those writing their own papers and may help writers consider what they should include based on what peer reviewers are looking for.

At some point in your scholarly career, you likely will get asked to review an article for a journal. In this post, I explain how I usually go about doing a peer review. I imagine that each scholar has their own way of doing this, but it might be helpful to talk openly about this task, which we generally complete in isolation.

Step One:  Accept the invitation to peer review. The first step in reviewing a journal article is to accept the invitation. When deciding whether or not to accept, take into consideration three things: 1) Do you have time to do the review by the deadline? 2) Is the article within your area of expertise? 3) Are you sure you will complete the review by the deadline? Once you accept the invitation, set aside some time in your schedule to read the article and write the review.

Step Two: Read the article. I usually read the article with a pen in hand so that I can write my thoughts in the margins as I read. As I read, I underline parts of the article that seem important, write down any questions I have, and correct any mistakes I notice.

Step Three: Write a brief summary of the article and its contribution. When I am doing a peer review, I sometimes do it all in one sitting – which will take me about two hours – or I read it one day and write it the next. Often, I prefer to do the latter to give myself some time to think about the article and to process my thoughts. When writing a draft of the review, the first thing I do is summarize the article as best I can in three to four sentences. If I think favorably of the article and believe it should be published, I often will write a longer summary, and highlight the strengths of the article. Remember that even if you don’t have any (or very many) criticisms, you still need to write a review. Your critique and accolades may help convince the editor of the importance of the article. As you write up this summary, take into consideration the suitability of the article for the journal. If you are reviewing for the top journal in your field, for example, an article simply being factually correct and having a sound analysis is not enough for it to be published in that journal. Instead, it would need to change the way we think about some aspect of your field.

Step Four: Write out your major criticisms of the article. When doing a peer review, I usually begin with the larger issues and end with minutiae. Here are some major areas of criticism to consider:

–          Is the article well-organized?

–          Does the article contain all of the components you would expect (Introduction, Methods, Theory, Analysis, etc)?

–          Are the sections well-developed?

–          Does the author do a good job of synthesizing the literature?

–          Does the author answer the questions he/she sets out to answer?

–          Is the methodology clearly explained?

–          Does the theory connect to the data?

–          Is the article well-written and easy to understand?

–          Are you convinced by the author’s results? Why or why not?

Step Five: Write out any minor criticisms of the article.  Once you have laid out the pros and cons of the article, it is perfectly acceptable (and often welcome) for you to point out that the table on page 3 is mislabeled, that the author wrote “compliment” instead of “complement” on page 7, or other minutiae. Correcting those minor errors will make the author’s paper look more professional if it goes out for another peer review, and certainly will have to be corrected before being accepted for publication.

Step Six: Review. Go over your review and make sure that it makes sense and that you are communicating your critiques and suggestions in as helpful a way as possible.

Finally, I will say that, when writing a review, be mindful that you are critiquing the article in question – not the author. Thus, make sure your critiques are constructive. For example, it is not appropriate to write: “The author clearly has not read any Foucault.” Instead, say: “The analysis of Foucault is not as developed as I would expect to see in an academic journal article.” Also, be careful not to write: “The author is a poor writer.” Instead, you can say: “This article would benefit from a close editing. I found it difficult to follow the author’s argument due to the many stylistic and grammatical errors.” Although you are an anonymous reviewer, the Editor knows who you are, and it never looks good when you make personal attacks on others. So, in addition to being nice, it is in your best interest.

Tanya Golash-Boza is  Associate Professor of Sociology and American Studies at the University of Kansas. She Tweets as @tanyagolashboza and has her own website.

The editors, taken by Imke van Heerden on 3 February 2012, University of York. From left: James Fraser, Ben Madden (postcolonial hanger-on), Isabelle Hesse, Anna Bocking-Welch, and me (Sarah Pett).

Four PhD students at the University of York are currently piloting an innovative peer review process for developing postgraduate conference papers into an edited collection. In the first of a series of posts, Sarah Pett (whose has her own blog and Tweets as @essiepett) discusses the project’s ethos, as well as the practicalities of turning an ambitious idea into a successful reality.

Prompted by a shared experience of the difficulties inherent in positioning our individual research projects in relation to postcolonial studies, in 2010 Anna Bocking-Welch, Isabelle Hesse, James Fraser and myself established Postcolonial Perspectives, an interdisciplinary reading group for postgraduates at the University of York. From the beginning, the group focused on unconventional approaches to the postcolonial, with an emphasis on contexts that troubled its paradigms. It soon became apparent that we were not alone in our frustrations – discussions with postgraduates from across the UK revealed that we were grappling with an issue of increasing relevance and concern to PGRs working in a range of disciplines, periods, and contexts. Thus the Living Beyond Theory: Interdisciplinary Perspectives on the Postcolonial postgraduate conference was born. The event was a resounding success, and highlighted an emerging body of research into contexts that trouble the established paradigm of postcolonial studies. But how, we wondered, to sustain the dialogue that shaped the event, and disseminate the wealth of ideas it generated? Given the different stages of our projects, it will be several years until our individual monographs appear, while their disciplinary and contextual diversity means that our shared concerns would inevitably be diluted. With the help of Dr Jason Edwards at the University of York, as well as funding from the Postcolonial Studies Association and York’s Centre for Modern Studies, we decided to keep the momentum generated during the conference going by developing a selection of the papers into an edited collection.

Why open peer review?

“Blind peer review is dead. It just doesn’t know it yet.”

Inside Higher Ed’s Scott Jaschik begins his report on the Future of Peer Review session at the 2012 MLA convention with this provocative statement from CUNY’s Dr Aaron J Barlow. As Barlow points out in his paper, “The Bearable Light of Openness: Renovating Obsolete Peer-Review Bottlenecks”, the rise of digital publishing has brought traditional peer review procedures into question. For Barlow, its impression of “quality control” is no longer a fair exchange for the publication delays and complex, occasionally unethical, personal and institutional agendas blind peer review entails – a foible I know all too well. Full of the bravado of youth, I thought I’d try my hand at academic publishing soon after completing my BA. Without any knowledge of established protocol, I made the mistake of submitting my paper to two journals simultaneously. Not a wise move, but it did open my eyes at an early stage to the inconsistencies of the field. One journal returned my article within the fortnight, accompanied by a largely positive review that recommended only a handful of minor revisions. Several months later, I received a two page review from the other journal, which included an ultimatum: significantly shift the focus of the paper, or it won’t be published. The recommended shift seemed to reflect the reviewer’s research interests, rather than my own, which was an unpleasant and demoralizing experience for a young researcher. More importantly, however, it was disabling, leaving me with no platform from which to respond to the reviewer’s diktat.

Clearly, this is not something we wished to replicate in the preparation of the edited collection. What’s more, as postgraduate researchers, we have been aware from the outset that the collection has to be tip-top to stand a chance with a “proper” academic press. And finally, with an editorial committee made up of four researchers in the final stages of their PhDs, we simply couldn’t afford to commit to providing each participant with the level of feedback and writing support we hoped to offer. To optimize what we could do in the time available, we opted to select and improve articles via an intensive, collaborative process based around realtime participation. To do so, we designed a series of open peer review workshops that allowed our authors to participate in providing and responding to feedback over the course of several months: a model that closely resembles that employed by Kairos, whose editor Cheryl Ball appeared alongside Barlow at the MLA. Kairos—a journal of rhetoric, technology, and pedagogy—employs a three tier review process. In tiers one and two, submissions are evaluated by individual editors before being forwarded for discussion to the editorial board as a whole. In tier three, a staff member is assigned to mentor the author in implementing revisions for up to three months. The Postcolonial Perspectives publication workshop series sought to emulate and even build on this process. In stage one, the editorial committee selected papers from the conference to invite as contributors; stage two involved refining the ethos of the project and requesting that contributors develop their papers with this in mind. Stage three is when the realtime workshops came into play, allowing contributors and reviewers to meet and discuss feedback over the course of a day.

The first workshop, which took place at York on 16 January 2012, was extremely successful—one academic staff member said he was keen to adopt our model in his own work—and demonstrates how a collaborative open peer review model can be implemented at a grassroots level to support the career development of PGRs and ECRs. The second workshop is scheduled to take place in early May, during which participants will go over the final revisions and collaborate in refining the book proposal and editorial introduction as a group. We are also looking into using an open source manuscript management and publishing system such as the Public Knowledge Project’s Open Journal Systems as a means of continuing the workshops’ collaborative format remotely. While the project’s aim—publication with an academic press—is ambitious, the workshop format means that, at worst, our contributors can walk away with a carefully revised paper for submission to an international peer reviewed journal, a committed mentor, and a handful of supportive peers with shared research interests and career goals.

Workshop One: From left: Dr Jason Edwards (York), Professor David Attwell (York), James Fraser (York), Anna Bocking-Welch (York), Rebecca Jones (Birmingham), and Katherine Ebury (York).

The radical in me would love the project to culminate in a high profile open access publication, accompanied by a creative and thought-provoking social media campaign to raise awareness about postcolonial studies, its contributions and its limitations. For the time being, we’ll continue to play it safe, but it won’t stop me thinking about the possibilities for reform in academic publishing, and the instrumental role PGRs have to play in changing the game. Hopefully before too long there will be a copy of Beyond the Postcolonial Paradigm: Interdisciplinary Perspectives on the Parapostcolonial on a library shelf—or Kindle—near you…