Should I accept that review request?
A constant question that comes up for academics is: Should I accept that review request? When I look back on how I managed my time as a junior faculty member, this was one area where I wasted time by saying yes way too often. Pre-tenure, I nervously said yes to all requests. I was afraid to say no and didn’t understand when it was prudent to say no. Over time, I’ve developed some rules of thumb for when to say yes that I’d like to share.
First (and this is obvious but it needs to be said), I only accept review requests where the content is in my area of expertise. Sometimes part of the content is in my area and part is not, and I will note that in my review. (“I’m not an expert on machine learning, and will comment mainly on the HCI parts of this paper.”)
Second, if I have submitted to a venue (conference, journal), then I owe them reviews back. I have generated a need for reviews, and I need to give back.
If I’m overwhelmingly busy, I sometimes say no even if I feel I owe them. But I’ll explain the details, and encourage the editor to ask me again another time. No one wants to hear about your full to-do list, but it is helpful to say something short like “I have a major grant deadline coming up,” and people will understand. It’s also polite to suggest other possible reviewers.
If a venue is low quality and is somewhere I would never consider submitting, I say no. Pre-tenure when I nervously accepted every request, I spent way too much time reviewing material that I should not have wasted my time on. If the venue has different social norms and standards than what you are accustomed to, then you also may be too harsh a critic of the work, because you don’t know their standards.
If a venue is not one I’ve ever submitted to or is from another discipline but is high quality, then I ask myself some simple questions: Does this look interesting? Am I the best person to review it? Often when someone from a different field sends me a review request, it’s because they really did their homework—they realize that the work is interdisciplinary and put in some effort to identify an interdisciplinary reviewer. All of us who do interdisciplinary work have received reviews where this was not done and reviewers missed the point. I respect the editor who went to the extra effort to find someone with that missing expertise, and I try to say yes if I can to those requests.
Finally, the older you get, the more likely it is that you know the person asking you to review. And in that case, you need to consider the pros and cons of the request in the context of your relationship to that person. However, don’t let a friend who is an editor abuse your friendship. It’s OK to politely decline if they ask you too many times too close together.
Do you have different rules for what to agree to review? Leave me a comment!
The Next Bison: Social Computing and Culture 
I often use another piece of advice you’ve given me, on deciding whether to take on a particular volunteer role: It’s six months later, and you’re in the role, and now they’re going to meet. Do you dread the meeting, or look forward to it? If the former, say no now.
I have a different challenge that I’m sure that you’ve faced as well: When you’re asked to contribute a chapter to an edited volume, or (even more difficult to say no to) asked to write a foreword to a book, how do you make the decision? On the one hand, it’s such an honor to be invited. On the other hand, it’s a big time commitment (bigger than a review, or a letter of review/evaluation). I’ve been invited to write a forward for a book that only intersects with my areas of expertise, from editors whom I know and respect. It’s complicated. Any advice?
The book foreword is especially hard because you have to read the entire manuscript. So I guess in that case I’d think is it: a) I don’t have time to read this, or b) I would love to read this and if I say yes to the foreword I’ll have to make time. And say yes in the case of b. The book chapter? Just no. :). Haven’t written one in ages. (Though I might if I had a point I really wanted to make that’s hard to say in peer reviewed format.)
I use the “imagine you’re about to meet” trick to decide which grad students I want to work with!