[Humanist] 23.297 on peer-review
Humanist Discussion Group
willard.mccarty at mccarty.org.uk
Tue Sep 15 07:27:03 CEST 2009
Humanist Discussion Group, Vol. 23, No. 297.
Centre for Computing in the Humanities, King's College London
Submit to: humanist at lists.digitalhumanities.org
Date: Mon, 14 Sep 2009 14:15:38 +0100
From: Bryn Mawr Classical Review <bmcreview at brynmawr.edu>
Subject: from the BMCR: O'Donnell on Lamont, How Professors Think
[The following book-review will, I suspect, interest many here. Although the conclusions at which the author of the book arrives may not be terribly surprising to anyone doing academic work outside the older disciplines, it is certainly better to have a careful study to hand than merely one's own hunches. And it is very good indeed to have a review of this study by such a distinguished member of Humanist.
Allow me, however, to question as well as to recommend and to praise. How many of us here have given up applying for grants because of the prejudices built into the reviewing system?
Bryn Mawr Classical Review 2009.09.42
Michèle Lamont, How Professors Think: Inside the Curious World of Academic Judgment. Cambridge, Mass.: Harvard University Press, 2009. Pp. 330. ISBN 9780674032668. $27.95.
Reviewed by James J. O'Donnell, Georgetown University (provost at georgetown.edu)
Word count: 732 words
As a reasonable and thoughtful academic of some experience, I judge that I am a skilled reviewer of other people's work, well able to take into account my own biases and render a fair and objective judgment for the benefit of the profession whenever I am called upon to do so.
I am wrong.
Those of you who entertain similar opinions of yourselves are equally wrong. Not, I am happy to say, disastrously wrong, but certainly wrong enough to be embarrassing and a source of concern for us all. Michèle Lamont is a sociologist and Faculty Advisor on Development and Diversity at Harvard, melding in this book her professional skill and her administrative concern. The title and subtitle of the book rather overstate what the book delivers, which is a shame inasmuch as it might leave a reader disappointed with a study that does not in fact disappoint.
What Lamont has done is study in detail the workings of peer review panels (the limitation is important, inasmuch as an entirely separate study would be needed of the ways journal and university press peer reviewing work, when the reviewers work alone and not in groups) for a group of important funding agencies in the humanities and social sciences: the American Council of Learned Societies, the Society of Fellows at "a top research university" (Lamont is at Harvard, recall), the Social Science Research Council, the Woodrow Wilson National Fellowship Foundation, and an unnamed foundation in the social sciences. She did detailed interviews with program officers, panelists, and program chairs.
The findings will be variously striking to various readers. For example, she teases out well the cultures of some representative disciplines: English, History, Anthropology, and Political Science. The historians come across phlegmatic, the literature scholars edgy and uncertain, and the anthropologists curiously isolated: but no summary in one or two adjectives is a fair representation of her discussion.
She is perhaps best, though, on the way reviewers bring to the table a host of judgments, expectations, and anecdotes that are not to be found in the folders they read, but that strongly influence their judgment. A sociologist admits, for example, that a good proposal on the table gets a better judgment because of the institution from which it comes. That institution is strong in that field, so this is more likely to be good work than if it came from (the sociologist's patronizing words) "some tiny little hole-in-the-wall college" (227). One grid that many bring with them has much to be said for it: "significance" and "originality" stand high on the list of things that reviewers seek in proposals, and I think I can live with that. But Lamont identifies as well a curious overlay of ethical inquiry and judgment: "humility," "determination", and "authenticity" are judged well, while "cultural ease" and "elegance" score points as well. How many of these categories are screens for finding "people like us" is a difficult question with no ultimate resolution.
For there is, on the one hand, some relationship between the recognized quality of the institution and the work that is likely to be done there, given the effort and resources that go into recruitment and selection of students and faculty; on the other hand, a bias that intrudes between the particular qualifications of the scholar and the merit of the proposal runs a real risk of unfairness.
The book that has resulted from these inquiries is not lively reading, but worth serious attention. At the same time, it will not be widely read or easily digested by many. Lamont's best prescription for improving the quality of the work that is done by many conscientious people is to raise the issue and focus the attention of scholars on their own practices and expectations. "Know thyself" remains good advice. An administrator reading the volume thinks about ways in which the unacknowledged forces that influence judgment can be made part of broader discussion, the better to improve the quality of judgment within institutions and beyond their walls. The topic deserves to remain open and remain the subject of continuing discussion.
Table of Contents:
1. Opening the black box of peer review
2. How panels work
3. On disciplinary cultures
4. Pragmatic fairness: customary rules of deliberation
5. Recognizing various kinds of excellence
6. Considering interdisciplinarity and diversity
7. Implications in the United States and abroad.
BMCR, Bryn Mawr College, 101 N. Merion Ave., Bryn Mawr, PA 19010
More information about the Humanist