{I realize that I have said much the same in several earlier recent posts. I recently discovered a memo I wrote in 2003, based on examining the flows of candidates through the promotion processes, that also said the same things.}

What inspired this note was reading a review of Daniel Kahneman's new book on decisionmaking, and also Moneyball. [I have left out the need for better mentoring, at all stages in people's careers, and more sophisticated hiring.] One of my colleagues, reading this memo, wrote me: "The topic of personnel evaluation must have a large literature of its own. Having a multi-year probationary period, work output to example, external referees, multiple levels, etc., may make the tenure process the most careful of such evaluations." The argument of Kahneman and others is that perhaps the most careful is not necessarily the most accurate.

http://www.nybooks.com/articles/archives/2011/dec/22/how-dispel-your-illusions/?pagination=false

links to the review

1. Moneyball and Kahneman's "Cognitive Illusion": the job, as would be described by Kahneman or by Dyson (here) is to figure out who will be the strongest faculty in the future. [Some would argue, our job is to see if someone has "earned" tenure, a different perspective...]

That is, might we find some sort of criteria/rule/scoring system that would make it more likely we promote candidates with strong future performance, and much less likely we promote those who will wither, "slams." (I call them "slams" from the biologists' characterization of a drosophila gene: slow as molasses.) Put differently, might we do much better than the usual promotion and tenure procedures, very deliberative process.

Kaheman refers to Paul Meehl. See Meehl's essay, "Why I Do Not Attend Case Conferences" in his book Psychodiagnosis. Would he write, Why I Do Note Serve on Tenure Committees?

2. Deans and departments are subject to what Kahneman calls "endowment effects": namely, overvaluing what they have in front of them. Hence, one expects and usually gets memoranda from the dean and the department that would seem to be hyperbolic considering the evidence.

3. All the research on tenure suggests that What You See Is What You Will Get: namely, if someone has been slow to produce, slams, they won't change and will be disappointments. And if someone has done first rate work, they will continue to do so. If a candidate is in-between, it is likely they won't change either. Again, in each case will you be happy with this person's achievements over the next 30+ years.

4. Too many dossiers are "fishy." Maybe the dean ought to sign something like a Sarbanes-Oxley statement to the effect that the dean has stands behind the dossier. Maybe there needs to be a devil's advocate, in the tenure committee or the school, who examines dossiers and points out their weaknesses (the blatant ones) before the university committee sees them.(I gather that Harvard under Bok had a faculty member that took such a role.