Metrics for the Evaluation of Research Output

Recently, there has been some discussion on this blog of the important issue of the evaluation of the research output of individual scientists, university departments and research centres. Some obvious and simple statistics are number of papers, number of citations, and other pure bean-counting metrics. I have already argued elsewhere why I'd like to see papers weighed in some way by the quality of the outlet in which they appear, so I am not going to reiterate that point here. 

What I'd like to point out, with apologies to those of you who know this already, is that there are other, more refined measures of impact that are widely used in the world of academia throughout the world. Moreover, some of these metrics can be calculated at the press of a button using tools like Anne-Wil Harzing's Publish or Perish. There is really no excuse for not using such a resource in a qualitative/quantitative evaluation!

A widely-used metric is the so-called h-index. The h-index was proposed by J.E. Hirsch in the paper An index to quantify an individual's scientific research output, arXiv:physics/0508025. It is defined as follows:

A scientist has index h if h of his/her N papers have at least h citations each, and the other (N-h) papers have fewer than h citations each.

By way of example, from 1 August 2007 there will be at least 10 full-time members of staff at the School of Computer Science at Reykjavík University whose h-index is 5 or more, and at least 5 of those will have h-index 10 or larger, meaning roughly that they have written 10 or more papers that have 10 or more citations. My former BRICS colleague Jens Palsberg (who is now a professor of computer science at UCLA) maintains a partial list of computer science researchers who each has h-index 40 or higher. 

Another metric of research impact and activity that is becoming popular is the g-index. The g-index was proposed by Leo Egghe in his paper Theory and practice of the g-index, Scientometrics, Vol. 69, No 1 (2006), pp. 131-152. It is defined as follows:

[Given a set of articles] ranked in decreasing order of the number of citations that they received, the g-index is the (unique) largest number such that the top g articles received (together) at least g² citations.

The g-index aims to improve on the h-index by giving more weight to highly-cited articles.

Continuing with my example, from 1 August 2007 at least 8 full-time members of staff at the School of Computer Science at Reykjavík University will have g-index 13 or higher. 

The Nobel Prize winner Harry Kroto, whom I mentioned repeatedly in other posts, has h-index 31 and g-index 80 according to a quick-and-dirty search. 

Readers who are keen on qualitative analysis of research will find that  Publish or Perish lets them easily compute other metrics such as the Contemporary h-index and the Individual h-index. I encourage them to play with that tool, but be warned! It is addictive!