Towards Measuring Journal Impact—Properly

Melissa Vergara Fernández
Publication Number: 
Publication Date: 
Wednesday, December 23, 2020

In this paper I evaluate the Journal Impact Factor using a theory of measurement. I argue that JIF does not stand up to close scrutiny. To measure a concept adequately, our theory of measurement

requires correspondence between three steps: the characterisation of the concept, its representation, and the procedures followed to carry out the measurement. Characterisation

involves defining the concept: identifying its boundaries, which fixes the features that belong to it. Representation involves defining a metrical system that appropriately represents the concept. The

procedures are the rules formulated for applying the metrical system to the tokens. These three steps do not line up together neatly for JIF. There are at least two problems. First, the procedures

to measure JIF do not reflect an unequivocal characterisation. Second, the representation strategy of JIF is inappropriate and not justified, given the kind of concept it tries to capture: one without

strict boundaries. The bottom line is not that the JIF ought to be eschewed. Sufficient reasons related to how JIF distorts scientists’ incentives have been provided to this end. But path dependence

is a tricky issue—the longevity of qwerty keyboards demonstrates it. The bottom line is that, given that JIF is unlikely to vanish, we better start giving it some proper scientific basis.

Erasmus University Rotterdam