Jeroen Smeets

A short note on impact factors

The quality of a journal is frequently described using ISI's impact factor. Reviewers regard the impact factor of the journal a paper is published as a measure for the quality of the publications themselves. This factor measures how frequently a paper in that journal is cited in the first two years after its appearance. This means that an article that is cited twice in the two years after its appearance (and never afterwards) contributes as much to the impact factor of a journal as a paper that is also cited 50 times in later years.

In some (sub)fields of science, many researchers are working on the same questions, and use each others results at very short latency. Communication is fast. These fields are characterised by races: who is the first to discover a certain mechanism. In other areas, the research is slower. Experiments are not performed in parallel, but in response to each other. It is not uncommon that it takes (without major revisions) more than a year between submission and publication. If major revisions are required, two years is possible. So, if somebody reads an inspiring paper, performs a clever experiment, submits the paper, and gets a long review process, the citation to the inspiring paper does not contribute to the impact factor of the journal.

A small example based on my own publications illustrates this point. Three of the papers that I published have been cited enough (more than 40 times until 2005, and published at least eight years before) to provide a good idea of the latency distribution of citations. One was published in 1989 in the physics journal Surface Science. The other two are in journals that are partly specialised in my present field of interest: visuo-motor co-ordination (Experimental Brain Research and Journal of Experimental Psychology: Human Perception and Performance). Thin curves give the number of citations to the average paper in those journals. As I selected papers that are cited well, it is not surprising that they are cited more frequently than the average paper in those journals. More surprising is that the surplus in citations in the visuo-motor papers occurred after the second year, whereas for the physics paper, they occurred in the first two years. According to ISI, the paper in Surface Science had more impact than the other papers. In my humble opinion, that is not the case. Another nice example: the very impactfull initial paper on mirror neurones did not improve the impact factor of Experimental Brain Research, as it was only cited three times in the first two years after publication.

If you are interested in the citations to your papers (and to the average paper in the journal you publish in), you can go to the web-site of Web of Science (if you are at an institution that has a subscription). In the "Web of Science", you can find your own papers, and find all citations to each paper. The "Journal Citation Reports" tell you how your favourite journals are doing. Another source of citation information is Google Scholar

Amsterdam, May 2006, Jeroen Smeets