How to evaluate the publication activity and scientific performance? Is it correct to compare the citation counts of different scientists, laboratories or scientific organizations with each other? Why are Russian publications quoted on average worse than foreign ones? These and other questions are answered by the expert and scientometric specialist of Web of Science, Pavel Kasyanov.

Web of Science is one of the world's largest search platforms that indexes various types of scientific publications (articles in journals, conference proceedings, chapters in monographs, and others). Based on the data on citation of materials included in the database, various scientometric indicators are calculated which can help both scientists and managers in everyday practice. However, for different tasks (choosing a journal to publish scientific results or comparing the scientific performance of two organizations), one needs to use different tools and indicators. Pavel Kasyanov, an expert and specialist on scientometry of Web of Science, told about the possibilities of modern scientometrics to scientists and specialists of the Krasnoyarsk Science Center SB RAS.

Number of publications: indicator of scientific performance

The simplest indicator which can be used to analyze scientific performance is the number of publications. However, there are some difficulties. On the one hand, the quality of scientific journals and sources of scientific information may vary. In some cases, the manuscript review system includes up to five anonymous experts, while other publishing houses can publish anything for a small fee. This means that the database should not include all publications, but only those subjected to the preliminary examination. According to Pavel Kasyanov, Web of Science is not only famous, but also boasts the most stringent procedure in the world for the selection of indexed journals and other sources of scientific information. “If you published your results in a journal indexed by our database, this, at least, speaks of the quality of the study,” emphasizes the expert.

However, this gross indicator does not work well when it is necessary to compare authors or organizations working in fundamentally different areas. For example, geneticists or material scientists can conduct several simultaneous experiments and publish several scientific papers within a year. However, historians or archaeologists should not expect articles to be published every three months. If we compare the publication activity of conventional molecular biologists and philologists, the former is most likely to have a much higher indicator. But based on this, it is impossible to draw a conclusion about different productivity of researchers.

Total citation rate: An Indicator of Scientific Influence

The total number of citations of the works of a particular researcher  is one of the most common scientometric indicators. The expression “citation index” is often, and quite incorrectly, used instead of this term. There is no indicator like this one in the database and scientometrics. It is correct to speak about the total citation index. It is assumed that the more often a particular article is cited, the greater is its influence on the development of science. Accordingly, the greater contribution was made by the authors of the article to the development of this direction. However, this parameter is often criticized as being dependent on the field of knowledge.  For example, among physicians, the citation rate is higher than among physicists, and among physicists, in turn, it is higher than among mathematicians. “Unfortunately, the total citation rate is not a good indicator for making decisions,” notes Pavel Kasyanov, “With this indicator, scientists and organizations working in different areas cannot be compared.”

When calculating the total citation rate, the problem often arises in estimating the number of self-citations - cases where the author refers to his own works. According to the expert, a conditionally acceptable portion of the author’s references to his works is 20-30%. If a scientist continues to work on the same subject, then a reasonable number of references to his previous articles is even a necessity. However, if the proportion of self-citations in the total citation rate of an author is 70 percent or more, then with high probability, we are talking about abuse.

Of course, in this case, the nuances are important. It is okay if a young scientist has published only a few articles and most of his citations are references to his own works. Another situation is when an author comes into view with dozens of articles and hundreds of citations, most of which are cross-references. In this case, the scientist can harm his academic reputation. “The fact is that many scientific journals analyze publication profiles of scientists who submit manuscripts for publication. A high rate of self-citations may result in the manuscript not even reaching the reviewers; it will be turned down by a scientific editor, ” confirms the expert.