The ISI Database: The Journal Selection Process

ISI's basic mission as a database publishing company is to provide comprehensive coverage of the world's most important and influential research conducted throughout the world. Today the ISI database covers over 16,000 international journals, books and proceedings in the sciences, social sciences and arts and humanities.

An important part of this data is the 8,000 international journals that ISI covers on an annual basis. ISI indexes complete bibliographic data for every item covered, including English-language author abstracts, author and publisher addresses, as well as the cited references of every journal.

ISI is committed to providing comprehensive coverage of the world's most important and influential journals for its subscribers' current awareness and retrospective information retrieval needs. But comprehensive does not necessarily mean all-inclusive.1

Why Be Selective?

It would appear that, in order to be comprehensive, an index to the journal literature of science might be expected to cover all the scientific journals published. This approach would be not only impractical economically, but, as analyses of the scientific literature have shown, unnecessary. It has been demonstrated that a relatively small number of journals publish the bulk of significant scientific results. This principle is often referred to as Bradford's Law.2

In the mid-1930's S. C. Bradford realized that the core literature for any given scientific discipline was composed of fewer than 1,000 journals. Of this 1,000 journals, there are relatively few with a very strong relevance to the given topic, whereas there are many with a weaker relevance to it. Those with a weak relevance to the given discipline or topic, however, typically have a strong relevance to another discipline. Thus, the core scientific literature can form itself around various topics, with individual journals becoming more or less relevant depending on the topic. Bradford understood that an essential core of journals forms the literature basis for all disciplines, and that, therefore, most of the important papers are published in relatively few journals.3

Recent citation analyses have shown that as few as 150 journals account for half of what is cited and one quarter of what is published. It has also been shown that a core of approximately 2,000 journals now account for about 85% of published articles and 95% of cited articles.4 But this core is not static. Its basic composition changes constantly. The ISI editorial team's mission is to identify and evaluate promising new journals that will be useful to ISI subscribers, and to delete journals that have become less useful.

The Evaluation Process

Journal evaluation and selection is conducted on an ongoing basis at ISI with journals added to and deleted from the database as frequently as every two weeks. ISI's editorial staff reviews nearly 2,000 new journal titles annually, but only 10-12% of the journals evaluated are selected. Moreover, existing journal coverage in ISI products is also constantly under review. Journals now covered are monitored to ensure that they are maintaining high standards and a clear relevance to the ISI products in which they are covered.

Each journal goes through an extensive evaluation process before being selected or rejected. The ISI editors performing journal evaluations have educational backgrounds relevant to their areas of responsibility as well as experience and education in information science. Their knowledge of the literature of their field is extended by consultation with established networks of advisors who participate in the evaluation process when needed.

Many factors are taken into account when evaluating journals for coverage, ranging from the qualitative to the quantitative. The journal's basic publishing standards, its editorial content, internationality of authorship, and the citation data associated with it are all considered. No one factor is considered in isolation, but by combining and interrelating the data, the editor is able to determine the journal's overall strengths and weaknesses.

Basic Journal Standards

Timeliness of publication is one of the most basic criteria in the evaluation process, and it is of primary importance. A journal must be publishing according to its stated frequency to be considered for initial inclusion in the ISI database. The ability to publish on time implies a healthy backlog of manuscripts essential for ongoing viability. It is not acceptable for a journal to appear chronically late, weeks or months after its cover date.5 To measure timeliness adequately, a coverage decision can never be based on just one issue; generally, the editor needs to see at least three issues.

ISI also notes whether or not the journal follows international editorial conventions, which optimize retrievability of source articles. These conventions include informative journal titles, fully descriptive article titles and abstracts, complete bibliographic information for all cited references, and full address information for every author.

English language article titles, abstracts, and keywords are essential. English language cited references are also recommended. Although, important scientific information is published in all languages, authors must provide English translations of article titles, author keywords, and abstracts if they hope to reach the widest possible audience. Likewise, as a purely practical matter, it would not be feasible for ISI to take on the task of translating this material.

Application of the peer review process is another indication of the journal's standards and indicates overall quality of the research presented and the completeness of cited references.6

Editorial Content

The true core of the scientific literature is embodied in a relatively small number of journals. However, scientific research continues to give rise to specialized fields of studies, and new journals emerge as published research on a new topic achieves critical mass. The ISI editor determines if the content of a new journal will enrich the database or if the topic is already adequately covered.

The enormous amount of data at their fingertips, and their daily observation of virtually every new science journal published, position the ISI editorial team to spot emerging topics, and "hot fields" in the literature.

Internationality

Geographic representation of a journal is another consideration. To meet the needs of its international subscriber base, ISI seeks to cover journals with international diversity among authors of both source articles and cited articles.

To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI seeks to cover the best regional journals as well. However, rather than compare a regional journal with all journals in its particularly category, the ISI editor considers it in terms of journals from the same geographic area. High journal publishing standards, especially timeliness, and English language bibliographic elements remain essential.

Citation Analysis

The ISI evaluation process is unique in that our editors have a wealth of citation data at their disposal. The importance of correctly interpreting and understanding these data when evaluating journals cannot be emphasized too strongly. Because the number of authors and journals varies greatly among disciplines, discipline-specific citation levels and rates also vary greatly. Smaller fields like botany or mathematics do not generate as many articles or citations as do larger fields such as biotechnology or genetics. Likewise, in some areas, particularly in the arts and humanities, it may take a relatively long time, even several years for an article to attract a meaningful number of citations, whereas in other areas, such 7 as the life sciences, it is not unusual for citations to peak after only a few years. These facts must be taken into consideration if citation data are to be used correctly.

Several types of citation data are used. For established journals, these include overall citation rate, impact factor, and immediacy index. For brand new journals, the editors examine the publishing record of the journal's authors and editorial board members, noting where their articles have been published and if their work has been cited. Also, because ISI captures all cited references from each of the 8,000 journals covered, citation information is available on covered journals as well as those not covered but that have been cited by any of the 8,000 core journals.

Electronic Journals

As stated earlier, ISI's basic mission is to provide access to the world's most important and influential journals. This commitment extends to the evaluation and inclusion of electronic journals.

Although the electronic medium calls for some revisions in the evaluation process, ISI still looks for all the marks of quality found in traditional journals. Editorial content, the caliber of the editorial board and authors, grant funding, peer-review, and internationality are as important as ever.

Timeliness must be measured somewhat differently for electronic formats. Electronic research journals can take two basic forms -- they can be issued in the traditional publishing format as "issues" containing a collection of articles or they can publish one article at a time. The latter format is an appealing feature of electronic journals making it possible to disseminate information even faster. At this early stage in the history of electronic journal publishing, fixed standards for timeliness are still being developed. A good method of determining the "health" of an electronic journal is to observe how regularly articles are posted. Of course, the number of articles posted will depend on the discipline. In general, though, we would not expect any six-month period to pass without some activity.

Print journals often develop electronic counterparts. The electronic version may provide supplementary editorial material, and frequently will get information from the publisher to the researcher more rapidly. If the electronic format provides these kinds of positive changes, then we would consider covering it rather than the print version.

In September of 1994, ISI accepted its first electronic journal, The Online Journal of Knowledge Synthesis for Nursing.Since then, we have added 16 more. Our editors have been mandated to identify all relevant electronic journals and evaluate them for possible coverage. We expect our coverage of electronic journals to grow steadily.

How to Recommend Journals

ISI welcomes suggestions and recommendations for coverage. We regularly conduct customer surveys and market research. If you would like to recommend a journal for evaluation, please contact the Editorial Development Department by FAX at 215-386-6362 or send e-mail to journals@isinet.com Please provide the full name of the journal, its ISSN, the name and address of the editor, and the journal's publisher. If possible, send the most current issue of the journal and then the next two or three issues as soon as they are published. Enclose a brief statement explaining the unique features of this journal and how it is distinguished from other journals in its field. Send sample journal issues to the Publication Processing Department, ISI, 3501 Market Street, Philadelphia, PA 19104 USA.



This essay was prepared by: James Testa, Senior Manager, Editorial Department, Publisher Relations, Institute for Scientific Information.

1. Garfield, E., How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations. Current Contents, May 28, 1990.

2. Garfield, E., Citation Indexing (New York: John Wiley & Sons, 1979)

3. Ibid.

4.Garfield, E., The Significant Scientific Literature Appears in a Small Core of Journals. The Scientist V10(17), Sept. 2, 1996.

5. Garfield, E., How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations. Current Contents, May 28, 1990.

6. Ibid.

7. Ibid.

For more information, please contact webmaster@isinet.com. (Please include your address and phone number when writing, as it helps us respond.)

Copyright ©1998 Institute for Scientific Information.