Scientific papers in peer-reviewed journals

Most of the new medical literature appears as papers in peer-reviewed journals. To keep up with this part of the information flow, you have to follow steps 1–8, below:

• 1.
  

  Define your problem.

  
  
• 2.
  

  Create a search strategy.

  
  
• 3.
  

  Select the right bibliographic databases.

  
  
• 4.
  

  Search.

  
  
• 5.
  

  Select suitable references from those that have been retrieved.

  
  
• 6.
  

  Assess whether the search was satisfactory.

  
  
• 7.
  

  Redesign the search strategy and/or choose other databases/search tools, where needed.

  
  
• 8.
  

  Repeat steps 2–6, if necessary.

Define your problem

A successful search is based on looking for the key issues, but how do you ensure that you do exactly that? As an illustration, suppose that you wish to be updated on the effects of biologics on rheumatoid arthritis (RA). This is not exactly a well-defined problem in search terms. There are three important key issues: effects, biologics and RA.

Starting with RA, this might very well be as fully defined as it should be, but ask yourself whether a further specification is needed. Is it a particular patient group in terms of age, gender, genetics or similar? It could be that the group concerned is ‘young women with RA during pregnancy’ or another specific group of RA patients.

Effects must also be specified – effects measured in what manner and compared to what? An example of the question that has to be addressed could be: What is the effect of treatment with biologics compared to non-steroidal anti-inflammatory drugs (NSAIDs) treatment or to steroids? Furthermore, is it the effect of a specific type of biologics compared with a specific type of NSAIDs? The other part of the question is which effect am I looking at? Is it DAS28, pain reduction, joint destruction, function or quality of life? There are many types of outcome measures, and usually you will have a fair idea of the important ones for a particular patient group or for a specific treatment.

The last step towards a clear definition of the problem in question is to define biologics. How broad a definition is allowed and which biologics are the most important ones to include. Is it really a comparison between biologics in general with another defined type of treatment, or is it a specified biologic treatment, you have in mind?

The definition of the problem about which you want to find information is the base on which the whole procedure is built, and more experienced practitioners and researchers will have a great advantage here, being able to write out the problem of interest quickly. Often, it will be necessary to break down the problem into sub-questions to create clear search strategies, which will lead to better results. You must also decide if your question asks for an epidemiological approach, where you are looking at effects of the past in a whole population and therefore cannot ask for randomised controlled trials (RCTs), or if you are mainly interested in looking for designed studies in controlled and, if possible, randomised studies. In all this, you also have to think clearly, and make the best use of the material available. Nearly all high-powered RCTs started as pilot studies. Many large epidemiological studies began as more humble studies of smaller groups, which provided the ideas for the full-scale studies. It is important to understand what type of studies you are looking for to get a valid answer to your question .

Create a search strategy

Having delineated a well-defined question, it is possible to create a search strategy based on a search table. Instead of rushing into a search by typing in the first words that come to mind, it is worthwhile working out a search table. If the question is effect of exercise on physical function in juvenile idiopathic arthritis, a search table could look like Fig. 1 .

A possible search table for the question ‘Effect of exercise on physical function in juvenile idiopathic arthritis’. The table may be developed further, especially in the Exercise and in the Physical function column.

Juvenile idiopathic arthritis has several names, and it is wise to check with the MeSH database in PubMed or with keywords in EMBASE (Exerpta medica) (see below) to get further ideas concerning the various names used for the same condition by different authors.

For the juvenile idiopathic arthritis patients exercise will be limited, and it might be carried out under the supervision of a physiotherapist. Keeping this in mind when working out the search table, a list of suggestions covering exercise, again using some known keywords from the medical databases, is given, and more possibilities may be added. For the last term, physical function, a set of known outcome measures are given. There are more of these, and the choices here are scales that specifically are used for children, because this is a child-specific disease. In each column, each representing a key issue, at least one of the given terms has to be found in a reference to include this in the retrieved references. The terms in each column are therefore combined with ‘OR’ when searching. This will give rise to three sets of results, one from each column.

As all of the three main issues in this example (other problems could have more main issues) have to be included in the total search, the results of the three searches, one for each column, have to be combined with ‘AND’, when searched. The end search will be (juvenile idiopathic arthritis OR juvenile rheumatoid arthritis OR juvenile chronic arthritis OR juvenile onset Still’s disease) AND (exercise* OR physical therapy OR jogging OR swimming OR pool therapy OR dancing) AND (physical function OR exercise test OR CHAQ OR JASI OR JAFAS OR joint range of motion).

The above is a simple example. The search can usefully be extended much further, and the given list here is a mixture of ‘free text words’, words appearing anywhere in a reference in the searched database, and defined keywords (MeSH words), which might be specific for the particular database. It is necessary to search both terms that are given keywords in the database, if a keyword covering the term exists, and other words covering the term in question. Although it is important to search the keywords/MeSH words when available, they should in general be searched as both keywords and as free text words. This will be the case if all search words are searched as ‘free text words’. The reason for this is that a keyword may only have been in use for some of the years covered by a database and not always been used to index a relevant reference during these years. In the example, it should be noted that exercise has an * after the word. This causes the search to include any term that starts with ‘exercise’. This is called truncation of or adding a wild card to the search term.

Truncation is also used to account for different types of English language spelling, for instance *edema for edema or oedema (American and British English spellings).

The broader you search in terms of keeping each term less defined, the more ‘noise’ (useless/unwanted references) will come out of the search. As an example, you could search osteoarthritis without limiting your search to a joint such as the knee. You will get several studies on other joints than the knee. However, you will also catch some studies of knee osteoarthritis that did not appear in your narrower search. You will therefore achieve a higher sensitivity (a better coverage of the literature) with the broader search, but the cost is the high number of retrieved references you have to assess for inclusion, where several do not give an answer to your request.

The alternative is a very specific, narrow definition of the problem in question in your designed search strategy. This will give a high specificity (more or less all of the retrieved references will be relevant), but the coverage of the problem will most likely not be optimal. Depending on your information needs, you should aim at a search strategy that will give ‘enough information’ for your purpose. For updating, a more specific search may be preferred, whereas you may need several broader searches for research purposes.

Select the right bibliographic databases

When the question is clearly defined and a search strategy created, it is time to choose the right database(s) or other search tools. For medical literature, there is a good range of bibliographic databases.

Bibliographic databases cover only a certain chosen number of journals, and the selection varies from database to database. This is why you have to consider which databases are the ones to search for your particular subject area. Furthermore, it is important to remember that although databases may seem alike, they are not. Two databases may have the same structure but their keywords and names for the various fields (‘tags’ such as author, address and abstract) are very often different. The way one database searches may also differ from all the others. When designing a search strategy, it is important to look for possible keywords that define your search terms and make sure the meaning of these words really is the same as your understanding of the words. You have to understand that a term used in your local clinical or laboratory setting is not necessarily used for the same notion in a database and that a term used in a database might have a different meaning than the one you normally would expect. Further, the terminology and spelling will vary between countries and languages (e.g., between American English and British English). You will, as mentioned, take this into account by using truncation/wild cards.

In rheumatology there will be a need to search a set of bibliographic databases, depending on the area of interest. For a quick update, you will get by with a search in Medline and/or EMBASE , but if you want to get a complete update of your field, you will probably need to search more than these two databases.

Table 1 shows a selection of bibliographic databases of interest, with a short explanation and a suggestion of what to search where. It is worth knowing that new research areas first will appear as meeting abstracts, and these are mainly found in Web of Science , Biosis Previews and EMBASE . For special areas, a set of smaller databases is also shown in Table 1 .

EMBASE and – to a certain degree – Medline cover some journals written in languages other than English, but if you wish to search articles in other languages you may find that the country speaking the language provides a bibliographic database in which you can search in the given language.

Bibliographic databases come in various wrappings, depending on who delivers them to the user. The delivery firm is called the ‘database host’, and one host can give access to a wide variety of databases. What the host provides is the design of the search page. PubMed is really the host for several databases, apart from Medline. Other examples of hosts are STN, OVID and EBSCO. A database will therefore look different on different hosts, but – despite this – the database behind will be exactly the same, the search system will be the same and you should not search the same database twice by searching the same database via two hosts. For example, it is only necessary to search Medline either via PubMed or via OVID, despite the different appearances of this database in the two hosting systems.

Another problem can occur when searching several databases provided from the same host in one search. This will not give the best and most professional results because the databases behind the common search face are different in structure, and the full benefit of your selection of keywords, publication types, etc. will not be obtained. It is valuable to search each database separately. This will also allow you to download chosen references from your searches for import to reference handling systems (Reference Manager , Endnote , Procite and others), where knowledge of both database and host is demanded for successful import into your own reference databases.

Search

There is a continuous development in the ways search systems search databases.

Artificial intelligence is part of many search systems, but its quality varies. Occasionally you will get great benefits from the artificial intelligence and get a better search, but at other times you will find some search results that seem very far from your intended search. It all depends on the way you approach your database in the form it is made available to you. In Medline searched via PubMed , which uses artificial intelligence, you can search by introducing the whole search string (the end search). You can see how the database has been searched by looking at ‘search details’ (the box at the right-hand side of the screen), which explains why your search gives the references retrieved. Instead of relying on the artificial-intelligence approach, you could choose to search each term alone. When you have carried out the single-term searches, you can then go to ‘Advanced’ where from ‘History’ you combine the terms with AND or OR as appropriate, where each searched term will be represented with #1 or #2, etc. Your further search will look something like this: (#1 OR #3) AND (#4 OR #5).

In the PubMed version of Medline, it is easier to learn to handle the total search string created from your search strategy as soon as you feel confident with the whole search technique and have designed a good search strategy for updating. The advantage in searching one term at a time to start with is, on the other hand, that you are able to understand if your choice of search terms is satisfactory. If you have only two hits on one of your search terms, and you know the area is well described, you know you have to find another word for your term.

Apart from the operators AND and OR, you can use NOT, although this operator should be used carefully and only in situations in which you are completely certain about what to exclude. For instance, you can use NOT when you have carried out two different searches with different search strategies and you want to exclude the references in the second set of results that have already appeared in the first set. If your first set of results come from search #15 and your second set of results come from search #26, you will get the results only appearing in #26 and not in #15 by searching: #26 NOT #15.

The consequences of using the Boolean operators AND, OR or NOT is shown Fig. 2 .

The Boolean operators.

The main medical databases such as Medline and EMBASE have keywords organised as a tree structure, with main headings and subheadings. In PubMed , you can find these in the MeSH Database (choose MeSH in the search field next to the PubMed logo). If you search the term ‘osteoarthritis’ you will get an explanation of the use of this term in Medline. By going one step deeper, you will get subheadings and the tree structure (where this term appears in the Medline hierarchy of keywords):

• 
  

  All MeSH categories

  
  
• 
  

  Disease category

  
  
• 
  

  Musculoskeletal diseases

  
  
  
  
  • 
    

    Joint diseases

    
    
  • 
    

    Arthritis

    
    
  • 
    

    Osteoarthritis

    
    
  • 
    

    Osteoarthritis, hip

    
    
  • 
    

    Osteoarthritis, knee

    
    
  • 
    

    Osteoarthritis, spine

  
  
  
  
• 
  

  All MeSH categories

  
  
• 
  

  Disease category

  
  
• 
  

  Musculoskeletal diseases

  
  
  
  
  • 
    

    Rheumatic diseases

    
    
  • 
    

    Osteoarthritis

    
    
  • 
    

    Osteoarthritis, hip

    
    
  • 
    

    Osteoarthritis, knee

    
    
  • 
    

    Osteoarthritis, spine

  
  

You can see that osteoarthritis appears in two parts of the tree: rheumatic diseases and joint diseases.

In Medline you will search the term in all parts of the tree, and you will also search all narrower terms belonging to the particular MeSH. If you search for ‘osteoarthritis’, you will automatically search references with osteoarthritis; osteoarthritis, hip; osteoarthritis, knee; and osteoarthritis, spine. However, if you wish to limit your search to, for instance, diagnosis, you may in the PubMed MeSH database do so by choosing the subheading Diagnosis and ‘send to box with AND’, search and see the results.

In Medline searches via other hosts than PubMed, and in EMBASE, there might be a choice of ‘explode search’ or not to do so. This will achieve exactly what it describes: ‘explode search’ will include all references with the search term or with any of the subheadings; by not exploding, you will limit yourself to the chosen term.

Select suitable references from the retrieved ones

Depending on how much you choose to limit your search, you are likely to find several (and perhaps many) references which are not relevant for your purpose. As mentioned above, you have to find the right balance between sensitivity and specificity in the given situation. Around 200 or less is a reasonable number to skim through by looking at titles and abstracts, where these are available. Often, the title will help you decide whether you read further or not. Many databases have the possibility of looking at ‘related articles’ and, if you go just one layer into the ‘related articles’ search for the chosen references, you will get a useful supplement and a more complete coverage of the literature with your search.

The last step is to go through the reference lists of your chosen references to determine whether there are other references you have missed, either because they have not been included in the databases searched or because your search has not been able to catch them. You need to be aware that the electronic versions of the bibliographic databases start at different times; some go back a long way whereas others start around 1980. Reference lists are good sources for covering important, older literature, as well as covering important book chapters and conference proceedings.

Assess whether the search was acceptable

When your initial set of relevant references has been selected, it is time to check whether something is missing. A person who is new to a particular field will have a problem in assessing the expected number of references, because it may not be easy to see whether one should expect five studies or 1000. If the subject is a ‘hot topic’, a lot of studies ought to appear in that area. However, if the subject is brand new, there might not be more than – say – three conference abstracts. If you already know about one or two important studies, these ought to appear as part of your retrieved references. If they do not, your search strategies are not good enough, or you have not chosen the right combination of databases, or the references in question are not included in any of the available databases and have to be found elsewhere.

Redesign the search strategy and/or choose other databases/search tools

If your coverage of the literature was found to be unsatisfactory, you have to go back to step 1 and start again. You may get some help by looking at the references you know were missing to see whether you can get a clue about which additional search terms could usefully be added. In addition, look at Table 1 to see whether a search in another database would be the answer. You will then have to repeat steps 2–6. Otherwise, try other search tools.

Keeping up with the peer-reviewed journal articles

When you are satisfied with your search result, save your search strategy for use at later up-dates. Every time you update your knowledge in a field, you have to consider whether it is necessary to improve your search strategies. This is because your field develops all the time and you have to follow the various ways diseases and treatments change names over time to be able to include the new terms in your future searches. You will also find that keywords develop over time. As an example, fibromyalgia as a MeSH word appeared in 1989. If you want to search a term as a keyword earlier than the keyword appears in a database, you can try to search the keyword word above in the hierarchy/tree.

Most rheumatologists will have access to a variety of bibliographic databases via their workplace or research library. The licence fees for many of these databases are high, and you cannot expect to have access to all databases shown in Table 1 . For anybody who has no access to databases paid by their study or workplace, the US National Library of Medicine (NLM) provides free access to Medline and some other databases via PubMed, and PEDro is also free.

A major part of the medical profession will have access to the full-text versions of the journals through their hospital or research libraries. For those who do not have access to all licenced scientific journals, it is important to be aware of that more and more journals are becoming ‘open access’. This means that it is free to access the papers published and that the author or the author’s employer have paid a fee to make their paper freely available. Today many grant-giving bodies demand that the results of the research paid by their grants must be published as an open-access paper. Several of the old journals will therefore now provide an open-access option, if the authors or the grant-giving body will pay for this. This has suddenly given a much wider free access to electronic journals. A large selection of open-access papers in health science can be searched in PubMed Central, which is a part of PubMed. To get to PubMed Central, choose PMC in the box for choosing database at the top of the front page next to the PubMed logo. When you open PubMed, this box shows ‘PubMed’, which is the entrance to Medline via PubMed. There is a choice of several databases you can search via the search host PubMed.

Some open-access journals, which may be too new to fulfil the entrance criteria for bibliographic databases such as Medline or EMBASE, will be acceptable if they at least are searchable via CrossRef and have a digital object identifier (DOI) number. This will guarantee peer review and some scientific standard. All electronic scientific papers should have a DOI registration number, which is presented at the front page of the paper.

Other search tools

Bibliographic databases are the ‘safe’ places to search because the references included in these databases are from peer-reviewed journals or, in cases where books or book chapters are included, from recognised scientific publishers. However, it should be recognised that there are other limits to this ‘safe strategy’, because a useful part of the available information is published in journals that are not included in the described databases and these journals may also be peer reviewed and of perfectly acceptable quality. The major bibliographic databases concentrate on covering publications from publishers in Europe and the United States, and to some degree Japan, Australia and New Zealand, but valuable information could still be missed by searching these major bibliographic databases. The other issue is that many patients, and their relatives, now get their information about illnesses and treatment from the Internet. It is therefore also important to keep up with what the layman’s sources of knowledge are communicating in your field in order to be ahead of your patients and to supplement your more scientific sources of medical information, before you are asked questions about a certain condition or therapy or drug treatment.

Search engines

For scientific search via Internet search engines, it is important to understand the difference in search techniques in these engines from search in bibliographic databases. When using a search engine such as Google or Ixquick , you have to think of the main words and put them in the order of importance in the search field. Different search engines search in different ways, but most place a high importance to words appearing in headings and in the first paragraph of the text. The word placed as number one in a search will be counted as more important than the second or the third. Although it is possible to perform an ‘advanced search’, this does not make a great difference to the result and you cannot combine searches in the way described for bibliographic databases.

For scientific questions, it is recommended to search either Google Scholar or Scirus , which aim to provide high-quality answers. Another choice could be a multi-search engine such as Ixquick (this type of tool harvests results from searches in several search engines and presents them in one list) due to its large coverage.

Through search engines you will find many homepages on rheumatology subjects, including everything from stories of individual patients or from a next of kin to the information pages of patient organisations, homepages of learnt societies and lectures from university courses. You can also find online encyclopaedias such as Wikipedia . Wikipedia is not peer reviewed but because it is free on the Internet and anybody can make corrections in the written articles, you can often find solid and up-to-date information. However, it is also possible for the information to be incomplete or even sometimes incorrect, and you therefore should consider yourself responsible for evaluating any information that you take from Wikipedia or similar sources.

There are some important sites on the Internet if you need health statistics. The main ones are the World Health Organization’s (WHO) extended homepage and free pages of useful statistics from various countries’ National Board of Health. These sources can be considered reliable and need no evaluation.

Another important issue, well covered by the Internet, is bioethics. If you need up-to-date advice on bioethics, you can find the legal documents or interpretations of these under NLM’s Bioethics Information Resources or under Council of Bioethics Europe Division .

Evaluation of Internet resources where peer-review is not applied

It is necessary to evaluate any found useful resource before the information provided can be taken into account. The evaluation is simple and builds on common sense:

• 1.
  

  Does the page cover the information you are looking for at an acceptable level?

  
  
• 2.
  

  What is the URL (Uniform Resource Locator) address?

  
  
• 3.
  

  Is the information given sufficiently complete, correct and precise?

  
  
• 4.
  

  Who is the creator of this page?

  
  
• 5.
  

  Has the page an acceptable structure?

  
  
• 6.
  

  If there are links, what quality are these?

If you find that a page gives you the desired information, you can check whether the address indicates commercial interests or a private person, or whether it takes you to a well-known university, hospital site or a similar acceptable institution. Although the medical industry has commercial interests, research laboratories may post acceptable information on the Internet. You just have to use your judgement about the firms’ main marketing objectives when assessing the information given.

If a page gives any information you know is incorrect, you should not use any of the information given there. You should also require some recognisable expertise of the creator of a page that you will accept as an information source. Today, it is fairly easy to create a well-structured homepage, so you may ask yourself if you can trust a page where it is very difficult to find the information. Assessing links is usually easy at sites of interest for rheumatologists. There will most often be only a couple of references to peer-reviewed journals or to health-care institutions, and there will be very few others. Only commercial pages will have advertising and similar and must be looked upon with some apprehension. Useful sites are homepages from patients’ organisations who, in many cases, will have professional staff providing the information on their homepages.

Books and book chapters

During your professional life, it is important to keep up with the development of the subjects in your areas, and this is often most easily done at any given time by reading the newest textbooks in the field. Another source of useful books is the many doctoral theses. To follow the books published in your field over time, you can search any medical research library’s catalogue. You may not find all published literature of interest, but you will, if you search at yearly intervals, be made aware when it is time to bring your knowledge up-to-date. If there are a couple of theses and two new textbooks published during the year, it is time to read up on the subject.

Open access has also reached the book market. You may therefore find several new subject-specific books in monograph series such as for instance InTech Press or Future Medicine , where all books are edited by specialists and all are free to access. Older handbooks may also appear on the Internet for free. With these you must be aware that the freely available version often may be an older edition, or for book chapters, perhaps a pre-proof version with yet-to-be-corrected mistakes.

Searching for a systematic review

The most advanced type of literature search is the search necessary to be able to write a systematic review. A systematic review has to qualify for being ‘systematic’ and not being an editorial or similar, which means that it has to cover ‘all’ relevant published studies in the area. The review has to be based on a protocol, and this protocol has to be published via sites such as PROSPERO prior to starting the search. The Cochrane Handbook will guide through how such a review – and a possible planned meta-analysis – has to be structured, and the (PRISMA) Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement must also be considered, but this is outside the scope of this paper.

The main message when preparing a search for a systematic review is:

• ▪
  

  Define the aim and objectives of the systematic review clearly, you are not just writing an essay.

  
  
• ▪
  

  Work hard on the search strategy, it has to be good and comprehensive, as you are bound to follow it after publishing your protocol.

  
  
• ▪
  

  Choose all bibliographic databases you can imagine.

  
  
• ▪
  

  Search meeting abstracts, as well as databases covering protocols for studies such as Clinical Trials.gov .

  
  
• ▪
  

  Scrutinise reference lists of chosen studies and major reviews.