Post-publication peer review of systematic review search strategies

  • Author(s) / Creator(s)
  • Presented at the CHLA/ABSC 2021 Virtual Conference.

    Systematic reviews (SRs) rely on well-designed searches to locate all relevant studies in order to avoid bias in their conclusions. Clear, accurate and detailed reporting of SR searches ensures transparency and replicability. This project aims to evaluate the quality of current SR search strategy reporting from five top-tier medical journals (Lancet, JAMA, BMJ, PLOS Medicine, and BMC Medicine) published between 2017-2019.


    To identify current SRs, we searched Ovid MEDLINE for “systematic review” in the title and abstract fields, limited to selected journal names, and restricted by date. We randomly selected 100 articles to include in the data analysis. The team developed a data extraction form and code book, incorporating Peer Review of Electronic Search Strategies (PRESS) criteria for consistent evaluation. Two librarians independently extracted data from each article. Disagreements were resolved through consensus.

    Results show that there is inconsistent reporting of searches in SR publications in top-tier medical journals. Seventeen percent of SRs failed to make a replicable search available for review and 55% failed to clearly indicate who was responsible for the search strategy. Only 14% clearly indicated a librarian was responsible for the search and 28% reported some degree of librarian involvement. Only 2% of the articles included a librarian as a co-author.

    The peer review and editorial process needs to include more consideration of the search methods, strategies, and reporting for quality assurance purposes. This project has been an opportunity for all team members to reflect on their own search strategy development process.

  • Date created
  • Subjects / Keywords
  • Type of Item
    Conference/Workshop Presentation
  • DOI
  • License
    Attribution-NonCommercial 4.0 International