The identification of gaps from systematic reviews is essential to the practice of “evidence-based research.” Health care research should begin and end with a systematic review.1-3 A comprehensive and explicit consideration of the existing evidence is necessary for the identification and development of an unanswered and answerable question, for the design of a study most likely to answer that question, and for the interpretation of the results of the study.4
In a systematic review, the consideration of existing evidence often highlights important areas where deficiencies in information limit our ability to make decisions. We define a research gap as a topic or area for which missing or inadequate information limits the ability of reviewers to reach a conclusion for a given question. A research gap may be further developed, such as through stakeholder engagement in prioritization, into research needs. Research needs are those areas where the gaps in the evidence limit decision making by patients, clinicians, and policy makers. A research gap may not be a research need if filling the gap would not be of use to stakeholders that make decisions in health care. The clear and explicit identification of research gaps is a necessary step in developing a research agenda. Evidence reports produced by Evidence-based Practice Centers (EPCs) have always included a future research section. However, in contrast to the explicit and transparent steps taken in the completion of a systematic review, there has not been a systematic process for the identification of research gaps.
In a prior methods project, our EPC set out to identify and pilot test a framework for the identification of research gaps.5,6 We searched the literature, conducted an audit of EPC evidence reports, and sought information from other organizations which conduct evidence synthesis. Despite these efforts, we identified little detail or consistency in the frameworks used to determine research gaps within systematic reviews. In general, we found no widespread use or endorsement of a specific formal process or framework for identifying research gaps using systematic reviews.
We developed a framework to systematically identify research gaps from systematic reviews. This framework facilitates the classification of where the current evidence falls short and why the evidence falls short. The framework included two elements: (1) the characterization the gaps and (2) the identification and classification of the reason(s) for the research gap.
The PICOS structure (Population, Intervention, Comparison, Outcome and Setting) was used in this framework to describe questions or parts of questions inadequately addressed by the evidence synthesized in the systematic review. The issue of timing, sometimes included as PICOTS, was considered separately for Intervention, Comparison, and Outcome. The PICOS elements were the only sort of framework we had identified in an audit of existing methods for the identification of gaps used by EPCs and other related organizations (i.e., health technology assessment organizations). We chose to use this structure as it is one familiar to EPCs, and others, in developing questions.
It is not only important to identify research gaps but also to determine how the evidence falls short, in order to maximally inform researchers, policy makers, and funders on the types of questions that need to be addressed and the types of studies needed to address these questions. Thus, the second element of the framework was the classification of the reasons for the existence of a research gap. For each research gap, the reason(s) that most preclude conclusions from being made in the systematic review is chosen by the review team completing the framework. To leverage work already being completed by review teams, we mapped the reasons for research gaps to concepts from commonly used evidence grading systems. Briefly, these categories of reasons, explained in detail in the prior JHU EPC report 5, are:
The framework facilitates a systematic approach to identifying research gaps and the reasons for those gaps. The identification of where the evidence falls short and how the evidence falls short is essential to the development of important research questions and in providing guidance in how to address these questions.
As part of the previous methods product, we developed a worksheet and instructions to facilitate the use of the framework when completing a systematic review (See Appendix A). Preliminary evaluation of the framework and worksheet was completed by applying the framework to two completed EPC evidence reports. The framework was further refined through peer review. In this current project, we extend our work on this research gaps framework.
Our objective in this project was to complete two types of further evaluation: (1) application of the framework across a larger sample of existing systematic reviews in different topic areas, and (2) implementation of the framework by EPCs. These two objectives were used to evaluate the framework and instructions for usability and to evaluate the application of the framework by others, outside of our EPC, including as part of the process of completing an EPC report. Our overall goal was to produce a revised framework with guidance that could be used by EPCs to explicitly identify research gaps from systematic reviews.
