by Alison O'Mara-Eves
Poster presentation at the 2011 Cochrane Collaboration Colloquium.
Suggested Citation:
O’Mara AJ, Jamal F, Parry W, Lorenc T, Cooper C. (2011, Oct). Guidelines for conducting and reporting reviews of reviews: Dealing with topic relevance and double-counting. Presented at the 19th Cochrane Colloquium, 19-22 October 2011, Madrid, Spain.
Background:
Reviews of review-level evidence (tertiary reviews) are desirable when a research question is... more
Background:
Reviews of review-level evidence (tertiary reviews) are desirable when a research question is time-sensitive and/or the scope is broad. However, reviews included in a tertiary review often have only partial overlap with the tertiary review’s research question and, consequently, not all included studies are relevant to the tertiary review. Additionally, the reviews can include some of the same primary studies (known as double-counting). These concerns might lead to biases in the evidence base.
Objectives:
To explore the issues of relevance and double-counting in a tertiary review and to present guidelines for identifying and addressing potential related problems.
Methods:
We examined data from a completed systematic tertiary review on a public health effectiveness topic. We established the relevance of the included reviews by determining how many of the primary studies included in each review met our inclusion criteria and by analysing the included reviews’ synthesis sections using the PICO elements (population, intervention, comparison, outcome) in relation to the tertiary review research question. We demonstrate graphical (plot-based) approaches and a matrix-based approach to establishing the extent of double-counting.
Results:
Of twenty reviews that met our inclusion criteria, ten reviews had less than 50% of the primary studies included that were relevant to our research question. Exploring the synthesis sections of included reviews using a PICO framework was useful in establishing the degree of relevance of the findings to the tertiary review research question and yielded a review ‘utility’ rating. Our graphical and matrix-based approaches allowed us to evaluate the extent of double-counting across reviews; 14 primary studies were included in more than one review, with some studies appearing in four reviews.
Conclusions:
Issues of relevance and double-counting need to be assessed in tertiary reviews, but are often overlooked. The guidelines proposed can help identify potential biases and attempt to address them.
Download (.pdf) 