Systematic reviews of quantitative studies often involve statistical integration of findings through meta-analysis, a procedure whose advantages include objectivity, enhanced power, and precision; meta-analysis is not appropriate, however, for broad questions or when there is substantial inconsistency of findings.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins
Chapter 29
Systematic Reviews of Research Evidence: Meta-Analysis,
Metasynthesis, and Mixed Studies Review
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
Evidence-based practice relies on rigorous integration of research evidence on a topic through systematic reviews.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
True
Evidence-based practice relies on rigorous integration of research evidence on a topic through systematic reviews. A systematic review methodically integrates research evidence about a specific research question using carefully developed sampling and data collection procedures that are spelled out in advanced in a protocol.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Systematic Review
- Integrates research evidence about research question
- Carefully developed:
- Sampling
- Data collection procedures
- Spelled out in advance in a protocol
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
Systematic reviews of qualitative studies often involve statistical integration of findings through meta-analysis.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
False
Systematic reviews of quantitative studies often involve statistical integration of findings through meta-analysis, a procedure whose advantages include objectivity, enhanced power, and precision; meta-analysis is not appropriate, however, for broad questions or when there is substantial inconsistency of findings.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Meta-Analysis
- Objectivity
- Enhanced power
- Precision
- Not appropriate
- Broad questions
- Substantial inconsistency of findings
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Criteria for Using Meta-Analytic Techniques in a Systematic Review
- The research question being addressed or the hypothesis being tested across studies should be very similar, if not identical.
- Concern regarding whether there is a sufficient base of knowledge for statistical integration
- Consistency of evidence
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Steps in a Meta-Analysis
- Formulating the problem
- Designing the meta-analysis study
- Searching the literature for data
- Evaluating study quality
- Extracting and encoding data for analysis
- Calculating effects
- Analyzing the data
- Writing a meta-analytic report
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
The steps in both quantitative and qualitative integration are similar.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
True
The steps in both quantitative and qualitative integration are similar and involve formulating the problem, designing the study, searching the literature for a sample of primary studies, evaluating study quality, extracting and encoding data for analysis, analyzing the data, and reporting the findings.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Quantitative and Qualitative Research
- Formulating the problem
- Designing the study
- Searching the literature for a sample of primary studies
- Evaluating study quality
- Extracting and encoding data for analysis
- Analyzing the data
- Reporting the findings
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
There is consensus on systematic reviews should include the grey literature.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
False
There is no consensus on whether systematic reviews should include the grey literature—that is, unpublished reports.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Concerns
- No consensus on whether systematic reviews should include the grey literature
- Quantitative studies
- Bias against the null hypothesis
- Publication bias
- Examined with funnel plot
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Meta-Analysis
- Effect size index: quantifies the magnitude and direction of relationship between variables
- Two common effect size in nursing
- Standardized mean difference
- Odds ratio
- Pooled to yield an estimate of the population effect size
- Weighted average
- Inverse variance
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Statistical Heterogeneity
- Fixed effects model: assumes a single true effect size
- Random effects model: assumes a distribution of effects
- Forest plot
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Nonrandom Heterogeneity
- Subgroup analyses
- Meta regression
- Purpose: identify clinical or methodologic features systematically related to variation in effects
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Quality Assessments
- Exclude weak studies from reviews
- Differentially weight studies or sensitivity analyses: test whether including or excluding weaker studies changes conclusions
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
Metasyntheses are more than just summaries of prior quantitative findings; they involve a discovery of essential features of a body of findings and, typically, a transformation that yields new insights and interpretations.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
False
Metasyntheses are more than just summaries of prior qualitative findings; they involve a discovery of essential features of a body of findings and, typically, a transformation that yields new insights and interpretations.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Undertaking Metasynthesis
- “A family of methodologic approaches to developing new knowledge based on rigorous analysis of existing qualitative research findings”
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Approaches
- Meta-ethnography
- Metastudy
- Metasummary
- Critical interpretive synthesis (CIS)
- Grounded formal theory
- Thematic synthesis
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Metasynthesis Analysis
- Analyzing and interpreting data
-Noblit and Hare approach
-Paterson and colleagues approach
-Sandelowski and Barroso approach
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Paterson and Colleagues
- Metadata analysis: the study of results in a specific substantive area through analysis of the processed data
- Metamethod: the study of the studies’ methodologic rigor
- Metatheory: the analysis of the theoretical underpinnings on which the studies are grounded
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Sandelowski and Barroso
- Summaries: descriptive synopses
- Syntheses: interpretative explanations of the data
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Metasummary
- Developing a list of abstracted findings from the primary studies
- Manifest effect sizes
- Frequency effect size
- Intensity effect size
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Mixed Methods Research
- Systematic mixed studies reviews
- Systematic reviews
- Disciplined
- Auditable
- Integrate and synthesize findings
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Reporting Guidelines
- PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) is useful for writing up a systematic review of RCTs.
- MOOSE (Meta-analysis of Observational Studies in Epidemiology) guides reporting of meta-analyses of observational studies.


