Guide
Levels of evidence and systematic review vs meta-analysis: a plain guide
Levels of evidence rank research by how well its design protects against bias. For treatment questions, systematic reviews of randomised trials rank highest, then single randomised trials, then observational studies, then case series and expert opinion. A systematic review is a structured search and appraisal; a meta-analysis is the statistics that may pool its results.
This is information about reading evidence, not medical advice. Talk to your doctor or pharmacist before changing anything.
Why levels of evidence exist
Evidence-based medicine was defined by Sackett and colleagues in the BMJ in 1996 as "the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients." The word doing the work is best. Some study designs are much better than others at ruling out chance, bias and confounding, so a ranking helps you weigh a claim quickly.
A level is a starting point, not a verdict. A badly run trial can be worth less than a careful cohort study, and the right design depends on the question.
The evidence pyramid (printable)
This table follows the hierarchy most readers meet, with the matching Australian NHMRC level for intervention questions (does a treatment work?).
| Rank | Study design | What it is | NHMRC level (intervention) | Main weakness |
|---|---|---|---|---|
| 1 | Systematic review of randomised controlled trials (with or without meta-analysis) | A pre-planned search and appraisal of all trials on one question | I | Only as good as the trials it includes |
| 2 | Randomised controlled trial (RCT) | People are randomly allocated to treatment or comparison | II | Can be small, short, or unlike real patients |
| 3 | Pseudo-randomised trial | Allocation by a non-random rule (e.g. alternate patients) | III-1 | Allocation can be predicted and gamed |
| 4 | Comparative study with concurrent controls: cohort, case-control, non-randomised trial | Groups compared at the same time without randomisation | III-2 | Confounding: groups may differ in other ways |
| 5 | Comparative study without concurrent controls: historical controls, single-arm studies | Results compared with past patients or other single-arm studies | III-3 | Practice and patients change over time |
| 6 | Case series | Outcomes in a group given the treatment, no comparison | IV | No way to know what would have happened anyway |
| 7 | Case report, expert opinion, mechanism or lab studies | One patient, a view, or a plausible pathway | Not ranked in this NHMRC table | Signals and hypotheses, not answers |
Notes for using it:
- NHMRC has separate columns for other question types (diagnostic accuracy, prognosis, aetiology, screening). For prognosis, for example, a prospective cohort study is Level II. Pick the column that matches your question.
- A systematic review is only Level I if it reviews Level II studies. NHMRC's notes say a systematic review "will only be assigned a level of evidence as high as the studies it contains," except where those are Level II studies.
- Other hierarchies exist. The Oxford Centre for Evidence-Based Medicine publishes its own levels. They follow the same logic.
Systematic review vs meta-analysis
These are often used as if they were the same thing. They are not.
| Systematic review | Meta-analysis | |
|---|---|---|
| What it is | A method for finding, appraising and summarising all the research on a question | A statistical technique for combining numerical results from two or more studies |
| Cochrane Handbook definition | "attempts to collate all the empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question" | "the statistical combination of results from two or more separate studies" |
| Always present? | It is the whole project | Optional. "A systematic review need not contain any meta-analyses." |
| Output | Narrative and tables, often a GRADE summary of certainty | A pooled effect estimate with a confidence interval, usually shown as a forest plot |
| When it misleads | Narrow search, weak included studies | Pooling studies that are too different, or biased small studies |
A meta-analysis can also be done outside a systematic review, for example pooling a company's own trials. Without a systematic search behind it, the pool may be selective. The Cochrane Handbook warns that meta-analyses "have the potential to mislead seriously" if study designs, biases, variation between studies and reporting biases are not handled carefully.
Certainty of evidence: GRADE
Levels rank designs. GRADE rates the body of evidence for a specific outcome in four levels: high, moderate, low or very low (Guyatt and colleagues, BMJ, 2008, called this "quality of evidence"; Cochrane now says "certainty"). Evidence from randomised trials starts high and can be downgraded for study limitations, inconsistency, indirectness, imprecision or reporting bias. Observational studies start low and can occasionally be rated up, for example when an effect is very large. That is why a Cochrane review of trials can still report "low-certainty evidence": the trials were there, but they were small, unblinded or inconsistent.
When you read a review, find its summary of findings table. The certainty rating often tells you more than the headline.
A newer view of the pyramid
In 2016 Murad and colleagues (Evidence-Based Medicine) proposed a revised pyramid. They argued that systematic reviews and meta-analyses are better seen as a lens for "consuming evidence" (appraising, synthesising and applying it) than as the top layer, and that the lines between levels should be wavy, because quality can move a study up or down. It is a useful corrective to reading the pyramid too literally.
Five habits for reading a claim
- Name the design. Trial, cohort, case series, or review? The abstract usually says.
- Match the design to the question. Treatment questions favour trials. Harm and prognosis questions often rely on cohort studies.
- Check the certainty, not only the level. A review of poor trials is still weak.
- Treat an abstract as an abstract. Conference abstracts and preprints leave out methods. Hold them as signals.
- Separate standard of care from investigational. An early-phase or single-centre result is not a change in practice.
Where MIA fits
These habits are the ones MIA works by. Put a paper on the desk and the working separates what it shows (finding), what may reasonably be drawn from it (inference), and what it does not establish (gap). An abstract without the full paper stays a gap. Clinical Conservative is the default, so standard of care is kept apart from investigational or speculative claims, and thin evidence is named as thin. Research Expansion, for hypothesis work, always opens with a banner saying it is not standard of care. Deep Review runs two independent workings and settles them: agree, contest, or still a gap.
MIA reads the papers and files you bring. It is not a search engine over the literature, it is not a clinician, and it gives no dosing or treatment advice.
Test what a paper will bear: 150 free credits, no card.
FAQ
What are the levels of evidence? A ranking of study designs by how well they protect against bias. For treatment questions: systematic reviews of randomised trials, then randomised trials, then non-randomised and observational studies, then case series, with case reports and expert opinion lowest.
What is the difference between a systematic review and a meta-analysis? A systematic review is a structured method for finding and appraising all the studies on a question. A meta-analysis is a statistical technique that combines the numerical results of two or more studies. A systematic review may or may not include a meta-analysis.
Is a meta-analysis always the highest level of evidence? No. It is only as strong as the studies it pools and how well it handles differences between them. A meta-analysis of small, biased studies can mislead.
What is evidence-based medicine? Sackett and colleagues defined it in 1996 as the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients, integrated with clinical expertise.
This is information about reading evidence, not medical advice. Talk to your doctor or pharmacist before changing anything.
Sources
- Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71-72. doi:10.1136/bmj.312.7023.71 - National Health and Medical Research Council. NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Canberra: NHMRC; 2009. Table reproduced at https://www.ncbi.nlm.nih.gov/books/NBK121300/table/appb.t21/ - Higgins JPT, Thomas J, Chandler J, et al (eds). Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane, 2024. Chapter 1 and Chapter 10 (Deeks JJ, Higgins JPT, Altman DG, et al). https://training.cochrane.org/handbook - Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. doi:10.1136/bmj.39489.470347.AD - Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016;21(4):125-127. doi:10.1136/ebmed-2016-110401