Understanding the evidence base

Understanding the evidence base

The evidence around interventions for children and service delivery can be complex. What Works aims to be clear and transparent about what constitutes good evidence and to provide the necessary information about interventions that will make the site a useful tool to support decision making.

Although randomised control trials and systematic reviews are usually considered the gold standard of research, to avoid excluding interventions which may achieve good outcomes but which have not yet been evaluated in this way, we have not limited the evidence base to these levels.

Instead, we have included a range of evidence levels representing a good level of evaluation. A set of criteria is applied to every intervention, ensuring robust and transparent decisions are made on which interventions to include and at which level of evidence.

For more information on the evidence levels and the criteria, please see our guide, which explains each of the criteria in detail, and two new support documents authoured by James Law and Jenna Charlton. The guide and support documents explain how the Moderating Group make their decisions about which interventions to include and what makes up a good evidence base, as well as providing links to further information on accessing and conducting research. 

Strength of evidence versus strength of outcomes

It's essential whilst using the database to be aware of the important distinction between the strength of the evidence and the strength of the intervention's outcomes for children and young people. This is due to some interventions on the database not having wholly positive outcomes for children but still being included on the database because of the strength of research.  Strong evidence doesn't necessarily mean an intervention works well. Equally other interventions may only be classed as "indicative" in terms of research design, but the outcomes for children look very good indeed.

It is therefore important when making decisions around which interventions to use not to choose interventions based solely on levels of evidence, as an intervention at a lower level of evidence may be more appropriate and may show better outcomes.

Effect Sizes

Looking at the effect sizes of interventions on specific outcomes can support practitioners to interpret whether an intervention is appropriate for their setting. Effect sizes of interventions can be viewed on the What Works database via a graphic which is linked to from each intervention’s page. Each graphic provides information on the outcome that the intervention had the largest effect on, the smallest effect on and the effect size for the primary outcome as stated by the researchers (in other words what the researchers were primarily looking at in their study).

Effect sizes were drawn from the study which provided sufficient evidence to calculate them given the means and standard deviations, these may not be representative of all the studies for this type of intervention.

For more information on this, see our animation and guidance here.