Navigation path

Additional tools

  • Print version
  • Decrease text
  • Increase text

EPIC Evidence-Based Practices

Review Criteria and Process

This document summarizes the criteria that are used to designate practices as “evidence-based” and the process that this project uses to review practices.

Evidence Criteria

Evidence-based practices on this site are assigned one of three evidence levels:  either emerging, promising, or best practice.  Criteria to determine the evidence level are organized according to three categories, “evidence of effectiveness,” “transferability,” and “enduring impact.”

How do we determine evidence level ?

How do we assess…

How do we determine Evidence level?

Best Practice

A “best practice” has achieved at least a + in each of the three evidence categories, including “evidence of effectiveness”, “transferability” and “enduring impact.”

Promising Practice

A “promising practice” has achieved at least a + in “evidence of effectiveness” and a + in at least one of the other two categories, “transferability” and “enduring impact.”

Emergent Practice

An “emergent practice” has achieved at least a + in “evidence of effectiveness.”

Here is a graphical overview of how evidence designations are assigned:

The following tables summarize the criteria for each of the three evidence categories.

 

Evidence of effectiveness criteria

Category

Requirements for +

Requirements for ++

Comparison group

Evaluation utilizes at the minimum pre/post design with appropriate statistical adjustments employed in order to control for selection

Study design uses a convincing comparison group to identify practice impacts, including randomized-control trial (experimental design) or some quasi-experimental designs.

Statistical significance

Significant (p<0.1), positive results are shown on at least one relevant outcome

Significant (p<0.05), positive results are shown on at least one relevant outcome

Effect size

No requirement

Effect size of at least 10% of a standard deviation

Sample size

Sample size of at least 20 in each group

Sample size of at least 50 in each group

Outcomes

Outcomes are directly or indirectly related to outcomes identified in topic definitions

No significant negative outcomes reported (excluding those negative outcomes that might be due to chance)

 

 

Outcomes are directly related to outcomes identified in topic definitions

No significant negative outcomes reported (excluding those negative outcomes that might be due to chance)

Outcome assessments have been validated, where applicable

Outcome assessments conducted at baseline and follow-up, where applicable

Attrition

No requirement

Attrition is less than 25% or has been accounted for using an acceptable procedure, where applicable

Location

At least one evaluation that meets the above criteria must have been conducted within EU member state(s)

 

Transferability criteria

Category

Requirements

Replication

Practice has been evaluated in at least one additional population beyond the original study population (broadly defined) in such a way that at least meets the basic criteria for internal validity

 

Practice materials

Practice materials (curriculum, etc) are available, or documentation is sufficient, such that program can be replicated

 

Enduring Impact criteria

Category

Requirements

Follow-up conducted

An evaluation of the practice which meets the basic criteria for inclusion has conducted a follow-up of at least 2 years, and finds at least one positive outcome (p<0.1)

 

Review Process

Here, we provide an overview of the review process for practices.

How do We Identify Practices to Review?

We identify practices to review primarily through these sources:

  • Practices that have been submitted to the Practice User Registry and indicate that they have been evaluated.
  • Nominations from the project’s county contacts from the 27 EU Member States.
  • Suggestions from users, which can be submitted to EMPL-EPIC@ec.europa.eu.
  • Practices that project team members have identified in journals, email newsletters and other sources.

While these are the primary ways we identify practices to review, there are likely to be additional ad hoc ways that practices come to our attention as well.

What Are the Basic Requirements that a Practice Must Meet in Order to be Reviewed?

These are the most basic requirements that practices must meet in order for us to consider them for review:

  • The practice was implemented in one of the 27 EU Member States.
  • The practice was evaluated in one of the 27 EU Member States.
  • The evaluation includes some type of comparison group.
  • The evaluation findings are communicated in some written form.
  • The practice pertains to one of the topics that the EPIC website addresses.

Practices that meet these requirements are entered into the review process, where additional criteria are used to determine whether they are listed as an evidence-based practice.

What Are the General Procedures Used for the Review?

This is a brief summary of the general procedures used to review practices for the “Evidence-Based Practices” section:

  • Once it has been determined that the practice meets the basic requirements listed above, the practice is assigned a review team comprised of social scientists who are specially trained in this review process.  Each review team has one lead reviewer.
  • The lead reviewer conducts a comprehensive literature search to identify evaluations of the practice that are available and contacts that practice contact to see if there are any additional evaluations that were not identified in the literature search.
  • The lead reviewer abstracts data from the evaluations related to the evidence criteria and enters these data in a specially designed abstraction table and also recommends the evidence level for the practice.
  • Two additional project team members review the abstraction table and the evidence level designation.  The table may undergo revisions until both team members approve it.
  • Once the table is approved, the lead reviewer drafts a practice summary, which is organized in a template designed especially for this project.
  • The same two additional project team members also review the practice summary, which is revised until these team members approve of the summary.
  • This version of the summary is sent to a member of the project’s Expert Panel for additional peer review, and it is also sent to the practice contact person, who checks the content for accuracy.
  • Once the Expert Panel reviewer and the contact person have approved the practice summary, it gets posted on the website.

What Happens When New Evaluation Results Are Available for a Practice?

We strive to provide the most up-to-date evaluation information available for practices.  If you are aware of new evaluation findings that we can add to a practice summary, we hope you will let us know either by sending email to emailaddress.org.

If you have additional questions regarding reviews of evidence-based practices, please contact EMPL-EPIC@ec.europa.eu.