Cost-effectiveness analysis

SUMMARY

Why is this tool used in evaluation?

Cost-effectiveness analysis is a decision-making assistance tool. It identifies the economically most efficient way to fulfil an objective. In evaluation, the tool can be used to discuss the economic efficiency of a programme or a project.

Focused on the targeted major result of the activity - the number of jobs created - the tool estimates the cost of each job generated by a specific measure. The comparison of various programmes with similar impacts enables the comparison of the costs generated by each job created and provides useful quantitative indicators for the selection of comparative methodologies.

The tool compares policies, programmes or projects. It presents alternatives in order to identify the most appropriate one to achieve a result at least cost.

Cost-effectiveness analysis may contribute to answer the following questions:

• How much does a programme or a measure costs compared with the cost of a particular component of its objective?
• Is it preferable to invest resources in an intervention, to the detriment of another, to achieve the target?
• What kind of intervention or group of interventions yields the best outcomes regarding the final objectives and available resources?
• How can the use of the resources be optimised, given competing needs between programmes?
• At what level of additional investment will the chosen intervention clearly give an improved outcome?

What are the possible uses of cost-effectiveness analysis?

Specificities of the cost-effectiveness analysis:

• Effectiveness is measured with a single outcome which stands as the main expected impact of the intervention.
• It is an economic analysis methodology which assesses the effectiveness of indicators highlighting results and outcomes. It does not evaluate the monetary value of the outcomes.

Figure 1: The uses of cost-effectiveness analysis

Cost effectiveness analysis is an efficient way to evaluate projects, programs or sectors evaluation when the main objective of the policy can be reduced to a single result. This tool is designed for the economic analysis of the operational objectives at different levels. 

Cost-effectiveness analysis can be used in:

• Ex ante evaluations to support decision-making and guide the choices to be made.
  Depending on the cases, it can be used:
• To foster the debate among decision-makers prior to the decision
• To highlight the preferences of the groups representing different categories of stakeholders or actors involved in the sectors where the intervention is planned
• Ex post evaluations to measure the economic efficiency of an intervention already carried out.
• Intermediary evaluation to update the ex ante outcomes and choose which options should be selected to continue the intervention.

How is a cost-effectiveness analysis undertaken?

Figure 2 : Steps involved in cost-effectiveness analysis

Check the relevance of the analysis to the objectives of the programme

If the outcome of a programme cannot be defined as a priority outcome, or if homogeneous and quantifiable units cannot be determined, the use of cost-effectiveness analysis should be avoided.

The method is adapted for actions in which expected outcomes are clearly identified and whose direct and indirect costs are easily measurable.

Identify the availability and reliability of the data

The analysis requires reliable data, i.e.:

• In ex post evaluations, a quantification of the outcomes
• In ex ante evaluations, a rigorous modelling of the most probable results

Determine the effectiveness criteria and develop the relevant indicator

The choice of the effectiveness criteria depends on the main objective of the intervention.

• In an intervention where the objective is clearly determined, the identification of the effectiveness criteria is straightforward.
• Conversely, when the intervention's objective is broad, the identification of the intervention's main objective should be discussed.

For example, when an intervention aims at improving the effectiveness of the basic education provision, the effectiveness criteria could be the increase in the average level of primary school's basic knowledge. Other criteria may be more relevant depending on the context in which the intervention is implemented.

This increase can be measured through the evolution of the grades obtained in all the courses followed by the students, or in the two courses deemed as the most important, or through the organisation of a single examination for all primary school students.

How is the total cost of the programme evaluated?

Add direct costs

In this type of calculation, only the direct costs invested in the intervention are considered. In the context of development assistance intervention, these costs are often financial: grant, financial transfers, decreases in taxes, financing of projects and activities, etc.

Examine indirect costs

Indirect costs indicate the value of civil servants' work in charge of monitoring the programme or intervention.

Examine other types of costs

An additional level of complexity within cost calculations is required when other important costs are generated by the project's implementation. For example, cost calculations can include the loss of earnings and benefits due to the fact that public financing have been attributed to a specific objective (this is called the loss opportunity cost).

How is the impact of the programme measured?

Ex ante evaluations

The evaluator must forecast the quantitative results of the programme. Depending on the complexity of the intervention, the use of simulation techniques may be required.

Ex post evaluations

The evaluator can use empirical techniques if the primary data at his/her disposal are sufficiently numerous and reliable. If not, the evaluator needs to estimate the actual quantitative results from secondary data.

How is a costs-to-effectiveness ratio established?

The analysis requires stable elements to support the comparison between:

• Interventions of different nature, but similar objectives, occurring in the same country
• Similar interventions occurring in similar contexts
• The results and what would happen without the intervention, etc.

Comparison of programmes

When the analysis compares different programmes with identical outcomes, the chosen parameter is the cost comparison criteria.

When, for the same objective, the analysis compares different types of interventions with identical costs, it is supported by qualitative elements.

What are the preconditions for its use?

What are the advantages and limitations of the tool?