Problem diagram

SUMMARY

.

Why are this tools used in evaluation?

.

The analysis of problems is a means to test the validity of the objectives of a project, a programme or a strategy. As a programme aims at solving a range of problems, the evaluation should be concerned with the validity of its analysis. The evaluators should therefore check:

  • The validity of the procedure: how have the problems been identified and classified?
  • The apparent coherence of the problems' linkage: are their causal links relevant?
Figure 1: Analysis of the problems in the project cycle and in the strategy policy co-ordination cycle

.

Programmes and strategies depend on the analysis of the situation. This analysis presents the primary data relative to the problems addressed by the programmes and strategies, and the information about the context (economic, social, cultural, institutional, environmental) in which they will be implemented.

This crucial analysis identifies:

  • The problems to be addressed by the activities in order of priority
  • The positive elements on which activities can rely
  • The actors to be involved in the activities
  • The structural and dynamic constraints which may hinder the success of the activities

Therefore, the evaluation team can be asked to assess the quality of the analysis, and the conformity between the analysis of the situation and the adopted strategy (or programme). The problem diagram, as an ex post construction, can be one of the tools used to check the coherence and the relevance of the analysis in respect of the main contextual problems.

.

What are the possible uses of these diagrams?

.

When it is impossible to directly establish objectives diagrams, problem diagrams play an essential role in the organisation stage of the evaluation. Problem diagrams present a summarised vision of the situation to be improved, partially at least, by the strategy. The classified objectives included in the strategy should be deduced from the problems diagrams.

.

Figure 2: The objectives deduced from the problems

.

The reconstruction of the problem diagram includes a step which differentiates between context problems and intervention problems. As a consequence, the diagram resulting from this selection should be completely convertible into a logically reconstructed objectives diagram, i.e. the higher-order overall objective corresponds to the core problem, and each row of subordinated objectives to its equivalent row of problems.

.

How is the problem diagram constructed?

.

Stage 1: How to identify the problems?

Record the references to teh problems found in the documentation: quotations from the evaluation's baseline documents are used to identify problems which may not be systematically depicted as such in the texts. Sometimes, problems are identified as assistance objectives or impacts targeted by these objectives. Problems are thus expressed as:

  • problems (presented as such)
  • objectives, whose goal is to resolve explicitly or implicitly a problem
  • expected impacts of the CSP assistance

The problems directly targeted by the intervention may not be explicitly identified. Main problems and context problems can be intermingled, which complicates the construction of the diagram. The evaluator will only be able to differentiate the two types of problems after the completion of the diagram. 

The crucial stage of the process is the identification of the core problem among the variety of the selected problems. Three situations can be encountered:

  • The core problem is mentioned in the documentation
  • The core problem is not clearly mentioned, whereas the overall objective is. In this case, the evaluator can deduce the core problem from the higher-order objective.
  • Neither the core problem, nor the overall objective are explicitly mentioned

In the last situation, the evaluator plays the role of the planning manager, and decides which of the problems will be the core problem. He/She may:

  • Take the decision, on the basis of his/her rationale
  • Benefit from the assistance of specialists during interviews or working sessions

The selection of the core problem should be conducted concurrently with the classification of the problems into levels. Indeed, the selection of the core problem should be supported and justified by the coherence of the whole diagram.

Stage 2: How to construct a problem diagram?

Figure 3: Stages involved in the drafting of temporary diagram 

Usage highlights the fact that these two types of problems/causes are the easiest to identify, whereas intermediary causes are the hardest to determine and classify. Thus, it is recommended that the development of the diagram starts with its extremities at the same time.

Figure 4 : Problem diagram in the transport sector in Tanzania

Test the temporary diagram

Where possible, the authors of the documentation referred to above should test the diagram in order to validate the classification of the problems by rows and links. The aim is to check that the evaluator's interpretation reflects the authors' intention correctly. If the authors are not available or, in order to complement their original contribution, the evaluator should consider asking for the participation of other actors responsible for the drafting process. 

Contacting the authors is usually possible when the documentation is recent and the authors are still in their position or contactable in one of the services. This task is challenging when the documentation is old and its authors are not easily identifiable or contactable.

Develop the final version of the problem diagram

The final version takes into account the opinions collected during the testing of the temporary diagram. It is an accurate account of the initial intentions logically reconstructed of the European Commission, taken from official documentation.

.

What are the preconditions for its use?

.

The need for human resources varies with the tasks to be achieved and the situation encountered: reconstruction of the problem diagram with the analysis of the situation displayed in the documentation; retrospective reconstruction of the problem diagram as it should have been established to support the relevance of the strategy or programme under assessment. 

Most of the graphic problems can be solved with software like MS PowerPoint. 

The evaluator can also benefit from specific software designed to assist decision-making, such as MS Visio, TeamUp-PCM for Windows, and Palisade PrecisionTree.

Figure 5 : the preconditions for its use
The time span Collect: 5 to 10 working days
Data analysis: 5 to 10 working days
Test: 0,5 to 5 working days
Human resources Multidisciplinary team of experienced evaluators, whose specialities should cover the thematic evaluation scope. Knowledge of the European Commission's strategies and programmes development procedures.
Knowledge of computer tools. Experience in the fields covered by the strategies and programmes
Specific knowledge of the country, sector and theme under consideration
Financial resources A budget less than €5,000 should be fixed for the problem diagram, objective diagram and decision diagram.

.

What are the advantages and limitations of the tool?

.

Figure 6 : The advantages and limitations of the tool
Advantages Presentation of the problems 
The diagram presents the various problems and their relationship with the core problem through a system of rows. It displays the causal logical links between them or, conversely, the poor logic of these links. 

Main problems and contextual problems 
In the analysis of the situation, the diagram distinguishes the problems relating to the activity's context from the problems to be solved by the strategy and the planning. As a consequence, its construction requires the highlighting of priorities of the development assistance, and explains why certain problems are considered as important features of the strategy while others are not. 

Definition of the objectives The problem diagram enables the evaluator to: 

  • Reveal the implicit objectives of the strategy (or the programme)
  • Check the validity of the objectives presented in the strategy and programming documentation

It contributes to the organisation of the evaluation around a crucial question, which should be systematically answered: To what extent have the objectives been achieved?

Limitations Quality of the analysis
The methodology for the analysis does not guarantee high quality of the data, for neither the methodology, nor the sources of information are usually mentioned in the strategic, political and programming papers. The evaluator must therefore systematically enlarge the assessment to include the sources of the analysis displayed in these documents. The methodology used, the nature of the main sources of information and the identity of the authors should also be noted. 

Selection of the main problem
The selection of the main problem, which is crucial for the construction of the diagram, is particularly challenging when the objectives of the activities are general and the whole range of the country's problems (or the region's) are to be considered. The documentation may show two main problems which lack links between them, or may appear insufficient for the determination of a single core problem. 

Knowledge of the situation in the country or the region
The evaluation team may lack a sufficient understanding of the country or the region under consideration to assess the relevance of the analysis undertaken, the determination and the logical setting of the main problem.

Author

FC
Former Capacity4dev Member
last update
7 December 2022

More actions