Expert panel

SUMMARY

Why is this tool used in evaluation?

An expert panel usually comprises independent specialists, recognised in at least one of the fields addressed by the programme under evaluation. The panel specialists arrive at conclusions and recommendations through consensus. Depending on the project proposal, these recommendations deal with the implementation or the impact of a programme, or part of it.

The expert panel is specifically appointed for the evaluation, and in conformity with standard procedures. The panel holds meetings and provides conclusions and recommendations in accordance with a precise and replicable working plan.

The use of an expert panel in country/region evaluations can be helpful in several situations, such as:

• Studying very specific fields requiring a high level of competence (such as research support and assistance for high technologies)
• Studying subjects for which other tools are difficult to implement at a reasonable cost
• Carrying out limited-scope evaluations (such as assistance to small countries)
• Assisting the evaluators in their conclusions on a subject in complex evaluations
• Providing assistance in the drafting of final conclusions relating to the possible impacts of a programme in ex ante evaluations

How is a panel expert carried out?

What criteria should be used to appoint the panel?

The pre-requisite for the expert's selection is his/her professional experience. He/She should have specialised in the field under evaluation, and be recognised and respected by his/her peers.

Experts must be independent of the programme under evaluation, because they should not be judge and judged. Independence regarding the programme under evaluation is very important since the expert cannot be the judge and the judged.

Figure 1: Core criteria of the expert panel's composition

The ability to work in a group, listen to other experts and be open-minded is an essential criterion. Otherwise, working conditions may quickly turn out to be unmanageable, which would impede the panel process.

How are experts selected?

In a straightforward selection, the evaluation managers have access to a list of acknowledged experts in specific fields, and limit their selection process to ensuring the expert's independence regarding the programme under evaluation.

In gradual selections, preferred profiles of experts are developed with respect to the topics under scrutiny in the evaluation.

Elements to be taken into account in developing the panel profile are as follows:

• Project scope
• Degree of controversy (do the problems to be addressed have alternative resolutions which are controversial?)
• Available data
• Uncertainties (will the panel's conclusions discuss the uncertainties?)
• Number of required disciplines

What are the procedures for the management of the panel?

There is no unique working process, and the expert panel should be encouraged to plan and implement its own workplan. Experts can focus their work on documentation and sessions, or broaden it to include meetings with project managers, field visits, implementation of surveys, etc.

The first panel session must result in the experts having a full understanding of their role in the evaluation.

During this session, the applicable methodology for the management of the panel's work must be discussed and validated. The discussion usually focuses on:

• The panel's organisation and the role of each member
• The type of investigation, the data collection methodology, and details of each panellist's task (such as field visits)
• The intervention work programme, the organisation of future sessions and their contents

The next sessions (ranging from 3 to 5) will be directly linked to the panel's work. They will systematically deal with:

• The work carried out since the previous session
• Findings from investigations which are completed or in process
• Problems encountered
• Progress in editing the various documents, the review process, and quality control over these documents
• The tasks to be achieved before the next session and its envisaged content
• Confidentiality of the panel's debates and intermediate findings is another rule impacting on the panel's working arrangements.

What is the role of the panel chairman?

The panel chairman plays a crucial role. He/She guides the study panel, proposes the working arrangements, records findings, encourages contributions, facilitates debates and is the chief spokesperson for the panel. The quality of the working arrangements often depends on the chairman's leadership.

Figure 2: The various roles of the panel chairman

The chairman as Panel facilitator schedules the work of the panel and its production, and steers the panel's progress toward consensus.

The chairman as Report Architect and Integrator ensures a critical overview to the panellists' outputs, so as to improve the debate.

The chairman as Project Manager ensures that the available resources are sufficient and properly employed throughout the study. He/She ensures that the panel's sessions have been properly prepared by the technical writer.

The chairman as Spokesperson represents the panel in various bodies (such as monitoring committee and meetings with the commissioning agency and the press).

How does the expert panel report on its work?

The report, which supports the experts' contribution to the evaluation, is the only output from the panel which is made available to the commissioning agency. The report's structure should include: an executive summary, the mission's terms of reference, the composition of the panel, the evidence gathered and reviewed, the analyses carried out, The conclusion of the experts in the context of the report's consensus findings.

What are the preconditions for its use?

What are the advantages and limitations of the tool?