Introduction

1. Definition and Objectives of JICA's Evaluation

JICA's evaluation is a tool for judging as objectively as possible the relevance and effectiveness of JICA's cooperation activities at four different stages during the project cycle; ex-ante, mid-term, terminal and ex-post. The primary objective of evaluation is to improve the effectiveness and efficiency of projects by using evaluation results for better planning and implementation. JICA also intends to gain public support and understanding by using them to ensure accountability. JICA has been focusing its effort to bolster its evaluation with the following three objectives.

 •  Using Evaluation Feedback as a Means for Project Operation and Management

By using them in decision-making process, JICA refers to evaluation results when formulating its aid strategies and JICA Country Programs. It also uses them when making decisions regarding project execution, selecting target projects, reviewing plans and determining the continuation or termination of a project.

 •  Enhancing the Learning Effects of the Personnel and Organizations Concerned for More Effective Project Implementation

Lessons learned from evaluation help enhance learning effects for those concerned in various occasions. For instance, the lessons from past projects serve as useful reference for JICA staffs and officials of recipient countries when they plan and implement similar projects. Also, the evaluation process itself contributes to expanding the knowledge and developing the capacities of the people involved, and thus serves as a "learning process".

 •  Disclosing Information Widely to Secure JICA's Accountability

Disclosing evaluation results to the public and explaining that JICA is fulfilling its responsibility for its undertakings is indispensable for winning public support and understanding. In order to ensure accountability to taxpayers, JICA needs to ensure adequate information disclosure.

2. Targeted Level of JICA's Evaluation

The Ministry of Foreign Affairs' "Report on Improvement of ODA Evaluation System", released in March 2000, classified ODA evaluation into three levels: policy-level, program-level and project-level. The report called for enhancement of policy- and program-level evaluation. JICA conducts program- and project-level evaluation (see Figure 1).

Figure 1. ODA System and JICA Evaluation

3. Types of JICA Evaluation

Project-level Evaluation

Project-level evaluation covers individual projects. This type of evaluation, conducted by JICA's departments responsible for project implementation and overseas offices, is intended to be reflected in planning and reviewing individual projects, making decisions as to continuation of projects and revising project plans, planning and executing other similar projects and ensuring accountability into operations. Project-level evaluation is classified into four types conducted at different stages during the project cycle: ex-ante evaluation, mid-term evaluation, terminal evaluation, and ex-post evaluation (see Table 1).

Program-level Evaluation

Program-level evaluation includes comprehensive evaluation applied to such groups of projects as those that share the same overall goal and development issues. It is also directed at a set of projects under a specific cooperation scheme (see Table 2). These evaluations are principally conducted by the Office of Evaluation in JICA at ex-post stages as Country-Program evaluation or thematic evaluation. The evaluation results are used mainly for improving JICA Country Program and finding and formulating new projects.

Evaluation by Evaluators

JICA's evaluation can be classified by evaluators as follows: "Evaluation by Third Party (external evaluation)", "Evaluation by JICA (internal evaluation)" and "Joint Evaluation" (see Table 3).

Table 1. Project-level Evaluation (by Stages During the Project Cycle)

Ex-ante Evaluation	Ex-ante evaluation is conducted when a project is requested by a recipient country. It first involves a study of the project to determine its necessity as well as its conformity with JICA's Country Program. Details of the project and its expected outputs are clarified. Then, the relevance of the project is comprehensively examined and evaluated. In ex-ante evaluation, evaluation indicators are set and they are used to measure the effect of the project in subsequent evaluation, from the mid-term evaluation to the ex-post evaluation.
Mid-term Evaluation	Mid-term evaluation is conducted at the mid-point of projects. This evaluation aims at examining the achievements and process of the project, focusing on efficiency and relevance among the Five Evaluation Criteria. Based upon its results, the original project plan may be revised or the operation structure strengthened if necessary.
Terminal Evaluation	Terminal evaluation is performed upon completion of a project, focusing on its efficiency, effectiveness, and sustainability. Based upon the results of the evaluation, JICA determines whether it is appropriate to complete the project or necessary to extend follow-up cooperation.
Ex-post Evaluation	Ex-post evaluation is conducted after a certain period has passed since the completion of a target project with emphasis on the effectiveness and sustainability of the project. This evaluation aims at deriving lessons and recommendations for the improvement of JICA Country Programs and for the planning and implementation of more effective and efficient projects.

Table 2. Program-level Evaluation (by Evaluation Targets)

Country- program Evaluation

This comprehensive evaluation examines the overall effects of JICA's cooperation on the development of a targeted country across projects. After clarifying and analyzing the overall effects of JICA's cooperation and difficulties faced, this evaluation derives lessons and recommendations for the improvement of the JICA Country Program of the country in question.

Thematic Evaluation

This evaluation looks at a number of projects, by focusing on specific sectors, issues (environment, poverty, gender, peace-building, etc.) or cooperation schemes (Japan Overseas Cooperation Volunteer Program, etc.). After clarifying and analyzing the overall effects and common impeding factors of JICA's projects of the issue in question, this evaluation derives lessons and recommendations for the implementation of future projects focusing on those themes. It also considers the effective approaches and methods to implement projects focusing on the specific theme.

Table 3. Type of Evaluation by Evaluators

Evaluation by Third Parties (External Evaluation)	In order to improve the quality and objectivity of its evaluation, JICA entrusts a certain portion of its evaluation studies to external third parties that were not involved in planning and implementation of the projects to be evaluated as well as those that have high expertise in the targeted fields for evaluation, such as universities, research institutions, academics, consultants, etc..
Evaluation by JICA (Internal Evaluation)	In order to derive lessons and recommendations that meet the actual conditions or needs of recipient countries, this evaluation is conducted mainly by JICA with the knowledge of systems and other surroundings of a project or an issue. JICA also promotes the review of such internal evaluation results by third parties (academics, journalists, NGOs, etc.) with expertise in development assistance and familiarity with JICA's undertakings to assure transparency and objectivity.
Joint Evaluation	This evaluation is conducted in collaboration with organizations in the target countries or with aid agencies of other donor countries. Joint evaluation with the partner countries is effective for sharing recognition with JICA on effects and issues about projects. It also contributes to leaning evaluation methods and improving the capacity of those countries in carrying out evaluation. This evaluation is effective in promoting mutual learning of evaluation methods and aid coordination.

4. JICA's Management Cycles and its Evaluation

In considering the function of evaluation in JICA's management cycle, it can be put in two different ways. JICA has a "small cycle" and a "large cycle" in its management as shown in Figure 2. The small cycle represents the management cycle of individual projects, where evaluation is used for the effective management of projects. In so doing, an integrated process starting with ex-ante evaluation is necessary in order to establish a solid foundation for project monitoring and evaluation. Also, communicating feedback from evaluation results in the cycle of each project is crucial to reviewing the project plan, improving the operating approach as well as to planning and executing similar projects in the future. The large cycle represents the management cycle at the program-level. Individual projects belong to a specific cooperation program designed to solve larger development issues. The results of evaluations of individual projects and cooperation programs supplement each other in enabling aid operations to be more effective. Program-level evaluations can be performed by country, issue or sector. The results are to be used in formulating guidelines for each issue or sector as well as making improvements in the implementation of individual projects.

Figure 2. Evaluation Types by Stages during the Project Cycle

Figure 3. Example of Logical Framework (PDM)

Narrative Summery	Objectively Verifiable Indicators	Measures of Verification	Important Assumptions
Overall Goal Indirect/long-term effects, impact on target society	Indicators to measure the achievement degree of overall goal	Information resources of indicators to the left	Assumptions necessary for the effects of the project to be sustainable
Project Purpose Direct benefits to the target groups or society	Indicators to measure the achievement degree of project purpose	Information resources of indicators to the left	External factors that must be satisfied to achieve the overall goal, but that retain uncertainty as to whether they would be satisfied
Outputs Services and results produced by activities	Indicators to measure the achievement degree of outputs	Information resources of indicators to the left	External factors that must be satisfied to achieve the project purpose, but that retain uncertainty as to whether they would be satisfied
Activities Activities to realize outputs	Inputs Resources required for carrying out activities (human resources, funds, facilities and equipment, etc.).		External factors that must be satisfied to achieve the outputs, but that retain uncertainty as to whether they would be satisfied
			Prerequisite conditions must be met before the beginning the project