Ex-post Evaluation

1-1. Background of the Project

JICA assisted the implementation of Turkey's EPI policy by supporting the Biological Control and Research Laboratories of the Refik Saydam Hygiene Centre Presidency (hereinafter referred to as "RSHCP") from 1993 to 1996 with the project-type technical cooperation scheme, in the field of vaccine preventable infectious diseases. The project that was conducted between 1993-1996 aimed at improving laboratory techniques of biological control of vaccines and to establish a National Control Laboratory at international level.

With the success of this cooperation, the government of the Republic of Turkey requested JICA's further cooperation to get technical support purposes of monitoring the immunization status of people through epidemiological surveillance and other EPI-related laboratory techniques. With this request, JICA agreed to start a different project-type technical cooperation "Infectious Diseases Control Project" from October 1997 to 2002.

1-2. Project Overview

(1) Overall Goal

To control EPI related infectious diseases.

(2) Project Purpose

To establish a laboratory supported epidemiological surveillance system.

(3) Outputs

a) Laboratory techniques on EPI related infectious diseases are strengthened.
b) Management and technical skill for epidemiological surveillance on DPT, polio, measles, and hepatitis B are acquired.
c) Technical collaboration between RSHCP and Primary Health Care General Directorate is established
d) A serum-bank is established

(4) Inputs (as of the Project's termination)

Japanese side:

Turkish side:

3-1. Summary of Evaluation Results

(1) Impact

1) Achievement of Overall Goal

The project goal of "EPI related infectious diseases are controlled" is a long-term goal, which the project is believed to contribute by demonstrating a surveillance system in 3 cities, as the means of controlling infectious diseases. It is evident that the project will reach its goals in the long-term, if supporting projects and activities are conducted and sufficient resources are allocated to disseminate the technology and extend the pilot experience nation-wide. However, it is observed that health directorates in the pilot cities could comment on the decreased mortality and morbidity, and could relate this with the Project.

2) Other Positive Effects

The project accomplished a new vision in the Ministry of Health for surveillance of infectious diseases, and established the basis for updating of policies related with surveillance. The works conducted throughout the project brought about a model and helped identification of problems and constraints in the existing system that has been applied for many years.

Apart from the high technology that the project brought in together with the installed equipment, the counterpart staff at all project parties became highly conscious and motivated about their work and further development in their careers. They have also established a more organized work system. All contact persons involved in evaluation stated that they gained extensive knowledge from the project.

Currently, laboratory data related with infectious diseases are used in surveillance for:

• early detection of epidemics
• verification and classification of epidemics
• identification of sensitivity antibiotics and their classification
• analysis of micro-organisms with respect to their types

However, laboratory supported surveillance is fairly new in the country, therefore time is needed in order that sufficient data is collected, analysed and interpreted. This also requires that consciousness should be built upon all practicing medical staff. It is required that further trainings and consciousness building studies are continued at organizational level.

Ongoing cooperation between RSHCP and PHC includes vaccination services, identification of infectious diseases, laboratory based surveillance, infectious disease research. Besides, the two organizational units work together on legislation of surveillance, organize and conduct training programmes and participation in international seminars.

One of the important results of the project is that national project partners have conceived that they have to cooperate for effective surveillance, thereby; they are working in teams in surveillance studies, as an indication of their efforts for removing barriers in working together.

3) Other Negative Effects

No negative effects of the project are found.

(2) Sustainability

1) Technical Aspects

Published scientific bulletins and papers are indications of positive project impacts and also of the progress in the sense that efforts in surveillance are continued.

Problems with equipment maintenance have been the major factor that hinders project sustainability. Calibration and supply of consumable materials are delayed frequently, mainly due to bureaucratic reasons that can be overcome with organizational arrangements.

Two significant steps that imply project sustainability are the notification system which is already in place and the EU funded project on infectious disease control, which are very much depend on the results and outcomes of the JICA Project. The EU Project on infectious diseases aims at accomplishing necessary legal arrangements, training sufficient health personnel through an extensive and effective training programme and defining duties and responsibilities of all stakeholders in the system.

The EPI unit is established as a result of the project plays a key role for the sustainability of the technology transfer. The EPI unit is planned to be restructured in organizational terms so that the unit could serve more actively both in the Infectious Diseases Control Department and also nation-wide.

Despite that it is not a direct result of the project, establishment of the notification system makes big use of the project results. During the project, cases were described for only three diseases for which field guidelines were prepared and are still in use.

Although some contact points has some hesitations about the dissemination potential of the Project, the systematic in various procedures can be disseminated and replicated. Moreover, the regional laboratories are extending their surveillance area to include other cities in their geographical boundaries.

2) Organizational Aspects

The Project accomplished consciousness about the significance of improving the surveillance of infectious diseases, and integration of laboratories in the surveillance system. The Project also enabled discussion of different views about including laboratories within the overall system and forged understanding the importance of proof based medicine. Such conceptual understanding and changes in cognition of organizations are reflected in various steps of legislation change. The EU Project is developed as an extension of the project and anticipated to lead to necessary legal changes.

Personnel stability is a factor that affects project sustainability but it would be wrong to expect to control. Therefore, it is rather a risk that limits project sustainability. In Turkey, particularly the managerial level has a continuous change of personnel. This has been a factor for the Project, causing loss of time and resources

It is observed that the Project staff has attained a motivation for continuous self-improvement in technical terms, which is another indication of project sustainability.

3) Financial Aspects

Besides the budget allocation of the state, RSHCP puts efforts for raising funds through development of new projects with international funds. The EU Project will be providing 3 million € for the first phase, and about 6 million € is expected for the second phase.

3-2. Factors that have promoted the project

1) Impact

Overall project context and particularly the trainings related with the use of new technology was very motivating for the counterpart staff.

2) Sustainability

The project also enhanced preparation of research papers published in scientific media, which was another means of motivation for sustained interest of the counterpart staff.

3-3. Factors that have inhibited project

1) Impact

As compared to the highly motivated inputs of RSHCP in the project, it is clear that roles of PHC was not well-defined and cooperation among project parties was not sufficiently planned from the beginning. Along with the poor coordination between the national partners, it is also conceived that flow of knowledge and information between project units was rather weak. Today, efforts are concentrated by all parties on the compensation of the gaps in coordination.

2) Sustainability

Instability of personnel within the national partners is another inhibiting factor. Loosing trained staff is a particular reason that affected project efficiency and effectiveness. This has also caused inconsistency in common understandings and approaches among the managerial and technical staff.

Another factor that inhibits project sustainability is related with the problems with equipment maintenance. Calibration and supply of consumable materials are delayed frequently, mainly due to bureaucratic reasons that can be overcome with organizational arrangements.

3-4. Conclusions

It can be concluded that the project purpose of establishing a surveillance system as a tool for contributing to the project goal is achieved. Improvement of the system particularly in organizational and legislative aspects is planned to be achieved with a EU funded project developed as based on the results of the Project.

3-5. Recommendations

Based on the current situation of new activities along with problems confronted, recommendations for project sustainability are made as follows:

• Equipment and material plans should be made for laboratories in order to avoid idle equipment.
• Necessary financial procedures should be defined in order to ensure regular maintenance of project equipment.
• Mechanisms should be developed for better information flow between RSHCP and PHC.
• Coordination between Provincial Directorates of Health and regional laboratories of RSHCP in the pilot cities, PHC and RSHCP should be organized toward more effective and efficient surveillance.
• JICA support can be provided by dispatching a Japanese expert for an overview of the problems particularly related to the use of project equipment, making necessary recommendations for effective and practical solutions.
• A JICA expert can support a coordination team to identify bottlenecks in the existing surveillance system in the pilot cities and suggest practical, concrete and permanent solutions.

3-6. Lessons learnt

Together with the suggestions from the counterparts involved in the evaluation, the Consultants recommendations for similar projects are defined as follows:

• Structures of project partner organizations should be well-analyzed before project planning.
• Participatory and interactive mechanisms that incorporate project staff in project planning and preparation could attain a higher level of concensus among partners.
• Job descriptions should be made very clearly particularly for the counterpart staff involved in the project.
• Risks that can inhibit project sustainability should be defined and necessary measures should be proposed.
• The managerial level should be supplied with continuous information in projects.
• An effective (practically internet-based) media should be established for information sharing throughout project conduct period, which can also be used for attaining project sustainability after the project is completed.
• Besides technical staff, support staff should also be actively included in the Project in order that they can keep up with the technological development.
• Project schedule should be prepared in a clear way and not changed as much as possible.
• Progress reports should be prepared in cooperation among Project partners.
• Complementary steps and follow-up activities should be defined for reaching the project goal.