Ex-post Evaluation

I. Outline of the Project

• Country : Tanzania
• Project title : Maternal and Child Health Project
• Issue/Sector : Health/Medical
• Cooperation scheme : Project
• Section in charge : Human Development Dept. Group 4 (Health 2), MCH Team
• Total Cost : 292 million yen (local costs and equipment supply only)
• Period of Cooperation : December 1, 1994 – November 30, 2001 (including a follow-up term December 1, 1999 – November 30, 2001)
• Partner Country's Implementing Organization: Muhimbili Medical Centre (presently Muhimbili National Hospital and Muhimbili University College of Health Science), and Tanga region
• Supporting Organizations in Japan : Mie University
• Related Cooperation : Grant Aid: Medical Equipment Supply for National Reference Hospitals, 1994

1-1 Background of the Project

In 1993, the United Republic of Tanzania was promoting the enhancement of social service sector like Education, Health and Water resource under the Rolling Plan and Forward Budget for Tanzania 1993/94 - 1995/96. Nevertheless, due to the high economic depression, the Rolling Plan faced some difficulties even in the health sector. It caused significant shortage of medical staff and equipment and malfunction of the medical system, and the situation of primary health care like maternal and child health got worse. In 1994, being concerned with this situation, the Government of Tanzania requested the Government of Japan to provide a technical cooperation to improve the situation of maternal and child health at national level and rural level.

1-2 Project Overview

(1) Super goal

Maternal and infant mortality rates, and in Tanzania are decreased.

(2) Overall Goals

(1) Maternal and infant mortality rates in the pilot areas (Tanga and Korogwe Division) are decreased.

(2) EPI diseases are reduced and Polio should be eradicated in the United Republic of Tanzania.

(3) Paediatric services at Muhimbili Medical Centre (MMC, former Muhimbili National Hospital, MNH) are improved.

(3) Project Purposes

(1) Maternal and child health services in Tanga and Korogwe Districts are improved.

(2) Virology diagnostic capabilities of Expanded Programme for Immunization (EPI) diseases at MMC are strengthened.

(3) Laboratory Based Medicine by utilizing accurate lab-data for diagnosis is established.

(4) Outputs

(1-1) Implementation system for activities in Tanga is established.

(1-2) The present situation of health services is grasped.

(1-3) Traditional birth attendants can deliver babies safely.

(1-4) Village health workers can educate their villagers on health care.

(1-5) The competence of health service staff is strengthened.

(1-6) The function of health centres is strengthened.

(1-7) EPI activities in Tanga district are supported.

(2-1) Diagnostic capabilities of EPI diseases in MMC Virology Dept. are improved.

(2-2) Testing equipment to inspect viruses is installed in Virology Dept. of MMC and functions.

(3-1) The implementation system for activities in MMC is established.

(3-2) Basic medical information is firmly grasped.

(3-3) Diagnostic capabilities of medical staff in Paediatric Dept. of MMC is improved.

(3-4) Facilities and machinery in Paediatric Dept. of MMC are properly equipped.

(3-5) Clinical examination capabilities of laboratory technicians in Paediatric Dept. of MMC are improved.

(3-6) A laboratory in Paediatric Dept. of MMC is kept in good conditions.

(3-7) Management capabilities of a chief laboratory technician is improved.

(4) Inputs (as of the Project’s termination)

Japanese side:

• Long-term Expert
• 17pp

• Short-term Expert
• 24pp

• Trainees received
• 23pp

• Equipment
• 208,382,000 Yen

• Local cost
• 83,452,000 Yen

• Others

• Total
• 291,834,000 Yen

Tanzanian Side:

• Counterpart
• 27pp

II. Evaluation Team

Members of Evaluation Team:

Dr. Calista Simbakalia, HealthScope Tanzania Ltd.
Dr. Tengio F. Urrio, HealthScope Tanzania Ltd.

Period of Evaluation:

9th November, 2005 – 16th February, 2006

Type of Evaluation:

Ex-post evaluation study

III. Results of Evaluation

The Project is composed of three components, Virology Laboratory at Muhimbili University College of Health Science (MUCHS), Traditional Birth Attendants (TBAs) / Village Health Workers (VHWs) in Tanga, and Paediatrics at MNH as mentioned above. Each item below is discussed in each of these components.

3-1 Summary of Evaluation Results

(1) Impact

- Super goal
Statistics in Tanzania is as follow:

(*1) World Bank Database
(*2) Demographic Health Survey in Tanzania
(*3) Maternal and Child Health Survey in Tanzania
There is no morbidity data. The Infant Mortality Rate (IMR) and Under 5 Mortality Rate (U5MR) were decreased during the project term and after the project. However, the Maternal Mortality Rate (MMR) was not decreased even while the number of data was very a few.
Some of super goal of the Project was achieved.

- TBAs/VHWs in Tanga

(a) Achievement of overall goals

During the period of project implementation the MMR there fell from 392/100,000 in 1999 to 350/1000 in 2001. Thus it started rising again for the first 2 years after the end of the project before it started to decrease again. In 2004 it was still decreasing but has not reached the 2001 figure of 350/100,000

Korogwe also shows a decreasing trend 2002-2004. During the period of project implementation the MMR decreased from 254/100,000 in 1999 to 139/1000 in 2001. Thus it continued to decrease one year after the project before it started to rise and decreased again. In 2004 it was still decreasing but has not reached the 2001 figure of 139/100,000.
The fall of MMR a one of the overall goals was achieved somehow.

(b) Other positive effects

New TBA utilization guidelines came in place in 2001 after project implementation and evaluation. In these guidelines emphasis was put on Skilled Attendants (trained service providers) at delivery instead of TBAs. After the policy changed, the role of TBAs has therefore changed from conducting deliveries to early referral of pregnant women to health facilities. Service providers welcome the TBAs into the health facilities when they escort women and encourage them to stay with the women till they deliver or are referred to the next level of care. The roles of TBAs got limited.

Nevertheless, TBAs have continued to submit the reports of deliveries to the health facilities. There have been still some women who like TBAs to attend delivery at home. The health facilities have continued to encourage them to report every month. TBAs reported that individual women have requested them to deliver them in the village rather than go to the health facilities. Scarcity of health workers has necessitated TBAs to have increased work load for delivering women. Health workers get transferred and others go on training for upgrading. As a result, TBAs increasingly deliver women at home.

The trained TBAs are in the community and despite the change in policy and decrease utilization of TBAs in deliveries they are being used to deal with other community problems: Vitamin A distribution, Non obstetrical services, Outreach activities, National Immunization Days (NIDs), Malaria prevention and nutrition, and Post natal services.

The Local Government Authorities (LGAs) have not drawn out transitional strategies to move forward the policy. One of the strategies would be the gradual phase out of the TBA and increasingly Skilled Attendants.

Some health facilities were built or renovated through the Project. Community participation in health and in setting up health facilities has improved since the project ended and involved not only health but also the other sectors notably the education sector.

- Virology Laboratory:

(a) Achievement of overall goals

There has not been reported of any polio patients in Tanzania since 1996.
WHO Afro was about to declare in 2004 that Tanzania would be a polio free country.
The overall goal is almost achieved.

(b) Other positive effects

A great impact is shown on the surveillance of measles and rubella. Whenever there is an outbreak of measles / rubella specimen are sent to the Virology Laboratory for serological investigations in order to confirm the diagnosis and thereafter appropriate treatment and investigation are undertaken based on accurate diagnosis.

In addition, serological investigations that test accuracy of HIV tests are also being performed at Virology Laboratory. This test is very useful because it assist to confirm the accuracy of tests performed elsewhere and therefore remove the doubt of false positives and negatives in the diagnosis of HIV.

The examination of the requests and results were that from 2000 to 2005 the following number of measles and rubella investigations were done 488(2000), 415(2001), 177(2002), 641(2003), 712 (2004) and 552(2005). All the requests are for measles investigations. Rubella investigations are performed only when the results of measles investigation are negative.

- Paediatrics

(a) Achievement of overall goals

A JICA Expert of a paediatrician had worked for the Paediatric Dept. of MNH after the Project in order to reduce the mortality rate of in-patients in the wards of the department. Therefore, it is difficult to measure the impact of the Project for reduction of this mortality rate.

(b) Other positive effects

Establishment of the Specialized Paediatric Laboratory (SPL) has developed capacity in performing advanced tests in various areas. Test results from the laboratory have contributed to the improvement in Evidence Based Medicine.

The laboratory also improved capabilities of medical staff (doctors, nurses, technicians and medical students) on laboratory based medical skills. The staff members of SPL trained during the project period have also continued to train Laboratory Technicians and Medical Students. To date, 250 medical students have been trained in Biochemistry, Microbiology, and Hematology. In addition 90 laboratory technicians who specialize in Hematology, Biochemistry and Microbiology have been trained at the SPL.

The SPL has been recognized by researchers as a Laboratory where research can be undertaken.

Due to its reliable equipment, water electricity and well trained staff the SPL has become a centre of Excellency for conducting Laboratory Based Research.

Currently, researchers are using SPL to study the followings: Metabolic syndromes, Sickle cell diseases, and Psychiatric, Traditional medicine and Pharmacology departments are conducting research in the SPL.

SPL assisted Central Pathology Laboratory (CPL) during its major renovation that took place from July 2004 to August 2005. During that period, SPL performed mainly 7700 Biochemistry tests and 1,281 Hematology tests. This performance was a great assistance to CPL that otherwise it would have been very difficult for CPL to handle these tests during renovation.

(2) Sustainability

- TBAs/VHWs in Tanga

(a) Technical aspects

Due to the change of the policy, there has been change in the support being given to TBAs. The supervisory system that had been set up has been weakened.

After the project ended in 2001, the TBAs have submitted reports to the health facilities individually every month. They are encouraged to bring in reports even while they do not deliver. Thus the system of monitoring deliveries of TBAs has been sustained. The TBAs have continued to maintain contact with the health facility as they refer and escort women to the health facility.

TBAs are still active and doing deliveries in their villages. An example of an active group was seen at Kerenge ward. There is a total population of 6,485 and the number of deliveries by TBAs has been increasing from 75 in 2002, 90 in 2003 and 100 in 2004 showing that the TBAs are gaining popularity. In Korogwe District almost 20% of all deliveries are done by TBAs.

(b) Financial aspects

This activity had not been incorporated into the Council Comprehensive Health Plan (CCHP). The supervision meetings are not being held due to not being budgeted for in the CCHP.

However, re-sensitization seminars were conducted in Tanga in the year 2002 and 2005. The cost of these seminars was funded by the Tanga City Council and co-funded by a Japan Overseas Cooperation Volunteer. In Korogwe the Council Health Management Team organized and funded a one day seminar in 2003 with the objective of discussing with the TBAs the new guidelines on utilization of TBAs and the responsibilities of the TBAs in this guideline

Tanga City Council and Korogwe District Council allocate a budget every year for renovation of health facilities which were built or renovated through the Project. All such health facilities were found staffed and operational during the post evaluation. This factor has increased ANC accessibility for pregnant women and therefore improved Maternal and Child Health. The community leaders appreciated the effects from renovation of health facilities.

Equipments that were provided during the project were used well but necessary to be replaced due to time decay. Even the bicycles which were supplied to the TBAs assist them in visiting their clients in the villages need spares and some require replacement.

After the end of the project, the revolving fund system that used to replace the expendables in the TBA kits is no longer in place in some of the facilities. In Korogwe for example only one of the four facilities visited continued to maintain the revolving fund. In the Antenatal Clinic (ANC), women are asked to buy and keep the supplies at home in preparing for delivery. The TBAs encourage them to do so.

c) Organizational aspects

At the end of the project, the LGAs had not taken over this activity. Staff members of LGAs think that supporting of TBAs is supporting deliveries by TBAs and thus working contrary to the policy.

- The Situation of VHWs

- Organizational aspects

In Tanga they started with 96 VHWs and now they have 37 after 59 had dropped out. These VHWs in Korogwe unlike in Tanga have been retained. Where they have dropped out by migration or getting gainful employment, the villages selected new VHWs for being trained.

They are a link between the community and the health facility; they encourage the villages to keep the environment clean. They visit the health facilities to collect statistics about diseases and deaths and send these to the community.

However the VHWs programme has faced the same problem that has been faced by the programme in other parts of the country where the programme has been run on voluntary basis. There has been a high drop out rate like Tanga municipality.

The VHWs leave the voluntary work and seek employment opportunities in the private sector.

- Virology Laboratory

(a) Technical aspects

The laboratory is kept clean and does not need any renovation for the time being. These aspects of capacity building have facilitated the sustainability of the Virology laboratory after the project period.

(b) Financial aspects

The Government of Tanzania is still supporting and has an intention to continue to support the Virology Laboratory though MUCHS in terms of provision of recurrent commodities like reagents.

(c) Organizational aspects

All the four (4) of staff members trained in Japan still remain. 95% of the equipments are still in good condition.

- Paediatrics

(a) Technical aspects

Even after the project ended, the quality and quantity of services of the SPL have increased because the staff members are skilled well and they work in a conducive environment where modern equipments are available. The equipments facilitate the trained laboratory technicians to produce reliable results thus leading to efficiency in diagnosing and managing diseases.

(b) Financial aspects

The revolving-fund system of SPL established during the Project period is still functioning. Even though the revenues from patients are low, most of who are under five years children exempted of fees, the SPL staff came with innovative ideas of working with researchers. By being collaborate with SPL in a study, the research organization has to pay a fee to SPL. This is an Income Generating Activity that has assisted SPL to increase its revenue.

The Government of Tanzania has continued to support the SPL through MNH. The SPL Manager usually requests funds from MNH and funds are allocated as per the request without having failed to get funds from MNH upon request. MNH also allocated funds for maintenance and repair of equipment and it has been outsourced and is being performed by engineers from University of Dar es Salaam and Roche Engineers from South Africa.

(c) Organizational aspects

The SPL is managed well to maintain by one of the Doctors trained in Japan for managerial skills during the project period with assistance of the laboratory technician in charge trained in Japan for biochemistry.

Collaboration with CPL and other departments is being done through consultative meetings and through technical assistance.

3-2 Factors that have promoted project

(1) Impact

- TBAs/VHWs in Tanga

Rural Communities still demand TBAs rather than health facilities and TBAs use what they learned through the Project.

The villages participation in providing of labour and other contribution in building health facilities have stimulated the spirit of community participation.

- Virology laboratory

All of staff members, trained in Japan during the project period, have still worked for the laboratory. Appropriate machinery and equipments were provided and are used well.
The government treats crucially the laboratory as a reference laboratory.

- Paediatrics

All of seven staff members trained in Japan during the project period have worked for the SPL.

The staff has kept elaborating the capacity of SPL by themselves even after the Project.

The doctors in the Paediatircs also rely on the examination results put out by the SPL.

(2) Sustainability

- TBAs/VHWs in Tanga

Rural Communities still demand TBAs rather than health facilities. The health providers in health facilities like them to submit monthly reports and the reports have been continued to submit.

The villages participation in providing of labour and other contribution in building health facilities has stimulated the spirit of community participation.

- Virology laboratory

All of staff members, trained in Japan during the project period, remain.

The provided machinery and equipments are maintained well.

The government supports it with provision of recurrent commodities.

- Paediatrics

All of seven staff members, trained in Japan during the project period, remain.

The provided machinery and equipments are used carefully.

The revolving fund system was established.

3-3. Factors that have hindered the Project

(1) Impact

- TBSs/VHWs in Tanga

New TBA utilization guidelines came in place in 2001 after project implementation and evaluation. In these guidelines emphasis was put on Skilled Attendants (trained service providers) at Delivery instead of TBAs. The Ministry of Health (MoH) guidelines are in congruent with WHO guidelines on utilization of TBA. They are now allowed to deliver only in emergency situations.

The role of TBAs are limited solely to encouragement of delivery at a health facility and TBAs lost opportunity to use what they learned from the Project.

- Virology Laboratory

WHO has changed the plan of Polio reference laboratory network in Africa to spell out Tanzania from countries which have a Polio reference laboratory.

- Paediatrics

The management of Muhimbili National Hospital does not prioritize the Paediatrics or agree that the Paediatrics had its own laboratory now. However, the dialogue among MoH, MNH and the Paediatrics is continued to solve this issue.

(2) Sustainability

- TBSs/VHWs in Tanga

Due to the policy change mentioned above, the local government authorities in Tanga region quitted to support activities of TBAs. It caused to phase out the revolving system for TBA Kit. Thus the TBAs activities undertaken by JICA and the Municipality Medical Officer of Health in Tanga and the Korogwe District Council were not incorporated into the planning and budgeting systems of the LGAs when the project ended.

3-4. Conclusions

The Evidence Based Medicine is well-established and provided to a number of child patients in the Paediatrics of Muhimbili National Hospital. The Virology Laboratory keeps has sufficient capability for serological examination and work as the national reference laboratory for measles and the centre of excellency for research and academic purposes. Furthermore, both organizations are operated by their own staff and supported strongly by the government.

On the other hand, the policy change for TBAs spoils the output of the component of the Project while TBAs are still active, motivated to renew their knowledge and skills and dependent of rural community.

3-5. Recommendations

- TBAs/VHWs in Tanga

The activities of TBAs in Tanga have been depressed due to losing supports from the government. The policy for TBAs should be reviewed to utilize TBAs for many aspects for improvement for maternal health at community level.

- Virology Laboratory

The Government of Tanzania and the management of Muhimbli University College of Health Science are strongly recommended to maintain financial support to this laboratory as the national reference laboratory to keep the same capability as of now. Furthermore, a financial amount should be prepared for replacement of decayed equipments in the laboratory. Recruitment for strengthening and/or taking over of the present personnel of the laboratory is also crucial.

- Paediatrics

The management of MNH should pay high consideration to the Paediatrics in order to contribute for reduction of the IMR and U5MR in this country as one of roles of responsibility of the top referral hospital in this country. Therefore, the hospital will allocate so sufficient human and financial resource as to maintain running of SPL and realizing of Evidence Based Medicine.

3-6. Lessons Learnt

To achieve the Goal 5 of the Millennium Development Goals in this country, it is recommended that JICA and the MOH develop and implement a comprehensive Reproductive and Child Health programme that covers a larger population where impact can be measured: e.g. Tanga and Morogoro Regions.

Projects for maternal health at community level should be designed in such a way that the implementation follows a continuum of care approach; i.e. all the community, including not only women but men and the youth, should be fully involved in prevention of maternal mortality and those people should be linked to a functional health facility (functional health facility has Skilled Attendants, appropriate and basic equipment to deal with Emergency Obstetric Care).

From the experience of the policy change after the end of the Project, a government should have a transitional strategy so as to let the outputs of the Project remain.

Adequate training to counterparts and preparation of the working circumstance to use the acquired knowledge and skills are necessary. Furthermore, training to counterparts for maintenance of working circumstance, management of facilities and equipments, will develop sustainability. The revolving fund system of SPL is a good example.

Support from the government is required. Virology laboratory and SPL cannot sustain if losing supports from the government. TBAs in Tanga is another example.

During implementation of a project, a government and funding partners should develop a mechanism of financial sustainability. The government will gradually take over the funding activities for the project activities in order to ensure that such funding activities become parts of the budgeting and planning systems of the government or local government authorities.

3-7. Follow-up Situation

In order to consolidate the capacity of the SPL and to strengthen relationship between the SPL and medical doctors in the Paediatrics for the enhancement of Evidence Based Medicine, one Japanese expert for Paediatrics was dispatched for May 2002 to June 2005.