Ex-post Evaluation

1.1 Background of the project

In order to improve the testing capability on electrical and electronics appliances and enhance the industrial standardization and certification system, the Government of the Philippines (GOP) requested the Government of Japan (GOJ) for a technical cooperation in 1991. Based on the request, the GOJ, through JICA, extended to BPS-DTI a 4-year project-type technical cooperation titled, Industrial Standardization and Electrical Testing Project, from 1993 to 1997. With the success of the project, the GOP planned to enhance the testing functions of the BPS laboratory in examining and testing home electrical appliances. The testing techniques on this field however were found insufficient. Under this circumstance, the GOP again requested the GOJ for another technical cooperation in 1998. Based on this request, the GOJ, through JICA, extended to BPS-DTI another project-type technical cooperation titled, the Project on Electrical and Electronics Appliances Testing, from 1999 to 2003.

1.2 Project Overview

In order to improve the testing techniques on electrical and electronic appliances in the Philippines, the project transferred technologies to BPS counterpart staff through on-the-job training and seminars.

(1) Overall Goal

The safety of the electrical and electronic appliances in the market of the Philippines is improved.

(2) Project Purpose

The BPS laboratory is able to provide appropriate technical services on electrical and electronic appliances testing.

(3) Outputs

1) The machinery and equipment related to electrical and electronic appliances testing is provided, installed, operated and maintained properly
2) Testing of main electrical and electronic appliances is implemented by counterpart personnel
3) Seminars and training courses related to electrical and electronic appliances testing are implemented.

(4) Inputs

Japanese side:

Philippine side:

3.1 Summary of Evaluation Results

(1) Impact

By conducting on-the-job trainings and by providing BPS with testing equipment, the project had enabled the BPS laboratory staff to set testing standards for electrical and electronics home appliances such as airpots, microwave ovens, oven toasters, washing machines, rice cookers, refrigerators, flat irons, etc. It also enabled BPS staff to continuously conduct testing activities for such products and likewise conduct trainings to manufacturers, accredited testing laboratories and DTI product inspectors in the Regions. Through the standards set by BPS and its continued testing services, manufacturers adjusted and improved their safety design techniques in producing electrical and electronic home appliances. This resulted to a decrease in failure rates on tested similar items by 20 to 30% and consequently, an increase of certified products over the years, thus contributing to the improvement of safety measures on electrical and electronics appliances in the Philippine market. With the testing capability being improved through the project, BPS laboratory staff continues to contribute to the formulation of Philippine National Standards on electrical and electronics home appliances, such as the DTI Department Order No. 02, defining the responsibilities and liabilities of manufacturers and sellers of products under mandatory product certification. In accrediting private testing centers, BPS applies the guidelines and procedures set by the International Standards Organization (ISO 17025) such that all the 69 private testing laboratories comply with the international testing standards, thus further ensuring the safety of electrical and electronics home appliances in the Philippine market.

There was no negative impact produced by the project.

(2) Sustainability

The Project is technically sustainable. The DTI BPS management is capable of maintaining the benefits accrued as a result of achieving the purpose and overall goal because: management has prioritized the transfer of knowledge and skills between the trained counterparts and other staff; the knowledge and skills learned can be applied to a wide range of types and models of electrical and electronic appliances; the services provided by the BPS are more than acceptable to its clients; and the equipment is being used maximally.

The project is organizationally sustainable. The proactive response of the management to the situation has contributed to the organizational sustainability. As such, there has been no change in the organizational structure of BPS and the same staff still holds the same position.

The BPSTC is financially limited. Based on the Actual Budget and Income of the Center, there is a consistent deficit in their budget for the period 1994-2005. The budget includes personnel services, maintenance and operating expenses and capital/equipment outlay. Their income comes solely from the testing fees. The situation is critical considering that there was a huge gap between the income and the required budget. On the average, there is an annual deficit of PhP 6,613,000 in the budget of BPSTC covering the period 1994-2005.

3.2 Factors affecting Impact and Sustainability

The timing of the Project and the enforcement of mandatory policy for testing of manufacturers' products by the BPS were the two factors that promoted impact. The approval of PTTC in 1999 enabled the Philippines to apply safety standards to these appliances to the benefit of consumers. While the two factors promoting sustainability were management prioritization of the transfer of skills in and outside of the organization and availability of testing equipment. On the other hand, economic trends in the Philippines have hindered impact and sustainability. The budget deficit of the GOP has prevented the appropriation of the required budget for the BPS.

3.3 Conclusion

The overall goal of the Project is being achieved given the decrease in failure rate of items tested and an increase of certified electrical and electronic appliances in the market (as attested by the manufacturers and C/Ps). On the project purpose level, increased competencies for electrical or electronic testing to ensure safety and compliance with international standards contributed to the attainment of the overall goal. These factors are highly correlated with the realization of the project purpose. On the output level, the factors namely: availability of equipment for electrical and electronic testing and the use of operations manuals within the industry contributed to the attainment of the overall goal.

The Project likewise resulted in five significant effects that were not anticipated at project completion. It has influenced the development of national policies on the responsibilities and liabilities of manufacturers and sellers of products for mandatory testing. It has resulted in the development of national standards. It has increased the capability of the DTI to develop standards. It has increased the capability of the private sector for product testing. Lastly, it has contributed to the HRD, specifically technical vocational education and training.

The Project is technically sustainable. The DTI BPS management is capable of maintaining the benefits accrued as a result of achieving the purpose and overall goal because: the transfer of knowledge and skills between the trained counterparts and other staff has been prioritized by management; the knowledge and skills learned can be applied to a wide range of types and models of electrical and electronic appliances; the services provided by the BPS are more than acceptable to its clients; and the equipment use is being maximized.

The Project is organizationally sustainable. The proactive response of the management to the situation has contributed to the organizational sustainability. As such, there has been no change in the organizational structure of BPS and the same staff still holds the same position. The financial sustainability of the BPS is also limited. Based on the Actual Budget and Income of the BPSTC, there is a consistent deficit in their budget for the period 1994-2005. Likewise, there was a huge difference between the income and the required budget.

3.4 Recommendations

(1) Recommendations for the Implementing Agency.

Until at present, the BPS is operating with a very limited budget that effectively prevented the hiring of necessary number of testing engineers and the repair or purchase of new testing equipment. This budget limitation has been hindering the continued production of positive effects the project had started. In view of this, it is recommended that BPS has to come up with a long-term sustainability plan (on electrical and electronics home appliances testing), through which operations and maintenance (O&M) budget requirements every year are clarified and alternatives for revenue utilization are identified. Such alternatives may include the adoption of a new law by Philippine Congress authorizing the BPS to use its generated revenues and deposit the same through a thrust account system instead of remitting such incomes to the Bureau of Treasury. After the formulation of a well-thought Sustainability Plan, BPS has to intensify its advocacy activities not only at the DTI level but also at the Philippine Congress as well to ensure the passage of such law that may secure financial independence of BPS in the future.

(2) Recommendations for JICA

The Japan International Cooperation Agency should consider the lessons learned during planning and implementation of the project and apply them to future projects with the BPS to ensure success, impact generation, project sustainability. If still applicable, the lessons learned should also be applied to the on-going TCP Project with the BPS. In the formulation of future projects, JICA should also consider conducting continuous training to equip participants with required knowledge and skills. JICA should also consider including representatives from manufacturers and DTI regional and provincial offices staff as training participants for the smooth flow of the testing procedures.

3.5 Lessons Learned

(1) In project formulation process, it is important to establish baseline information of every project indicator being identified. In this project, base information about test failure rates and certified appliances had not been established before the project started.

(2) Beneficiary-oriented indicators are important indicators at the Project Purpose and Overall Goal. For example, capability-enhancing indicators at the Project Purpose level for this project could have included the frequency and quality of developing standards by BPS counterparts including the quantity and quality of trainings extended by BPS counterpart staff to manufacturers, private testing centers and DTI staff in the Regions. At the Overall Goal level, this project could have included the level of satisfaction of consumers on tested and certified appliances.s