Ex-post Evaluation

1.1 Background of the Project

Industrial accidents and occupational diseases in Thailand had been increased due to a rapid development of industrialization. One of the objectives in the 7th National Economic and Social Development Plan of Thailand (1992-1996) was to protect workers from industrial accidents and occupational diseases. Therefore, in 1994 the Thai government requested the Japanese government to extend a technical cooperation to strengthen the functions of NICE, which is in charge of giving guidance, conducting researches and studies, providing training, etc. in order to establish and promote occupational safety and health standards. In response to the request the Japanese government, through the Japan International Cooperation Agency (JICA) as an implementing agency, conducted the Preliminary Study and long-term Study in 1996 and Implementation Study in 1997. The Project was then commenced in June 1997. This ex-post study is conducted for the follow up after the project has terminated 3 years ago.

1.2 Project Overview

Japanese side has provided technical transfer by dispatching short and long term Experts, provision of equipment and machinery, and counterparts training in Japan. The Thai side has constructed the new building and provided C/Ps to work with the Project. Having workers in Thailand as Project's target group, main project activities include the preparation of guidelines and manuals for safety measures, the development of training curriculums on OSH and textbooks, and the improvement of public relations with regard to safety and health to be produced by NICE.

(1) Overall Goal

"Workers are protected from industrial accidents and occupational diseases."
(Its objectively verifiable indicator is that the number of occupational accidents and diseases should be decreased to the level of less than 26 per 1,000 workers by 2002)

(2) Project Purpose

Functions of NICE are strengthened.

(3) Outputs

a) Technical capability of NICE is improved.
b) Training courses for safety and health inspectors are improved.
c) Training courses for persons in charge of safety and health management in enterprises are
improved.
d) Public relations by NICE with regard to safety and health are improved.

(4) Inputs

Japanese side:

Thai Side:

3.1 Summary of Ex-Post Evaluation Results

(1) Impact

Overall Goal: At Project completion the overall goal has not yet been achieved but the number of injuries and diseases declined significantly from 36.24 per 1,000 workers in 1998 to 29.20 in 2002. Even until 2004 the total number of injuries, 29,18 per 1,000 workers, has not reached the set target. However, the rates of injuries occurrence indicate a trend of constant decrease every year.

Project Purpose: The progress is derived from (i) the strengthened functions of NICE, by which 283,039 SH Inspectors have been trained since the Project began until October 2005, (ii) the disseminated research results and (iii) public relations in various forms.

Project Outputs: (i) Ideas and practical experiences in preparation of manuals/guidelines derived from the Project enabled NICE to continuously develop another 21 guidelines/manuals and 6 research reports (excluded Safety Newsletter which was regularly published). New publications and the ones produced during the Project were distributed to target groups and stakeholders in various occasions such as SH Inspectors training. Many copies have also been sent to RICEs for further distribution. (ii) In order to cope with an increased demand for the SH Inspectors training courses, NICE has transferred the responsibility for the organization of the training to the out sourcing system. (iii) Many private firms have been established and received the license for the organization of the training courses. (iv) After several years some firms have problems with the declining quality, i.e. some of the trainers did not have practical experience. NICE has recommended those firms that experienced SH Inspectors should be involved as trainers. Moreover, the system for the quality assurance and the renewal process of the license is also being improved.

These activities have raised the awareness of the workers and the entrepreneurs about safety system at work sites. Therefore, it is evaluated that the Project has a positive impact on the achievement of the declining rate of injuries occurrence and on the expansion of NICE activities in the promotion of OSH.

(2) Sustainability

Technical aspect is evaluated as high

(1)Human resources: About 71% of the current officials at NICE headquarters are the counterparts who have been working at NICE before the Project until present. The C/Ps who have been promoted to other Divisions in DLPW continued to work in relation to occupational safety and labour protection. The total number of government officials of NICE was maintained at 72 the same as the Project period. With the additional budget allocation from the Workmen's Compensation Fund (Office of the Workmen's Compensation Fund, Social Security Office, DLPW), civil employee and temporary employees were hired on contractual basis to work at the headquarters and at RICEs. (2) According to recent organization's Key Performance Indicator (KPI), each staff should attend staff development activity at least once a year. To encourage the achievement of KPI, NICE has provided support for the necessary expenses such as registration fee, transportation and per-diem. Beside government budget NICE also received assistance for staff development from related agencies. According to the C/Ps survey result, 89% of the respondents (Total respondents =17 persons) have gone through staff development activity not less than once a year after Project completion.

Organizational Aspects

(1) Utilization of the new building and workshop is evaluated as high: On October 29, 2003 the new building was officially opened. The new building is being used as offices of NICE and OSHID. It has facilities for training, laboratories, safety library, and exhibition hall. The equipment provided by JICA for laboratory has been moved to install in the new lab as planned. At present the last two days of the training courses of senior SH Inspectors as well as trainings and workshops on OSH for DLPW staffs are regularly being held at the NICE building.

(2) Utilization of supplied machine and equipment. According to the survey result, 79% of the provided machinery and equipment are in good condition and being used in training activities and the assessment of working environment etc. Some equipment procured from Japan, had a problem with availability of the spare parts in Thailand or too expensive to repair. NICE has purchased only some necessary items locally for the replacements. As most of the inspector training courses are being held in the regions by the private firms, the provided machines and equipments are not as effectively used as before.

(3) Adjustment of organizational structure is evaluated as high: After the Project completion several adjustment have been initiated by NICE personnel themselves in an attempt to be more focused and to better response to a greater demand of advisory services esp. from small and medium enterprises (SMEs). According to the most recent structure, NICE will decrease some roles which can be more effectively managed by outsourcing system. Thus, NICE headquarters will emphasize on promotion, development, and service certification. While RICEs will provide guidance services and strengthen the SH inspector's network.

Conclusively, as NICE has shown a continuous attempt in the adjustment of the organizational structure and the management of the available resources in response to the increasing demands, the sustainability of the organizational aspects of NICE is evaluated as high.

Financial aspect is evaluated as high

After the Project, NICE received adequate budget for organizational operation from DLPW Annual Budget Allocation (in 2003=20.70 mill. Bht., 2004=24.00 mill. Bht.).In 2005 NICE received 31.54 mill. Bht., which included budget for buying cars for RICEs, for the maintenance of the new building and for the repair and maintenance of the mobile training car which was provided by JICA. In addition to annual budget allocation NICE also received financial supports from related parties for publication of some manuals/reports and for organization of some workshops on OSH.

Sustainability of the Project effects

(1) Safety and Health Inspectors training; evaluated as high: The average number of SH Inspectors trained per year after the Project (29,645 persons/year) is higher than during the Project (19,616 persons/year).

(2) The relevance of the training curriculums evaluated as maintained: 65% of the survey respondents on relevance of the training curriculum is moderate. The senior manager reflected that the training course could provide the trainee with necessary knowledge on OSH but not enough information about the cost for the establishment of SH system and no clear evidence of the benefits that the enterprise will get from the implementation.

(3) Research activity and production of manuals/ guidelines; high: As more action research/research at work site in close cooperation with private enterprises was done (in identifying research topic, conducting research and financial contribution).

Based on the higher and wider achievements mentioned above the overall sustainability of the Project effects is evaluated as high.

3.2 Factors that have promoted project

(1) Impact

• Law and regulations: Enactment of 9 Ministerial Regulations on occupational safety (4 regulations announced in 2004 and 5 regulations announced in 2005), and a legislation on occupational safety, health and working environment announced in 2005. To comply with the law, the entrepreneurs are required to establish the SH system at work sites. As a result, wider groups of workers/ types of work particularly the high risk sectors are better protected.
• Policy factor: Policy towards "Kitchen of the world" which promotes safe food production, and One Tambon one Product (OTOP) strengthening program. (Tambon means sub-district.) The application of these two policies by related agencies has contributed to a more effective promotion of quality and safety in production activities.

(2) Sustainability

• Organizational factor: Human resource with high technical competence and adequate budget allocation are crucial factors contributing to the sustainability of the Project activities such as the development of SH curriculums and broadening networks with strategic partners i.e. NGOs, consultancy firms and academic institutes.
• Linkages and cooperation with International Organizations: Technical and financial assistances for example from ILO/JAPAN Multi-Bilateral Program, the World Bank as well as ASEAN OSHNET (Occupational Safety and Health Network) have contributed not only to the staff development but also the enforcement of the important joint policy measures.

3.3 Factors that have inhibited project

(1) Impact

• Organizational factors The government policy to freeze the increase of permanent officials makes it not possible for NICE to recruit more officials though there is a bigger demand. Recently, NICE is in the process of adjusting its working approach to be more flexible and able to mobilize cooperation from working partners to work with high risk industry and SMEs more effectively.
• Socio-cultural factor Even though public relations in various forms with regard to OSH have been promoted by NICE but the majority of the people are not yet aware of OSH and not enough attention has been given to the implementation of safety system management. The report on occupational injuries tells that one of the important causes of the accidents/injuries is the carelessness for self protection of the responsible personnel. The interview with senior Inspectors also confirmed that the owners of some enterprises, in particular SMEs, do not establish safety system mainly because the worry about the cost, while the prevention measures did not receive much attention.
• Lack of proactive working approach in public relations as well as positive measure to motivate OSH management particularly by medium and small enterprises including the promotion OSH as a national agenda.

(2) Others

The delay in the establishment of APO under MOL: As the Thai government has resolved in principle for setting up an APO, and the study result on international experiences on OSHAPO also indicated that an APO needs to be developed, but there is no clear conclusion yet on this matter.

3.4 Conclusion

After the Project termination in 2002, the achievement of over all goal and project purpose is evaluated as high. The evaluation is based on the following three reasons. Firstly, the rates of injury occurrence per 1,000 workers indicate a trend of slightly decrease every year, even though it has not yet reached the over all goal target due to the strengthened functions of NICE. Secondly, 283,039 SH Inspectors have been trained since the Project begin until October, 2005. The average number of Inspectors trained per year after Project completion is higher than during the Project. Finally, research results were disseminated to wider target groups. Public relations were improved in various forms. These activities have raised awareness of wider public in particular the workers and entrepreneurs in OSH.

The sustainability of NICE in the field of counterpart personnel was evaluated as high. The reasons are as follows: (i) About 71% of the current officials at NICE headquarters are the C/Ps. The C/Ps who have been promoted to other Divisions in DLPW continued to work in close relation to occupational safety and labour protection. (ii) With additional budget allocated from Workmen's Compensation Fund, civil employees and temporary employee were hired on contractual basis. (iii) In addition, 89% of the C/Ps interviewed have gone through staff development activities more than once a year. Thus, the knowledge of NICE personnel is continuously refreshed and the technology transferred by the Project has been transferred from senior C/Ps to the junior staffs.

The utilization of the new building and workshop is evaluated as high. The construction of the new building was completed and equipped with equipments and machines supplied by JICA. It was used as offices of NICE and OSHID for training activities, laboratory, safety library and exhibition hall.

The utilization of equipment is evaluated as low because only 79% of the provided machines and equipments are still in good condition and being used in training activities and assessment of working environment at NICE and RICEs, while the rest are out of order or deteriorated. These machines were procured directly from Japan. The spare parts were not available in Thailand or sometimes too costly to repair, Moreover, the bio-lab was closed down due to the resignation of the responsible personnel.

According to the most recent organizational structure of NICE, NICE headquarters will give more emphasis on OSH promotion, development and service certification. Whereas, RICEs will provide guidance services to SMEs and strengthen the establishment of SH inspectors' network. Therefore, the roles of NICE will be more focused, it is evaluated as high.

With regards to the sustainability of Project effects, it is evaluated as high. (i) The average number of SH Inspectors trained per year after the Project is a little higher than during the Project. It is mainly the result of the out sourcing system that NICE has been used in the management of training organization. (ii) To maintain the quality of the training courses provided, NICE has recommended the private firms that experienced Inspectors should be included as trainers in the training team, while the monitoring and evaluation system for quality assurance and renewal of the license is also being improved. (iii) SH Inspector training curriculum generally meets the demands of private enterprises. (iv) Based on knowledge and practical experience gained during the Project, many research projects were continuously conducted, manuals and guidelines also published. More action researches or research at worksite in close cooperation with the enterprise were managed. Publication of some reports and manuals were supported by the enterprise and funding agencies.

Conclusively, the Project has successfully strengthened the functions of NICE Headquarters to be more efficient and have a system for tools and manuals development necessary for the provision of OSH services despites some difficulties with the communication problems of some Japanese Experts. According to the results of the Ex-post Evaluation Study, it was clear that NICE has not only maintained the supported equipments and activities but further developed and expanded the coverage of the activities. Several adjustments have been made to make the functions more focused and to have a clear organizational structure. It was also expected that there will be transfer of technical knowledge within the organization especially from the Headquarters to Regional Centers.

Based on the above results, recommendations are developed and lesson learned is presented below.

3.5 Recommendations

• Capacity Development of license for machines: in order to manage the certification function efficiently, OSH Certification Section needs capacity development of specialized area on provision of license for the controller of high risk machines including monitoring and evaluation system for the renewing of the license. Assistance from donors in this field may promote OSH further in Thailand.
• Strengthening of the roles of RICEs: RICEs are in a better position than the headquarters for working with enterprises at worksites, as RICEs are being located in the provinces in different regions of Thailand. To do this, NICE needs to provide RICEs with proper technical knowledge, better equipments and efficient monitoring and supporting system. Conclusively, RICEs can actively contribute to a proactive role of NICE in safety promotion if they are properly supported.
• More proactive public relations on OSH: For example the lessons learned from cases of Best and Good Practices on OSH managed by different types and sizes of the enterprises should be disseminated via publications, media and various forms of action oriented training.
• As NICE has limited personnel, the trained SH inspectors can function as one of NICE strategic partners as resource person in training activities as well as in dissemination of OSH information and positive experiences in SH management. At present there are only few active SH Inspectors Networks, which were established at some provinces. Therefore, a clear plan and supportive measures for the strengthening of existing SH Inspector Networks and the establishment of the new ones should be developed.
• Maintaining and enhancement of the collaboration with academic institutes: Wider cooperation with academic institutes should be enhanced i.e. in the development of Occupational Safety Curriculum and educational activities for schools and higher educational levels, and in doing practical research in priority areas that supports the preparation of safety standards suitable with Thai condition, including the promotion OSH as a national agenda.
• Positive measures for motivation of OSH management: Positive measures should be arranged to support or motivate the establishment of safety system in the enterprises. For example low interest or no interest credit for medium and small enterprises, and the adjustment of the calculation and reimbursement system of the workmen's compensation fund for good practiced enterprises (reduction rate according to injury records).
• The collaborations with related agencies at national and international levels should be maintained and enhanced to seek for necessary technical assistances and to promote OSH together.

3.6 Lesson Learned

• The true success of this Project is that the workers are protected from occupational injuries and diseases. At this point it is learned that the roles of RICEs, which are located in the provinces, are very important but they were not directly involved in the Project. Therefore, it should be noted that, the emphasis of the technical assistance for the organization like NICE should not cover only the headquarters but also pilot sites in the regions.
• Technical Transfer: Most of JICA experts were evaluated highly by Thai counterparts with respect of their knowledge and experience in the fields of OSH. However, some experts had difficulty in communicating in English. As a result the technical transfer could not be done efficiently. In order to avoid the preventable inhibiting factor, it is suggested that English language competency should be considered as a criterion for expert selection.
• Oversea training: In addition to technical transfer by short term and long term experts, oversea training is another effective means for practically intensive learning. To make it more beneficial, it is proposed that the agreement on capability building of the counterparts including oversea training curriculum should be carefully adjusted again by mutual agreement of both parties prior to actual implementation. Moreover, the training documents should be all translated into English in order to avoid limitation to learning.
• For the research, training and service certification institution like NICE, technical assistance emphasizing the capability development of the C/Ps is very helpful for the strengthening of the functions of the implementing agency in charge. However, as the production technology changes rapidly, it is recommended that the supports for machinery and equipment for both the training activity and the assessment of working environment should be carefully considered. Outsourcing system can be alternatives for more efficient management of some activities.