Ex-post Evaluation

1-1. Background of the Project

In Jamaica, the primary commodities, such as bauxite, agricultural products, etc., account for around 52.5 percent of exports in 1998 and around 66.0 percent in 2004 at the time of the ex-post evaluation study. As the value of these products in international markets has been flagging, with the country's high dependency on imports for most of its industrial products, a trade deficit has resulted. To improve the situation, the government of Jamaica has been developing its tourism industry, promoting domestic production of products, and expanding exports by improving the quality of textile and industrial products. However, the technical level of domestic companies was low, and efforts to improve the trade balance were not producing the desired results.

Under these circumstances, the government of Jamaica recognized that fostering the development of middle-level engineers was a prime task and organized the "Technical and Vocational Education and Training Improvement Project (TVET project, 1995-2000)" to strengthen its international competitiveness. The plan was aimed at establishing a practical technical education program through the teaching of up-to-date techniques in technical high schools, one of the avenues of secondary education. In implementing the plan, the Ministry of Education, Youth and Culture (MOEYC) identified Jose Marti Technical High School (JMTHS) located in Spanish Town (the former capital) as a pilot school for vocational education, in order to improve technical education in four specialized fields, i.e., machine shop, electronics, auto mechanics, and computer assisted drafting (CAD). At the same time, the government of Jamaica requested the assistance of the government of Japan, which has wide-ranging experience in these fields, to Project-type Technical Cooperation to achieve the aforementioned objective.

1-2. Project overview

The project had its primary aim at improving technical education in the four specialized fields: machine shop; electronics; auto mechanics; and CAD. It also involved the upgrade of facilities and equipment in line with the development of an educational curriculum and improved education, the transfer of techniques to the counterparts, and the implementation of training for high school teachers at JMTHS.

(1) Overall Goal

To improve the quality of TVET in Jamaica.

(2) Project Purpose

To conduct improved TVET in the fields of auto mechanics, CAD, electronics, and machine shop at Jose Marti Technical High School as a pilot school for technical high schools in Jamaica.

(3) Outputs

a) To improve TVET in the fields of auto mechanics, CAD, electronics, and machine shop at Jose Marti Technical High School.

b) To disseminate improved TVET to other technical high schools in Jamaica.

(4) Inputs (as of the Project's Termination)

Japanese side:

Jamaican side:

3-1 Summary of Evaluation Results

(1) Impact

a) Achievement of the Overall Goal

As an indicator of the Overall Goal, "employment by industrial sectors" is utilized, so the number of JMTHS graduates employed by the industrial sectors is shown in the Table 1 below. From the figures, more than 10 JMTHS graduates have constantly been hired in the industrial sectors since 2003.

In addition, according to the questionnaire survey directed to the employers selected by JMTHS with reference to the question as to whether graduates are competent enough to successfully complete their work task, one half the number of respondents rated "5: Strongly agree" in the five rank evaluation as a highest remark and all the remaining respondents rated "4: Agree". Additionally, all the employers answered that they would like to employ graduates from technical high schools again. Although the sample size is eight, the results indicate that the graduates are highly rated and their employers would like to keep hiring technical high school graduates from now on.

From a long-term perspective, therefore, it might be expected that a number of graduates will be employed in the industrial sectors as long as there is a need for highly skilled technical workers in the industrial sectors.

b) Relationship between the Overall Goal and the Project Purpose

The in-service training sessions at JMTHS have not been carried out since the termination of the project because the MOEYC cannot secure the budget for the implementation. However, the improved TVET in the four fields has been conducted at technical high schools in Jamaica through the teachers trained at JMTHS during the cooperation period. According to the questionnaire survey targeting 246 students at six technical high schools, students are quite satisfied with the technical knowledge and teaching skills of their teachers who participated in in-service training sessions at JMTHS. From the survey results, 93.5% of students responded that they were either "5: Strongly satisfied" or "4: Satisfied" with their teachers' technical knowledge. Moreover, 95.9% of students responded that the teaching skills of the teachers were either "5: Excellent" or "4: Good". However, it might be difficult to determine whether or not the quality of TVET is improved through the in-service training sessions alone.

(2) Sustainability

a) Policy and institutional aspects

The HEART Trust/NTA(1) contributes to development of all the technical high schools, including JMTHS, in Jamaica so as to offer training sessions to support for workforce development in Jamaica and financial support to assist in the upgrading and maintenance of facilities in the technical high schools. The HEART Trust/NTA is involved in professional development for teachers' training programs. Some teachers, who are not fully qualified, are able to improve their qualifications by participating in the Vocational Training Development Institute (VTDI)(2) program. It will contribute to the quality of education in technical high schools.

In addition, the HEART Trust/NTA has been addressing the equipment needs of the technical high schools through the Technical High Schools Development Project (THSDP). The THSDP started in 1997 and aimed at upgrading and maintaining facilities for the 14 technical high schools in Jamaica.

Furthermore, the TVET Rationalization project, which is to be formulated by the MOEYC, seeks to spread the use of equipment among technical high schools located in close proximity to each other, in order to share technological equipment at the satellite schools.

b) Financial aspects

As shown in Table 2, the MOEYC has kept the budget around 9 – 10 % of the national budget from FY 2000 – 2004. Also, the area of TVET in the MOEYC has been maintained around 3.5 % of the MOEYC budget from FY 2000 – 2004. Although multilateral/bilateral funding is allocated to projects for the TVET, the funds are not added in the budgetary sheet of Table 2. Overall, the budgetary trend has favorably been kept at the decent level in the past five years.

According to the comment from the MOEYC, however, there are factors that make it difficult to continue in-service training sessions at JMTHS, that is, the MOEYC does not have enough funds available to provide accommodation and transportation for the participants in the in-service training sessions.

c) Technological aspects

Although the conditions of the main equipment at JMTHS are good for the most parts, some of the equipment have been out of operation at JMTHS. For example, since some of the equipment from Japan are unique in Jamaica, it is difficult to repair them and to find an agent to fix them. The result is that they have not been utilized for the past several years. Although a senior volunteer was dispatched to JMTHS as an educational equipment maintenance advisor for one year after the termination of the project, remarkable effects did not show up.

In terms of JMTHS teachers, technical high school teachers participating in the in-service training sessions highly rated the JMTHS teachers with reference to their competency for improving TVET through the in-service training sessions. According to the questionnaire survey directed to 19 technical high school teachers, 75.0% of teachers responded that the JMTHS teachers were either "5: extremely competent" or "4: competent".

3-2. Factors that have promoted project

(1) Impact

The Caribbean Examination Council (CXC) is utilizing parts of the CAD syllabus developed by JMTHS teachers and Japanese experts. The CAD section is an optional section in the Technical Drawing of the CXC examination.

At Holmwood Technical High School, the CAD facilities were established through the grant assistance for grassroots projects by the Embassy of Japan in December 2002. The design of the CAD facilities at JMTHS has heavily influenced the design of the CAD facilities at Holmwood. Presently, JMTHS and Holmwood technical high school are planning a workshop for teachers of technical drawing (TD) in all technical high schools. This was agreed upon after discussions between the principals of both schools.

(2) Sustainability

The in-service training sessions brought teachers at technical high schools together at JMTHS. Thus, the training sessions provided technical high school teachers with opportunities to create a network, to share ideas, and to update each other on what was happening at their respective technical high schools from the long-term perspective.

The VTDI put in place by the HEART Trust/NTA helps provide tertiary level training for teachers, in order to enhance their technical knowledge and teaching skills.

3-3. Factor that have inhibited project

(1) Impact

The indicator of the Overall Goal was not set up appropriately. For instance, although the indicator of "employment (of technical high school graduates) by industrial sectors" was prepared, there were no tracer studies of the graduates in technical high schools, including JMTHS, in Jamaica. Under this circumstance, it is difficult to confirm the attainment of the Overall Goal as a part of positive impacts.

Several teachers in technical high schools have not yet received the certificates of completion for the in-service training sessions, which they were supposed to be provided with. This might hinder the TVET activities in terms of teachers' incentives and motivation for teaching. According to the interview survey, involving several teachers at the technical high schools, at the beginning of the in-service training sessions at JMTHS, the MOEYC explained to the trainees that certificates would be issued upon completion of the training sessions at JMTHS. However, certificates were not distributed to all the participants, even though they had completed the training sessions. This might be due a lack of effective administration by the project team and the MOEYC during the cooperation period.

A teacher at a technical high school who had the in-service training sessions was not teaching machine shop but teaching physical education instead. After returning from his leave, a new teacher for machine shop, who did not participate in the in-service training, was assigned to the machine shop and he was deployed as a physical education teacher. The school administration explained that he did not have the exact qualification for the field and he was the only teacher in the field at the time when the in-service training sessions were carried out. In terms of the efficiency of the project, it can be said that it is inefficient because the project resources were not utilized for the achievement of the Overall Goal.

There are not enough job opportunities for many graduates of technical high schools to practice their enhanced skills.

Violence of students in schools is widespread across the country, which negatively affects students' performance.

(2) Sustainability

The in-service training sessions at JMTHS have not been carried out since the termination of the project because the MOEYC cannot secure the budget for the implementation. However, it might be difficult to determine whether or not the quality of TVET is improved through the in-service training sessions alone. Furthermore, although it is necessary to improve teachers' technical knowledge and teaching skills at technical high schools continuously, there is the absence of any mechanism through which JMTHS and other technical high schools are able to collaborate and communicate with each other.

Some of equipment have not been fixed and utilized yet because of a lack of makers to repair the equipment which are unique in Jamaica and also due to financial resources. Also, in the in-service training sessions at JMTHS, teaching aids which are not readily available in technical high schools were utilized in the practical lessons. However, it is of no value to learn the practical lessons with the kits and equipment which are only used at JMTHS because trainees in the in-service training sessions cannot continuously employ them in their technical high schools.

Several highly trained teachers in Japan left technical high schools and were recruited to work abroad. Loss of the competent teachers negatively affects the sustainability of the project from the aspect of human resources. Thus, unless the trained teachers continue teaching at their schools for a certain period, it is limited to covey their knowledge and skills to teachers and students at their technical high schools.

3-4. Conclusions

The JMTHS graduates are highly rated by the employers, and they would like to keep hiring technical high school graduates from now on. From a long-term perspective, it might be expected that a number of graduates will be employed in the industrial sectors as long as there is a need for highly skilled technical workers in the industrial sectors. Furthermore, technical high school students are quite satisfied with the technical knowledge and teaching skills of their teachers who participated in in-service training sessions at JMTHS. Also, the CXC is utilizing parts of the CAD syllabus developed by the project and making use of it in the Technical Drawing of the CXC examination. These positive outcomes are produced by the efforts of the project-related personnel.

Although the positive outcomes appear in a certain degree at JMTHS as a pilot school, the impacts to other technical high schools have not been grasped because technical high schools did not track the indicator of the Overall Goal, "employment by industrial sectors", at all. Also, the in-service training sessions at JMTHS have not been carried out since the termination of the project because the MOEYC cannot secure the budget for the implementation. The MOEYC and JMTHS need to make efforts in order to achieve the Overall Goal. Altogether, it is considered that the impact of the project is low.

From the perspective of the project sustainability, there are two aspects of budget and equipment. In the budgetary aspect, the area of TVET has been maintained around 3.5 % of the MOEYC budget from FY 2000 – 2004 as shown in Table 2, but the MOEYC cannot secure the budget for the continuation of the in-service training sessions at JMTHS.

In the aspect of equipment, some of equipment have not been fixed and utilized yet because of a lack of makers to repair those equipment and also due to financial resources. Additionally, it is of no value to carry out the in-service training sessions with the practical kits and equipment which are only used at JMTHS.

In order to promote the sustainability of the project, it is desirable to conduct training sessions for teachers continuously and to upgrade and maintain facilities and equipment actively by employing the HEART Trust/NTA. Although it seems that the sustainability of the project is low at this time, it would be possible to enhance the sustainability of the project by strengthening the collaboration with the HEART Trust/NTA, etc.

3-5. Recommendations

To the MOEYC and Technical High Schools in Jamaica:

In order to improve the quality of TVET, a network between JMTHS and other technical high schools should be established to facilitate collaboration with technical high schools and to share the knowledge, skills, ideas, etc. with technical high school teachers.

There are no tracer studies of graduates who have been employed in the industrial sectors. In order to assess the achievement of TVET, the MOEYC should set up the benchmarks, such as employment rate and tertiary education rate, which should be recorded by all the technical high schools.

The MOEYC and technical high schools should establish a job placement service through which technical high schools will be able to provide assistance for the graduates by matching their skills with the needs of employers.

Several teachers participating in the counterpart training in Japan are leaving their technical high schools, so the responsible department in the schools should instruct those trained teachers to document their knowledge and skills learned through the training program before they leave their schools.

Teachers regard the certificate as a testimony to the enhanced qualifications obtained in the field they are teaching. Thus, it is necessary to provide certificates to motivate, to satisfy, and to elevate the confidence level of the teachers in their field.

Because there are several teachers who had received the in-service training sessions at JMTHS leaving their schools, these training sessions should be held for sharing the knowledge and skills with the teachers who did not participate in the training sessions before.

To JICA:

The certificates for some of teachers who completed the in-service training sessions have not yet been provided for them. This might be due a lack of effective administration during the training sessions. Thus, the certificates should be provided for teachers, as soon as possible, in consultation with the MOEYC.

3-6. Lessons learned

In order to reflect what teachers learned at a pilot school to their technical high schools, specific measures should have been put in place. Even if teachers learn practical skills with new kits and equipment, they cannot transfer the same practical lessons to their students without the necessary kits and equipment. Thus, kits and equipment which are readily available for each school at a reasonable cost should be employed in the practical lessons.

The indicators should be clearly set up before and during the project. Thus, it is important to prepare the indicators which counterparts are able to track even after the termination of the project. Simultaneously, the means of verifications must be examined thoroughly in terms of cost, the credibility of data sources, and ease/difficulty of obtaining data.

Certificates for trainees are very crucial for the enhancement of their qualifications. In order to satisfy the teachers in terms of the contents of training and administrative aspects, the preparation and pre-explanation for the certificates are necessary for the training sessions.

It should be confirmed whether or not trainees who join the counterpart training in Japan have the willingness to continue their present job for several years with a "written agreement" before launching the training program. Thus, it might be necessary to have teachers, trained by the implementing agency and JICA, bonded to teaching in the field in which they were trained for at least a couple of years.

The school should keep the teachers who are involved in the training sessions in the applied fields, and the Ministry concerned should give the schools directives, which aim at having teachers continue teaching in the relevant fields in which they were trained.

(1) The Heart Trust/National Training Agency is the quasi-governmental agency in Jamaica. It receives funding from a 3% levy on all employers in Jamaica, who have a payroll of over J$ 14, 000 per week.

(2) VTDI is the professional training arm of the Heart Trust/NTA. They offer tertiary level education in a variety of areas including information and communication technology, entertainment management, instructor training, and Career Guidance and Counseling.