Ex-post Evaluation

• Country : Indonesia
• Project title : Higher Education Development Support (HEDS) Project
• Issue/Sector : Education
• Cooperation scheme : Technical Cooperation
• Section in charge : Social Development Department
• Total Cost : 3,249 million yen
• Period of Cooperation : April 1990 – July 31, 2002
• Partner Country's Implementing Organization: Directorate General Higher Education (DGHE) Officer , Ministry of National Education.
• Supporting Organizations in Japan : Ministry of Education, Culture, Sports, Science
• Related Cooperation : No

1-1 Background of the Project

Rapid growth of engineering industry in Indonesia and especially in Sumatra and Kalimantan islands in 1990s required support with qualified human resources. Unfortunately, higher education on engineering at those islands could not fulfill the need of industry due to lack of quality of graduates from universities. This condition promoted the Government of Indonesia (GOI) via Ministry of National Education to proceeded Higher Education Development Support (HEDS) Project in order to improve the quality of teaching staff engaged in engineering field. This project has been implemented through cooperation with Japanese Government through JICA since 1990.

1-2 Project Overview

In April 1990 HEDS project was started in form of cooperation between the Government of Indonesia and JICA to improve the quality of the teaching staff and university administration at eleven universities in Sumatra and Kalimantan. In this project, JICA has been cooperating in the field of Engineering at 11 target universities (TUs).

To launch the improvement in the quality of teaching staff and university administration, HEDS has conducted several activities. Activities include establishment of project management system, enhancement of teaching staff expertise, improvement of equipments for core laboratory, promotion of research activities, improvement of the university administration, development of Indonesian language textbooks, establishment of human network, and dissemination of information project activities

(1) Overall Goal

The overall goal of HEDS project is "the graduates of 11 TUs support the development of engineering industry in Sumatra and Kalimantan islands". This overall goal has two objectively verifiable indicators. One is "the graduates employed in engineering related field is increased" and the other is "working ability of the graduates is upgraded".

(2) Project Purpose

The purpose of HEDS project is "the quality of teaching staff engaged in engineering education at the TUs is improved." The achievement indicator for project purpose is: the way of teaching, content of lectures and communication of TUs Teaching Staff toward students are improved.

(3) Outputs

The outputs of HEDS project include (a) The expertise of teaching staff is enhanced, (b) Core laboratories are well utilized for research and student practice, (c) Researches are actively conducted, (d) University administration is improved, (e) Textbooks in Indonesian language are developed, (f) Domestic and international research networks among teaching staff are established, (g) Information on the project activities is disseminated to TUs, and (h) Platform for self sustainability is well developed

(4) Inputs

The number of expert dispatched (for HEDS Project extension period) 

Members of Evaluation Team:

JICA Indonesia Office (cosigned to PT. Mitrapasific Consulindo International⁄PT. MCI)

• JB. Haryanto⁄PT.MCI
• Agus Wahyu Darmayanto ⁄ PT. MCI
• Zulkarnain⁄PT.MCI
• Ibin Sugandi ⁄ PT. MCI
• Diding Hardedi ⁄ PT. MCI

Period of Evaluation:

December 2005 – March 2006

Type of Evaluation:

Ex-post Evaluation

3-1 Summary of Evaluation Results

(1) Impact

The expected overall goal is "the graduates of 11TUs support the development of engineering industry in Sumatra and Kalimantan islands". The overall goal has two objectively verifiable indicators. One is "the number of graduates employed in the engineering field is increased" and the other is "working ability of the graduates is upgraded".

Most of the graduates of TUs were employed by industries in Sumatra and Kalimantan islands. Most of graduates from TUs are working as technicians in telecommunications, electricity, mining and other type of industries. Some of them work as bureaucrats at local government and even in banks. Economic crisis hit Indonesia in 1997 and caused a reduction in number of industrial companies. As a result, the decreasing number of companies weakened labor force absorption.

The graduates of TUs are receiving high reputation and having important position in the companies. The companies' perception on the performance of TUs graduates in the last three years commonly is "satisfy" (61%) and only 1 % said "dissatisfy". graduate performances. The improvement can be seen from increase in GPA attained, higher efficiency in education and shortening of waiting time for employment.

HEDS project has contributed to the improvement of quality management in TUs, this experience has disseminated to other universities by DGHE. HEDS has contributed through curriculum development which accommodates the industrial needs.

The evaluation result shows that the utilization of transferred technology in TUs is very high. Although some of technologies are out of date, but teaching staff and students can use those technologies for research and practice in laboratory. Furthermore, HEDS project has contributed to the community through the participation of teaching staff in the community, collaborative research and in the consultancy and research works at companies.

Institutionally, HEDS project has strengthened the organization of faculties and university, in forms of human resources quality improvement, managerial improvement and laboratory-facility improvement. The evaluation result shows that in general HEDS project also contributes to development or improvement of other universities in areas.

Based on the evaluation from National Accreditation Board (AB), there is improvement in accreditation status in the last three years. In the year of 2002 from 40 study programs operated by engineering faculty of TUs, most of them received grade C (57.62 %) and grade B (34.62 %), while only 7.69 % study programs received grade A. In 2005, this condition improved, because from 41 study programs 14.63% received grade A, 50.09% received grade B and 29.27 % received grade .

(2) Sustainability of the Project

Government of Indonesia has published "guidelines of higher education long term development program". The guideline is prepared based on the concepts of Total Quality Management that HEDS project has introduced. Furthermore, after HEDS has terminated, the network among TUs still sustained and even expanded, not only covered engineering field but also other disciplines such as: education, science, sociology, economy, and others. Five universities previously participated in HEDS project agreed to continue HEDS activities in form of Graduate School Establishment (GRASE).

The core laboratory facilitated by HEDS project has been used up to now. The trained technical staff is still working at TUs. Technologies and most equipment procured through HEDS project have been used up to now. Although some of the equipments were broken and not used anymore, due to unavailability of spare parts at local market. In the future there should be a policy to solve the problem related to such technical aspects.

The limited budget became the major constrain for TUs to operate core laboratory optimally. Some of TU have commercialized the equipments in core laboratory in order to obtain additional income a part of maintenance and rehabilitation costs. Unfortunately, the income generated from activities is not adequate.

3-2 Factors that have promoted project

The strong commitment of government to the development of higher education has promoted the impact and sustainability of the project. The higher education in Indonesia has a mission in education, research and community service. Community, as one of main funding source has a right to get the information related to the quality performance of higher education. The establishment of National Accreditation Board (NAB) for Higher Education can be used as a media in evaluating the quality performance of higher education.

The participating universities try to improve their accreditation status by improving the quality of education process in their university. The improvement has been achieved through: enhancement of teaching staff quality, improvement of facilities (learning facilities, laboratory, library) and improvement other support facilities. Since the components of accreditation are quite similar to HEDS activities, TUs have no difficulties in implementing these components.

Indonesia economy has started to grow in the past couple years and such a growth enhanced the improvement of community income, and promoted the growth in industries. The spill over from this improvement is "the interest of community to enroll higher education tends to increase". In the past three years, the enrollment to engineering faculty has increased and the capacity of industries to absorb the graduates from engineering field has also started to increase.

3.3.2. Factor Inhibiting impact & sustainability

Beside the success story of HEDS project activities, there are some factors that inhibit to the sustainability. The main cause of in-sustainability, according to respondents, is "no sufficient budget to support operational activities". The inefficient bureaucracy also inhibits sustainability of HEDS activities in the future.

Odd changes in management of engineering faculties or universities have inhibited the impact and sustainability. As the new management especially at university level does not know about the HEDS activities, the support to HEDS activities is limited. Due to limited laboratory equipments, the management of some engineering faculties even have distributed the core laboratory equipments to other laboratories. The impact was that the student or researchers found difficulties in utilizing the equipments. The lack of trained technical staff becomes hindering factors especially in maintaining the equipments. Short training for technical staff did not assure that the staff can maintain the equipment properly, because of that the longer training for technicians will bring better results.

3-4 Conclusion

• The empowerment of the WUAs involves a wide range of activities from the upstream area to downstream of irrigation scheme. The project had been designed to maximize the involvement of all those relevant to the activities of WUAs in its operational mechanism. This approach is considered to be highly practical and the comprehensive efforts from different institutions could contribute to the effective implementation of the Project.
• The positive impacts of HEDS projects among others are: (i) promoting some TUs at Sumatra Island to establish graduate schools, (ii) enlarging the network to covers not only engineering fields but also economics, law, education and arts, (iii) the TQM seminar has disseminated to other universities in Indonesia. The concept of TQM also has been used as the basis for the formulation of long term higher education policy, and (iv) the HEDS principles and activities are in line with the accreditation components by NAB.
• HEDS activities have been sustained. The government has a strong commitment to continue HEDS activities although in different project. Currently, DGHE has cooperated with some TUs and has designed a new project called GRASE (Graduate School Establishment). Technologically, most of the equipments and facilities procured can be used up to now. In line with the rapid change in technology, TUs has upgraded procured equipments, especially computer soft wares.

3-5 Recommendation

To executing agency:

• HEDS project has strengthened the capacity of faculty of engineering at TUs. Due to this strong impact, it is recommended that HEDS project should be followed-up. If possible, the future project not only covers engineering field but also other fields, such as economics, education science, and law.
• Directorate General of Higher Education (DGHE) should maintain the function of the HEDS Forum in order to facilitate communication and collaboration among TUs, other Indonesian Institutions, Japanese as well as other foreign universities and JICA.
• These equipments supplied by HEDS that do not match the needs of the laboratory of a specific university should be relocated to a more appropriate laboratory at other TUs, and repair or replace the broken equipment at TUs should be repaired or replaced.
• DGHE should continue a policy on empowerment of universities by facilitating and providing competitive grants.

To Target Universities:

• TUs should maintain the equipment procured and optimize its utilization. Since TUs have difficulties in providing sufficient budget for maintenance and replacement of broken equipments, TUs have to promote and enhance income generating activities. One effort that can be done is strengthening of networking with local government and private companies.
• TUs should continue to improve teaching staff qualification to enhance research, publication, writing and educational capabilities.
• TUs should develop new technologies that meet with the need of local industry.
• The TUs should improve academic atmosphere to achieve excellent reputation as evaluated by National Accreditation Board (NAB) for Higher Education.
• In the future, TUs have to develop and strengthen the networking with other institutions, whether, private, government or international institutions, in order to maintain and optimalize the utilization of core laboratory.

5-2 Lessons Learned:

According to the result of ex-post evaluation, the followings are respectfully submitted for consideration for other future project:

To Government of Indonesia (Executing Agency):

• The achievement of overall goal, more or less, is influenced by the changing of external conditions that has been set as assumptions. These assumptions should be monitored and evaluated. When the change in assumptions exists, the project management has to response in minimizing the associated negative impacts, if any.
• Sustainability of the project depends on the continuity in leadership, management and participation of stakeholders. Because of that, satisfaction and responsibility in regards to teaching and research as well as a sense of ownership of the cooperation project of academic staff are essential for technical success. The management of TUs should continue utilizing those trained staff so that they can contribute more to their university.
• Human resources development always requires a long time to really show productive results. Any weakness shall be identified at the earliest time, thus to become the key factors for successful effort.

To JICA:

• In the future the procurement of equipments should take into consideration the availability of the spare parts at local markets, so that the local technicians can repair or replace the broken equipments.
• JICA should consider the availability of infrastructure at the project target, when procuring the equipments. The experience showed that some of the equipments procured can not be used optimally due to lack of infrastructure support.
• Knowledge level, current capabilities and cultural background of human resources at project target should be considered in designing the project. The experience showed that the sophisticated equipments were under utilized because of the limited capability of local staff.