Ex-post Evaluation

1-1 Background of the Project

"Urban Transportation Human Resources Development Center", a project-type technical cooperation, was carried out at University of Brasilia (UnB) for four years from August 1998 to July 2002. The Pluriannual Investment Plan (1996-1999) elaborated by the Brazilian Government under President Cardoso cited modernization of the national transportation system as one of the prime action plans in transports sector. Major cities in Brazil have suffered from deterioration of traffic congestion and air pollution caused by an increase in vehicles on the roads. Establishment of an efficient urban transportation system was therefore required through improvement of public transportation institutions as a major priority.

Since the National Urban Transportation Company (EBTU) was abolished in 1990 by the new government's decentralization policy, technological development of urban transportation came to depend entirely on the work conducted by Universities and Research Institutes. Given this circumstance, the Brazilian Government requested technical cooperation from the Japanese Government for the establishment of an Urban Transportation Human Resources Development Center at University of Brasilia, for the purpose of upgrading urban transportation personnel. The aim was to develop this center into a transportation research center to hold training courses, conduct research, and serve in an academic capacity.

The project was begun in 1998 and completed in 2002. The Project's results were remarkable and widely recognized. CEFTRU was converted into one of Brazil's most important National Centers of Reference for Transportation training, studies and postgraduate specialization courses (more than 360 class hours).

The goal of this study was to assess the performance of the Urban Transportation Human Resources Development Center (CEFTRU) with regard to "impact" and "sustainability". This ex-post evaluation examined whether the effects of the Projects are sustained and have expanded even after its completion.

1-2 Project Overview

The Government of Japan provided technical cooperation for the improvement of the Training Center at the University of Brasília to develop Brazilian Urban Transportation human resources; to conduct research and serve in an academic capacity.

(1) Overall Goal

Capacity of personnel engaged in planning, management, operation and education of urban transports is improved through the training provided by CEFTRU.

(2) Project Purpose

CEFTRU is well established so that training of personnel engaged in planning, management, operation and education related to urban transport may be undertaken effectively.

(3) Outputs

1. Training programs which suit the demand of planning, management operation and education in the field of urban transport are prepared.

2. Capacity of instruction in the field of urban transport is obtained.

3. Facilities and equipment necessary for training are well prepared.

4. CEFTRU is properly managed in terms of organization, personnel and finance.

(4) Inputs (including the input for the follow-up activities)

Japanese Side:

Brazilian Side :

3-1 Summary of Evaluation Results

After completion of the Project, the Project Purpose has been fully achieved. CEFTRU has trained 4000 personnel from relevant organizations in the transports sector on 102 training courses, including postgraduate specialization courses. The trainees are engaged at public and private institutions from all 5 administrative regions of the country. CEFTRU has increased the number of sectors and beneficiaries of training courses with a high degree of satisfaction among beneficiaries of its activities in the fields of Training, Research and Consultancy.

(1) Impact

(1) Achievement of the Overall Goal:

From July 2002 to December 2005, the CEFTRU saw 115 students obtain postgraduate qualifications in 5 postgraduate specialization courses in Traffic Education, Road Surfacing, and Civil Aviation. In the same period, CEFTRU implemented different training courses for personnel from the Center–West, North and other regions of Brazil as follows: 2002 – 9 training courses in urban transportation with 335 trainees and 1 training course in civil aviation with 18 trainees; 2003 – 8 training courses in urban transportation with 292 trainees.; 2004 – 12 training courses in urban transportation with 709 trainees in 11 cities; and 2005 – 67 training courses in urban transportation with 1898 trainees implemented in 47 cities representative of Brazil's regions with professionals coming from neighboring cities.

Results for the period from August 2002 to December 2005: 3967 students were trained in 102 courses, including the postgraduate specialization courses. This represents an annual average of 1000 students trained, an achievement three times greater than expected as expressed in the indicator for the Overall Goal of the PDM, which is of 300 students trained per year for 2002-2007.

These training activities also respond to current institutional changes in the national transportation scenario, which took place in Brazil from 1999 to 2003. The main changes were as follows: (i) the present Brazilian Traffic Code was adopted in 1999, substantially expanding the authority of municipal administrations with regard to traffic regulations; (ii) the Ministry for Cities was established in January 2003; urban transportation is subject to Brazil's land use policy; and (iii) the Ministry of Transports announced important changes in June 2001. The following institutions were established: DNIT (National Department for Transportation Infrastructure), ANTT (National Terrestrial Transports Agency) and CONIT (National Board for Integration of Transportation Policies).

According to interviews with and questionnaires filled out by members of relevant institutions in the transports sector, the CEFTRU Project is a key component in the improvement of knowledge and skills among personnel working in the this sector in Brazil. Other training programs are developed by public institutions responding to the demand for training and qualifying human resources in this area, which is constantly increasing due to ongoing urbanization in Brazil. The Project is deemed highly effective 3 years of its completion, and remains adequate in this new institutional scenario.

Of those interviewed, 90% considered that up to 90% of the originally proposed results are continuously achieved by CEFTRU in the development of its training activities, including research and development of projects. All participants and public institutions (100%) interviewed confirmed the adequacy of the training contents and the urban transportation improvements obtained with the professionals trained.

The institutional partners highlighted the high degree of initiative, excellent technical capacity and quality of materials elaborated by CEFTRU. The relevance and the positive impact of the CEFTRU Project in providing training for professionals in this sector were underscored.

According to the replies provided, 100% of the alumni still working in the transports sector have been using the knowledge acquired; they consider this knowledge highly adequate; 95% have been handing on the training through distribution of technical materials; and 95% have been using it in their daily work. Among alumni, 85% would recommend that institution to which they belong send other students to the Center.

CEFTRU submitted evaluation questionnaires to all trainees upon completion of training courses during 2002-2005. Analysis of data available from final reports suggests that the degree of satisfaction with content and instructors is around 90% as stated by trainees and their Institutions.

Only 750 Brazilian municipalities were already integrated into the System in February 2006. To integrate the other 4500 municipalities in forthcoming years, the National Traffic Department (DENATRAN) has counted on support from CEFTRU, which ran these courses for about 1090 towns and cities to train 1759 trainees.

It is possible to state that in the research and project development fields CEFTRU has been achieving new technical improvements which have contributed to achievement of the Overall Goal of the Project.

After completion of the Project, the Overall Goal has been fully achieved. Despite being initially designed to focus on the Center-West and North regions, CEFTRU has expanded its activities and its network, increasing the number of beneficiaries of the training and research section, obtaining institutional, technical and social recognition for its capacity in the area of urban transportation and other sub-sectors of transportation.

(2) Causality:

The Project has had a very positive impact on the training of human resources in the urban transports sector as well as in the fields of Research & Development of Transportation Projects. No negative impact has been caused by implementation of the Project.

The Overall goal could be an impact it self produced through implementation of the Project after its conclusion. The important assumptions for the development of the Project and for the achievement of the Project Purpose are suitable for conditions in Brazil. The main assumptions for attaining the Project's Purpose have been secured and have significantly contributed to achieving the Overall Goal. The professionals hired continued to be engaged at CEFTRU and University of Brasilia. Additionally, CEFTRU has maintained a financial balance and adequate administrative standards over the past 3 years as an institution managing own resources.

(3) Ripple effects:

Concerning ripple effects, the Project's effects on policy can be observed in many cases in which human resources from CEFTRU transfer acquired technology and knowledge by contributing to the process of drafting laws as well as serving on various Governmental Commissions and in Professional Associations in the Transports sector. By implementing the Project, CEFTRU has acquired great visibility and technical respect among relevant agencies in the Transports sector, especially among professionals trained at the Center. CEFTRU's teaching staff and consultants have been invited to sit on Technical Committees. They have also been consulted on technical matters such as legislations or norms governing transportation projects (e.g. the new law on urban mobility).

According to the interviews and questionnaires submitted to former trainees (1998-2002) who work in the transports sector, the evaluation team can vouch that the CEFTRU Project has had continuity, generating significant impacts through training courses and research that have contributed to the achievement the Overall Goal for the Project.

(2) Sustainability

After the Project was completed, the Center divided a number of long-term courses into short-term courses and, in addition, introduced many new short-term courses. This change was made in order to meet the needs of the trainees and institutions in the transports sector. The Center's capacity to provide training was maintained and developed further after the Project's completion.

The evaluation team believes that technical sustainability of the Project has been ensured as the counterpart has improved technical knowledge introduced during implementation of the project and new short term courses.

(1) System and Political aspects:

CEFTRU enjoys institutional support from University of Brasília and the main institutions active in the Transports Sector (e.g. Ministry for Cities, Ministry of Transports, ANTT) and is actively engaged in the network of higher education institutions dealing with Transportation. Current legislation on Transports is likely to remain unaltered in the medium and long terms since modifications were introduced as recently as the year 2000. The Center has adapted and responded appropriately to the new demands the sector has generated. Through interviews and meetings with governmental institutions demanding training activities from the Center, positive feedback was provided on the sustainability of CEFTRU. Institutions indicated that in the years ahead they will be given follow-up on current training; they will also continue to consider CEFTRU a privileged center for research and development of Projects. The Center continues to maintain public relations activities (home page, participation in seminars, broadening its knowledge and obtaining high national visibility). The Project continued to be an effective mechanism for the training of human resources in the urban transportation and other relevant sub-sectors.

(2) Organizational and financial aspects:

The organizational structure has remained stable. CEFTRU continues to be administratively attached to the Chancellor's Office at University of Brasilia. Its Consultancy Council and Technical Committees hold regular meetings. On the financial side, CEFTRU's budget has increased substantially owing to the Center's capacity to attract new training, research and project activities. This likewise confirms its medium-term sustainability. Income for 2005 was ten times that for 2003.

Despite the fact that the Center has a limited budget allocation from UnB to cover only teaching staff expenses, it has found a way to boost self-generated income by charging for providing training programs, carrying out commissioned research and consultancy work outside the University, as well as obtaining financial support from contracts the Ministry for Cities, ANTT and other important public agencies operating in the urban transportation sector. CEFTRU's financial sustainability is permanently dependent of its own initiative to secure sufficient income in order to sustain and develop the Project's impact and new initiatives the Center has implemented after the Project's completion. CEFTRU is among a group of organizations that gained sufficient self-generating income by meeting demands in the market despite insufficient or reduced allocations from the Government's budget. The Center has been working on strategic actions for its sustainability in the years ahead (2007-2012).

(3) Technical aspects:

The total number of short courses has increased since conclusion of the Project. In 2002, the figures were 10 courses with 353 participants; in 2003, 8 courses with 292 participants; in 2004, 12 courses with 709 participants and in 2005, 67 courses with 1898 participants. This in fact provides evidence of CEFTRU's quality and credibility, indicating a strong likelihood that the center will also achieve long-term sustainability.

Due to requests for new short-term courses, CEFTRU has maintained high productivity in the elaboration of new materials and contents for upcoming courses. The Center has been updating technical contents since the Japanese cooperation and associated technology transfer have been assured over the last few years.

The equipment donated during the cooperation is still in use and is well maintained by CEFTRU, as is the new equipment the Institution has since acquired. The Center still maintains the necessary installations and facilities for training; however, it seems to be lacking a number of necessary pieces of IT equipment. The Center is working on a new strategy to implement distance education courses in the near future for which it will not need to increase the present number of classrooms.

CEFTRU has been working to provide services in the field of transports, with the University's support. Therefore, it has based its strategic management on current demands from the sector. The Center is able to sustain its activities due to the diversification of areas of work. It also shows great agility and capacity to adapt its management since it maintains an appropriate structure, a great majority of the technical personnel being hired temporarily for the Project.

(4) Social aspects, culture and environment:

No conspicuous impeding factors exist, resulting from a lack of considerable social, cultural and environmental factors.

3-2 Factors promoting the Project

(1) Planning Factor

The initial content and method for implementing the Project are based on: improving technical capabilities of CEFTRU staff; and implementing different levels of liaison with institutions in the Transports sector. Both forms of action were relevant to development of issues at CEFTRU for Planning and Operational Management of Training courses. These initial conditions secured the achievement of impacts and sustainability through recognition among relevant institutions in the Sector of CEFTRU as a center of excellence for training and research.

(2) Implementation Factors

1-Effective organizational management of implementation of activities, small-scale inputs, small budget, highly qualified human resources, and adequate installations and facilities;

2- Policy support obtained from the Government and relevant institutions in the Transports sector;

3-Pubic and private organizations' demands for CEFTRU's activities in the Transports sector: training; research and project development;

4- Excellent collaboration and cooperative liaison with public and private organizations in the Sector.

3-3 Factors inhibiting the Project

(1) Planning Factors

N/A

(2) Implementation Factors

Certain factors could be considered mildly inhibitive of expansion in the number of classrooms available for new courses. These include: delays in concluding construction of the second storey of the second CEFTRU building and the installation of laboratory equipment for road surfacing. However, CEFTRU has managed to get round these difficulties in implementing new courses by holding the courses at other venues in installations made available by the contracting institutions.

3-4 Conclusion

After completion of the Project, CEFTRU expanded its activities and network, increasing the number of beneficiaries of the training and research section, obtaining institutional, technical and social recognition of its capacity in the area of urban transportation and other transports sub-sectors. The total number of short courses has increased since conclusion of the Project. In 2002, the number was 10 with 353 participants; in 2003, 8 courses with 292 participants; in 2004, 12 courses with 709 participants and in 2005, 67 courses with 1898 participants. This provides factual evidence of CEFTRU's quality and credibility, ensuring strong impact and sustainability for the Center and the Project's achievements for improvement of the knowledge and skills of technicians and managers engaged in the Transports sector in Brazil.

The CEFTRU Project has contributed to transportation in Brazil in terms of developing both human resources and research activities. The expected impacts for this period have been fully achieved, and CEFTRU has amply secured its sustainability after Project completion and for the years ahead.

3-5 Recommendations

(1) CEFTRU must consider setting up exchange programs with other government institutions or the private sector. This would intensify its research, training and project development activities and strengthen its position in recommending transportation human resources policy. It would also increase CEFTRU's opportunities to be involved with practical transportation human resources training issues.

(2) It is also important to consider using CEFTRU for regular study grants for the Masters Degree Course, postgraduate specialization courses (more than 360 class hours) and short-term courses (80 class hours) from the Ministry for Cities, ANTT, DNIT, and DENATRAN. The Masters and Specialization courses are expensive and require professionals to absent themselves from their workplaces. Regular study grants from these institutions would provide these students with greater access to the Masters and Specialization courses. The needs of younger professionals who require better qualifications to enter today's labor market should also be taken into account since they generally do not have sufficient income to pay for their studies.

(3) CEFTRU must consider setting up third country training programs with other developing countries (public and private organizations) to maximize and share the experience of its training courses achieved after 04 years of JICA Technical Cooperation in this area.

(4) To enhance the effect of the Project, it could be recommended that every 2 years after conclusion of the Project experts be dispatched to update the needs and skills of Human Resources Centers like CEFTRU. Experience should be exchanged with Japanese university-based training centers similar to CEFTRU to facilitate understanding of the recipient's updated needs. The current collaboration with Nagoya Institute of Technology would be a good precedent. This could contribute to new areas of joint research for developing countries in the Japanese and Brazilian transport institutions.

3-6 Lessons Learned

(1) The most sustainable option was not to create an independent entity but rather to provide and develop knowledge and technology in an existing organization that promotes development in a particular area. Creating a new entity is not necessarily the best solution; a positive solution could be to develop and enhance the necessary capacity of an existing organization in accordance with its capacity and future improvements. It is more sustainable in the medium and long term to develop and enhance the capacity of an existing young entity than to create a new one from scratch.

3-7 Follow-up Situation

N/A