Ex-post Evaluation

1-1 Background of the Project

The modernization of farm management and improved social and economic welfare in rural areas through the mechanization of small- and medium-scale farmers and improved productivity are important issues in the Mexican agricultural policy. However, progress in agricultural mechanization has been slow due to, among other factors, the lack of a system of testing and evaluation of the quality and performance of agricultural machinery. Therefore, the Mexican Government, via the Secretariat of Agriculture, Livestock, Rural Development, Fisheries and Food (SAGARPA, formerly SAGAR), decided to introduce such a system, and requested Japanese government to provide technical assistance for establishment of testing methods and evaluation standards as well as training of technical personnel. Accepting this request, the Japanese Government sent study teams, and the cooperation project was implemented for 5 years from March 1999. The Project planned to equip facility and train personnel of the National Center for Standardization of Agricultural Machinery (CENEMA) located in the Valle de México experimental field of National Institute for Forestry, Agriculture and Livestock Research (INIFAP), the research institute of SAGARPA, prepare the testing-evaluation standards with CENEMA, enforce them as official standards, establish other testing laboratories in Mexico apart from CENEMA through its training course, operate the system of testing-evaluation and certification by newly-established laboratories and the National Center for Testing and Evaluation of Agricultural Machinery and Equipment (CENAPEMEA).

As follow-up cooperation after the end of the cooperation in February 2004, 3 Japanese experts were dispatched to Mexico and 4 Mexican counterparts were trained in Japan for technical improvement of tractor testing.

1-2 Project Overview

(1) Overall Goal

Agricultural machinery with appropriate performance and safety for small and medium farmers are developed and extended.

(2) Project Purpose

To strengthen evaluation test system through drafting of the methods and standards of evaluation tests as well as through the improvement of techniques and knowledge for the execution of evaluation test.

(3) Outputs

1. The types of machinery to be dealt with in the Project are selected on the results of preliminary surveys.

2. Techniques for evaluation tests are improved.

3. Evaluation standards are drafted.

4. Experts for evaluation tests are fostered.

5. Evaluation test system is strengthened.

(4) Inputs (including the input for the follow-up activities)

Japanese Side:

Mexican Side :

3-1 Summary of Evaluation Results

(1) Impact

At the time of the final evaluation in September 2003, the Project Purpose was not considered to be fully achieved. It is now considered to be achieved in terms of the number of testing-evaluation standards established. It is difficult to measure to what extent the indicator regarding the number of training courses and participants has been accomplished as no target value was either suggested. However, the indicator cannot be considered to have been met for the following two reasons: first, no training for testing tractors was held, and, second, there have been no institution that operates as a testing laboratory as a result of the training given by CENEMA, despite the fact that the training was intended for the establishment of other testing laboratories in Mexico apart from CENEMA.

As to the Overall Goal, it is considered to be achieved partially. The achievement of the Overall Goal is verified by four indicators. Nevertheless, the achievement cannot be measured because target values for each of the indicators are not set. Moreover, these indicators disregard the point of view of "small and medium farmers" referred to in the Overall Goal. For rectifying this problem, one of the four indicators was modified at the time of ex-post evaluation. The number of agricultural machines tested and certified is increasing steadily but most of them are tractors and there are few machines tested in case of other types of machinery (number of tractors cetified: 22, other machinery cetified: 2). This is due to the following three factors; 1) SAGARPA made it mandatory that farmers should purchase certified machines if he/she would like to buy a machine and use subsidies from Alianza Contigo, a national program to provide support in the agricultural sector. This is an incentive for agricultural machinery manufacturers to have their merchandise certified. Tractors represent 80-90% of all agricultural machinery purchased with subsidies provided by Alianza Contigo, and this means that it is more important for manufacturers to have tractors certified than other types of machinery, 2) machinery other than tractors requires more time and procedures to carry out evaluation tests, raising fees for both testing-evaluation and certification, and 3) small- and medium-scale manufacturers that specialize in implements are not technically capable of making quality products which are good enough to be certified. The distribution of certified agricultural machinery distributed to medium farmers seemed to be increasing but no evidence was found which indicated increasing distribution to small farmers.

(2) Sustainability

(1) Institutional and Organizational Sustainability

The "Sector Program for Agriculture, Livestock, Rural Development, Fisheries and Food 2001-2006" stresses the need for the use of agricultural machinery as a way to raise productivity. According to a SAGARPA official, the Secretariat will maintain the policy for promoting mechanization in agriculture. A new presidential administration will take office in December, 2006, in Mexico, but this will not affect the above-mentioned policy as the ruling party remains in power. Therefore, it is expected that the subsidy program will also continue under Alianza Contigo. Furthermore, SAGARPA plans to introduce assistance for machine renewal and a loan scheme for purchasing machines. Alianza Contigo is a crucial program for the promotion of agricultural mechanization and NMXs.

The CENEMA, the testing-evaluation center, has both the equipment necessary to perform testing-evaluation of agricultural machinery and staff with the knowledge, skills and experience. By the end of the ex-post evaluation, it was ready to perform the testing-evaluation for all agricultural machinery, with the exception of tractor traction. Initially, this project envisioned that testing-evaluation responsibilities would be divided among multiple institutions under the CENAPEMEA. The ex-post evaluation revealed that only CENEMA performs the testing-evaluation. There are other institutions that are interested in becoming testing laboratories. However, currently they are not functioning due to the lack of demand and infrastructure.

The CENAPEMEA never fulfilled its role as a certification organization as originally planned. However, in 2003 SAGARPA secured the budget to create a body to replace CENAPEMEA. As a result, OCIMA was established in June 2005, and began certifying agricultural machinery, except tractor traction, in September 2005.

CENEMA currently retains four core staff members, including the director and three experienced investigators. Three investigators joined the center, two in 2003 and one in 2006. Each of the new investigators is responsible for testing tractor PTO, tractor hydraulic lift and tractor ROPS, respectively. INFAP underwent institutional restructuring in 2005 and, as a result, the officials who were involved in the project during the implementation were relocated to other departments. Some of them are no longer with INIFAP. At present, an official is in charge of follow-up on CENEMA. A SAGARPA official have continued to oversee CENEMA-related affairs since the cooperation period of the Project. Some SAGARPA counterparts have now retired, but the official who supervises CENEMA was a counterpart of the project, and the present director of the OCIMA used to be a counterpart in SAGARPA.

The majority of the machinery and equipment donated during the project continue to be in use and are well maintained. However, some of the tools brought from Japan are not used as staff members do not know what they are for and how to use them.

(2) Financial Sustainability

CENEMA has two sources of funding: INIFAP and self-generated income from testing services. On the other hand, OCIMA operates on a self-sustained basis; their income originates from audits and their 15% share of the testing fee. Both CENEMA and OCIMA have seen gains since they began their respective operations, so far securing the necessary budget. Nevertheless, the willingness of manufacturers to have their products tested and certified depends to a high degree on the existence of Alianza Contigo. Therefore, it is safe to say that the ability of the two organizations to remain financially independent depends on the government policy. It is imperative that they develop additional fee-for-service activities to seek and secure a source of income that is not—or at least less—susceptible to the government policy.

(3) Technical Sustainability

The investigators can be said to possess an adequate level of technical skills and knowledge to render testing services. Japanese experts who were contacted for interviews commented that, although in some areas CENEMA still needs to polish their technique through practice, it will be able to fully function as a testing laboratory. For instance, CENEMA is now performing tests for Tractor PTO, for which technical assistance was provided last year. One of the experts stated that this was a significant advance. This illustrates their ability to not only learn but to grow. Most importantly, the expert mentioned that CENEMA staff is highly enthusiastic and hard-working. CENEMA investigators enhance their knowledge by attending workshops and maintaining overseas contacts.

Considering the nacional the policy for promoting mechanization in agriculture, the human resources, infrastructure, and budgetary development of CENEMA and OCIMA, we can conclude that the factors necessary for the consolidation of the agricultural machinery testing-evaluation and certification system have been satisfied. However, there are still several tasks to be carried out in order to develop and strengthen the system and secure the sustainability of the Project. These include Alianza Contigo and broader recognition of the importance and necessity of NMXs by manufacturers and consumers (farmers). Their importance and necessity have not been fully acknowledged in part because it has not been long since they were enacted.

3-2 Factors that have promoted project

(1) Impact

The biggest promoting factor should have been the subsidy from Alianza Contigo. SAGARPA made it mandatory that farmers should purchase certified machines if he/she would like to buy a machine and use subsidies from Alianza Contigo. And this is an incentive for agricultural machinery manufacturers to have their merchandise certified.

(2) Sustainability

The policy environment has continued to be favorable for agricultural mechanization and this has been the promoting factor the sustainability of the Project. SAGARPA plans to further expand its subsidy program for the purchase of agricultural machinery.

3-3 Factors that have inhibited project

(1) Impact

Farmers and manufacturers of agricultural machinery lack awareness and recognition of the existence as well as the importance of the testing-evaluation and certification system of agricultural machinery, due to the fact that no other measures for this diffusion were thought of besides the subsidy program under Alianza Contigo. This is considered as the principal inhibiting factor the Project.

(2) Sustainability

The standards of the testing-evaluation of agricultural machinery are voluntary standards, which means it is the discretion of manufacturers to decide whether of not to have their products tested and evaluated. At present, therefore, the testing-evaluation and certification system solely hinges on the subsidy program, which threatens the sustainability of the project.

3-4 Conclusions

As a result of the Project, CENEMA has accomplished substantial institutional development as a testing-evaluation body for agricultural machinery. Combining this with the establishment of a certifying body called OCIMA, it can be said that the testing-evaluation and certification system in Mexico has been completed.

Nonetheless, the current environment is not conducive to the system's full operability. Manufacturers do not have an adequate understanding of the importance of the NMXs and farmers do not pay much attention to the NMXs when selecting agricultural equipment. For further development of the system and fulfillment of its real function, it is vital to educate farmers and manufacturers on the need and importance of testing-evaluation and certification, motivate manufacturers to have their products tested, evaluated and certified, and make farmers understand the benefits that accrue to them from the system.

3-5 Recommendations

(1)Testing-Evaluation and certification shall be regulated for agricultural machinery manufacturers. Considering that NMXs were created following the governmental policy to provide consumers with safe and appropriate agricultural machinery and Mexico has yet to develop a ground where NMXs are effectively utilized as voluntary standards, it is necessary that the government rather than a private sector directs the agricultural testing-evaluation and certification system.

(2) In order to make safe and appropriate agricultural machinery available to farmers, it is essential that testing always corresponds to the present agricultural condition. To this end, CENEMA shall collaborate with farmers and manufactures to collect and analyze information on malfunctions, accidents and their causes and, hence, promote the improvement and development of agricultural machinery.

(3) CENEMA and the experimental fields on INIFAP provides technical assistance to small and medium manufactures in improving their products, which will lead to an increase in the number of machines tested and certified.

(4) SAGARPA and CENEMA in collaboration with other related organizations shall fortify its effort of awareness raising for farmers regarding the importance and benefits of the testing-evaluation and certification system of agricultural machinery.

3-6 Lessons Learned

(1) In the Project Design Matrix (PDM) of this Project, good performance of National Center for Testing and Evaluation of Agricultural Machinery and Equipment (CENAPEMEA) appeared as pre-conditions and important assumptions, which were not eventually met because of a lack of funding to operate and the motivation of participating institutions. It is important to carefully examine pre-conditions and assumptions of PDM. If not, it could lead to a failure of the project.

(2) The Project Purpose involved two elements: first, building a system, and second, extending the system to involve farmers so that they appreciate and make use of the system when they purchase agricultural machinery. The Overall Goal also had two elements: developing agricultural machinery that is certified, safe and appropriate, and distributing it to farmers. Both include a key component of "diffusion to farmers." The evaluation, however, demonstrated that no other measures for this diffusion were thought of besides the subsidy program under Alianza Contigo. Strategies to promote the NMXs that were suitable for the state of the Mexican agricultural equipment market were essential. If a similar project is formulated in the future, an understanding of the factors affecting the projects and strategies that are feasible in the particular environment are essential to attain the project purpose.

(3) At the time of the ex-post evaluation, the achievement of some indicators for the Project Purpose and Overall Goal cannot be measured because target values for each of the indicators are not set. Target Values are set to make the goal of project clear and helpful for the effective project management. Therefore, indicators with mesureable target values shall be specified in the future formulation of a project,

3-7 Follow-up Situation

At the end of the cooperation period, a follow-up cooperation was planned to provide technical assistance with regard to 4 parameters of tractor testing, providing that CENEMA would purchase equipment for the respective testing. Equipment of the testing of 3 out of 4 parameters were purchased and, therefore, 3 Japanese experts were dispatched to provide technical assistance (By October,2006).