Interview with Jochen Currle on the evaluation in Abune Ginde Beret
Evaluations by independent experts regularly serve to verify our project measures on site, especially for how they affect the population. When the first three-year project phase ended in Abune Ginde Beret in 2014, evaluator Jochen Currle traveled to Ethiopia to examine the projects on the ground together with his Ethiopian colleague Chali Guteta from the consulting organization FAKT. Naturally, this also focused on the goals that Menschen für Menschen had set at the beginning of the project for further developments in Abune Ginde Beret.
Mr Currle, how would you describe a typical day of an evaluator working on the project?
Usually, each day is packed with visits and meetings in villages or with farmers out in the fields. This includes talking to individual people such as farmers, advisers or administrators, with families, and certain groups such as the users of a spring or parts of whole settlements. In the process, I gather huge amounts of impressions and information, which I usually structure in the evening in order to separate the important from the unimportant and to compress the information into specific hypotheses. The best thing is when I have a couple of key findings in my notebook at the end of the day, along with questions that absolutely must be answered over the next few days.
How did you personally experience the work on the ground? Did you face certain challenges or obstacles when working with the team?
No, the team was incredibly open and prepared to think things over. It’s not unusual to have agreed an inspection program with the project team only for something interesting to unexpectedly come up. You then want to investigate more closely and the whole schedule goes out of the window. The fact that the team from Menschen für Menschen patiently went along with such detours and also organized appointments at short notice was very positive.
When is collaboration with the organization particularly required?
The organization’s co-operation is required for the entire duration of an evaluation. The best prerequisite is for the employees to have an open and interested attitude as well as the desire to learn and improve their own work. The most important criteria in this respect is the transparency and the willingness to make information and contacts available in an impartial and open way. Such an attitude makes an evaluation an exciting journey of discovery, on which all participants can think about why things happened the way they did, what a certain result means, and what lessons can be learned for the project’s future.
Is it even possible to talk about results in the context of an evaluation, or is it more about taking a snapshot?
During the evaluation, we try to capture and evaluate the results and outcomes of projects. As a rule, we do this by systematically applying certain tools and methods in order to bring the necessary information together so we can say something about previously defined indicators. These statements then become our results. Of course, these results depend on various factors: whether we really get relevant information, whether we can verify the information and perhaps discuss it with others, and much more besides. That means: Yes, an evaluation is a snapshot that is taken in a limited amount of time. But it’s a snapshot whose assertions are explained and made verifiable by means of the most systematic procedure possible.
Evaluations are also partly about measuring the success of a project. But what does success actually mean in this regard?
Not very easy – because various levels and perspectives are involved. In very formal terms, you could say: A project is a success when it meets the OECD/DAC criteria (more on the criteria here >>), i.e. when it has achieved the planned objectives for a reasonable cost. It can only do so in the area of rural development when these objectives are important and coherent for the largest possible number of actors and target groups and when the right approach is chosen. Only when that applies to a large extent will the project’s interventions also have a long-term and sustainable impact.
You could also say: A project in the area of rural development is a success when the people involved are prepared and able to take their own lives in hand and to achieve the shared goals that they set for themselves as a community.
How can this success be made measurable? Are there measures for which this measurability comes more easily than for others?
Everything that can be counted, weight or compared is relatively easy to measure, i.e. revenues, income, number of users of a particular practice. However, there’s also the difficulty that when I’m carrying out an evaluation, I should know exactly what I want to understand by it, which criteria for success I have to set, and how I want to measure them. So, if a project goal were to improve the economic situation of the population, I could measure the success of it by looking at household incomes, for example. However, without knowing what these incomes were like before the project, I can’t say anything about the success defined in this way.
This exercise becomes even more difficult, of course, when it involves such things as the “ability to help oneself” or “equality”. In such cases, I need to properly consider beforehand how I can prove or measure these abilities or circumstances, making the search for indicators more difficult.
Beyond that, how difficult is it to make a direct impact of the measures measurable?
It comes down to me being clear about the impacts I am actually striving to achieve with the project. The measures involved in preparing and planning the project are based on this. Moreover, I aim to establish – as early on as possible – how and against what I want to measure the impacts, whether these impacts occurred, i.e. I have to search for suitable parameters or indicators that say something about the impact. For example, if it is important to me that levels of health in a region improve, I will determine indicators such as the average or the occurrence of certain, typical illnesses (diarrhea illnesses, trachoma, maternal mortality during birth or labor). If it is important to me that the nutritional situation improves, I will, for example, trace the change in eating habits – what was on the menu for most families in year X and what is on the menu for most families five years later. As you can see from the indicators, some are really easy to survey while others are relatively complicated.
Menschen für Menschen is implementing a package of measures in whole regions – what are the biggest challenges of such projects?
What makes the projects of Menschen für Menschen so interesting is also its biggest challenge: their complexity and the interaction of diverse measures towards achieving a single objective. In order to achieve an improvement in a region’s health, wells are built, courses on growing vegetables and cooking are held, health stations are set up and staff trained, the construction of latrines initiated, and so on. Coordinating these measures requires a high degree of communication and collaboration within the project team. If this does not exist, things get out of sync and in the worst case are prevented from happening at all. However, if they are coordinated – and the project teams of Menschen für Menschen do everything they can to ensure this happens – the effects can be amazing.
Is there any one experience that particularly moved you?
I was particularly moved by the vitality and energy that the project unleashed, especially in women. This became clear to me especially during the final workshop, when delegates from almost all kebeles (municipalities, ed.) came together to discuss the effects of the project one more time. The strength and joy with which the women in particular engaged in the discussion made clear how important it is to them to be able to send their children to school, to be able to take control of their own lives, that was so moving.
About the person: Jochen Currle has worked in the area of rural development and consulting for 25 years. During this time, he has developed, planned and implemented a range of projects for and with people in rural areas. Against the background of this experience, he has been increasingly active in the areas of project consulting and evaluation at FAKT in recent years.