Interview with Annette Schmidt on the evaluation in Ginde Beret
Do the aid projects of Menschen für Menschen sustainably improve the lives of the population? Are the activities implemented together with the people adequate and viable? Keeping tabs on our own work through ongoing monitoring and regularly conducted evaluations is an important tool for Menschen für Menschen to check whether the projects achieve the set goals or whether measures can be adjusted and further improved. In October 2013, the development policy expert and anthropologist Annette Schmidt and her Ethiopian colleague, agricultural scientist Girma Mengistu from the consultancy organization FAKT, carried out an evaluation in the project areas of Borena and Ginde Beret on behalf of Menschen für Menschen.
INTERVIEW: NINA ROGGENBUCK-BAUER
You spent four weeks in Ethiopia and visited four project regions of Menschen für Menschen. What are your impressions?
I gained an excellent insight into the organization’s work. The projects are well planned and meet the needs of the population with their diverse activities. Everyone asked confirmed this without exception. The technical aspects are of high quality.
Why does an aid organization have its work evaluated? What’s the purpose of doing this?
Nowadays, evaluation is an important component of modern development co-operation – it’s simply part of the process. A neutral view from outside is very important for an organization. The working method is consciously examined to see how purposeful it is or where, from an external perspective, there is room for improvement. At the end of an evaluation, we make recommendations that are subsequently intended to serve as the basis for ongoing observation and discussion. We also did that at Menschen für Menschen, and although we found the projects to be very good, we naturally also made a few suggestions for how we thought things could be done even better. As Robert Bosch once said: “If you cease improving, you’ve ceased being good!”
What happens after these recommendations are made?
The suggestions are discussed by all relevant parties. That includes everyone in a position of responsibility, i.e. not only the managing directors but also, for example, the project leaders, who ultimately have to turn the measures into reality on the project with their staff. This involves considering step by step which recommendations should be followed and which can actually be implemented. Not all proposals can always be realized for an organization because that naturally also depends on a large number of factors. An organization such as Menschen für Menschen, which has been active in Ethiopia for many years already, naturally also has a great deal of special expertise and knows what is possible and meaningful in Ethiopia on the basis of aspects such as cultural circumstances. One can also come to the realization that a recommendation is worth striving for, but that the organization itself does not have the relevant resources or know-how to carry them out. In this case a partner organization can be brought in – as Menschen für Menschen has been doing for many years with eye doctors from the Ethiopian ALERT hospital in Addis Ababa. In any event, it is important for a discussion process and thematic debate to take place – as a way of creating a different awareness of one’s own work.
Did any of the findings comes as a surprise to you?
When I read in the project documents that the quality of teaching had supposedly improved because of the schools built by Menschen für Menschen, I thought that was a bit excessive. The general assumption is that the quality of the teaching and not the equipment of the school determines learning success. However, it does seem to be the case that many teachers try to get posted to one of the newly built schools, and the new facilities are proving to be a source of motivation for teaching staff and pupils alike. That is clearly evident when you look at the dilapidated, dark, old schools. So it is actually conceivable that even the teaching has improved – that surprised me.
You had never been to Ethiopia before the evaluation. How can you assess development projects that you know nothing about?
Good question. I have a wealth of experience in the evaluation of projects in the field of development co-operation. I am therefore able to estimate how good the planning is and what can be achieved with a set amount in a set timeframe – as well as what can’t be achieved. I also draw comparisons with other projects and ask lots of questions. But without my local colleague, who was always giving me background information, everything would have been much more difficult.
What are the criteria for selecting the experts for an evaluation? How is an evaluation team set up?
The experts should have a wealth of experience in the field of evaluation as well as in-depth knowledge of monitoring. A mixed team is ideal; in this case, that would be a person who comes from Ethiopia, who knows the regional conditions very well and, of course, also has the relevant technical background. Another person should come from Europe and be familiar with the requirements placed on evaluations in these countries. They should be right up to date as far as the debates over measuring efficiency or evaluating with the aid of control groups are concerned. A mixed-gender team is pretty much essential.
Why are monitoring and evaluation so important? Who profits?
Everyone profits! The target group profits from an evaluation because the projects become even more sustainable. Moreover, the beneficiaries are strongly involved within the scope of participative monitoring. The aid organization profits because it achieves better outcomes with its projects – which can be clearly traced back to individual activities. The donors profit because they get more detailed information about how their funds are utilized and can be certain that they are being put to good use. Monitoring and evaluation primarily aid learning and thus help to make projects even better. Continuous monitoring delivers information about what is going well and what needs tweaking. Regular evaluations, which are carried out by external experts, enable a constructive and critical view from outside.
What are the prerequisites for a good evaluation?
First and foremost, good preparation: You should have read pretty much everything available about the project. And you need a clear idea of the objectives. This must also be clarified with the project staff in order to create a common basis for the evaluation. Then you need enough time on site for meetings, workshops and inspections, and not least the confidence of the project staff so that you don’t end up nitpicking but discuss the projects with them and adjust your perspective where necessary.
Who do you talk to locally?
We speak to a wide range of people: With the project manager, with his or her staff, with the employees of the local authorities involved with the project, with the people who do the work in the villages, i.e. the development consultants and social workers. It is especially important that we always allocate most of the time to talk to the population itself. We speak with them for as long and in as much detail as possible and necessary. We also hold a 2-day workshop at which the beneficiaries of the activities are given the exclusive opportunity to have their say. It allows us to talk and learn in depth about developments in the region. These participative workshops form the heart of the evaluation.
Following your evaluation in Ethiopia, you held a workshop on “impact-based monitoring” with the project leaders of Menschen für Menschen. What is the goal of this monitoring?
The project staff know, for example, exactly how many wells have been built and how many people benefit from them. That is good. The goal of the impact-based monitoring is to clearly identify the other effects and trace them back to certain measures. For example: Does the construction of a well mean that more girls now go to school instead of having to walk for hours to fetch water from faraway places? Or which illnesses become less prevalent because of this? Impact-based monitoring is time-intensive and relatively complex but contributes to the further development of projects and simply shows everything that has been achieved!
About the person: Annette Schmidt has worked as an expert since 1999 and carries out evaluations in areas including poverty alleviation, rural development, water supply, and education. In addition, she provides support with the setting up of impact-based monitoring systems.
Help us to sustainable improve lives in other regions in Ethiopia as well!