A meta-evaluation of sustainable land management Initiatives in Senegal

Resumen

Evaluation is the systematic determination of the merit, worth and significance of a programme, initiative or intervention. As a young discipline, its empirical study is still limited, especially in Francophone Africa. Building on both the theory and the practice of evaluation in Senegal for the past decades, this study aims to identify strategies to improve evaluation practice and its usefulness for development results. It is focused on the evaluation of Sustainable Land Management (SLM) that presents specific challenges associated with the difficulty of considering simultaneously the different time and space scales of the environment, the economy and the society; and of dealing with the uncertainty and the limited quantity and quality of context data, among others. The study is based on an extensive review of the specialized literature on environment and development issues, the institutional and policy setting, complemented with semi-structured interviews with national authorities, donor representatives and evaluators. Participant observation also helped to ground the analysis and to access key grey literature and evaluation reports. Meta-evaluation (MEv) is proposed as the analytical framework to study the SLM evaluation practice in Senegal to improve evaluative knowledge. MEv is the evaluation of evaluations. The theoretical function of MEv has been developed to assess the role of evaluation in the SLM policy sector, including its adequacy and opportunity. A tailored MEv framework is applied to a set of 40 SLM project evaluation reports published since 2000, and complemented with 3 case studies. The study also analyses a parallel strand of evaluation practice in Senegal: capitalizations, conceived as participatory evaluative exercises focused on stakeholders¿ experiences, practices and learning. Results show that evaluation practice in SLM in Senegal is very heterogeneous and far from ¿sound evaluation standards¿. This is explained by constraints in the enabling environment and the institutional framework and limited capacities of stakeholders at all levels. In spite of some timid advances towards country-led evaluation, serious blockages to national ownership and evaluation utilization still persist. Although the majority of interviewees perceived SLM evaluations to be participative, this study contests their understanding of participation in most cases. SLM capitalization exercises, although far from the ideal features of this learning-oriented approach are able to engage more meaningfully with local-level actors. The set of evaluations and capitalizations analysed does not offer a coherent response to the challenges of evaluating Natural Resources Management interventions identified in the literature. For instance, they do not solve the tensions among different time and space scales or encompass a wide variety of values and perspectives about those interventions. Finally, the findings suggest that SLM evaluation is still much dominated by donor agendas and aid effectiveness concerns (accountability), with very limited efforts to promote their use for improvement or learning, and hardly any to inform national policy making. The study confirms the usefulness of MEv to guide critical reflection about real-world evaluations, surpassing the narrow conception of evaluation quality. It also allows the opening of a debate about evaluation capacities understood as the faculty to choose what, when and how interventions are evaluated. MEv could be used to promote a more active involvement of Senegalese research institutes, public administration and civil society in shaping a new public policy evaluation scenario. A broader national conception of evaluation should also encompass capitalizations and other similar approaches and foster learning organizations and institutions while promoting exchanges between applied research and project and policy-level evaluation.