Handbook on Impact Evaluation: Quantitative Methods and Practices

This book reviews quantitative methods and models of impact evaluation, presenting an analysis of the quantitative research underlying recent programme evaluations and case studies prepared for a series of impact evaluation workshops in different countries, sponsored by the World Bank Institute (WBI). The handbook also details challenges and goals in other realms of evaluation, including monitoring and evaluation (M&E), operational evaluation, and mixed-methods approaches combining quantitative and qualitative analyses. It is designed to put theory on evaluation methods and practices into practice in a hands-on fashion for practitioners, especially researchers new to the evaluation field and policymakers involved in implementing development programmes worldwide.

The authors outline and discuss econometric methods for designing and evaluating programmes; these include experimental and non-experimental approaches, as well as methods for examining distributional and structural impacts of programmes. Discussions and recent case studies are presented for each chapter that draw on the practical experience of programme officials and researchers in implementing different initiatives including policies on schooling, microfinance, health, and infrastructure.

Specifically, following an introductory chapter, the handbook is organised as follows:

• Chapter 2 reviews the basic issues pertaining to an evaluation of an intervention to reach certain targets and goals. It distinguishes impact evaluation from related concepts such as M&E, operational evaluation, qualitative versus quantitative evaluation, and ex-ante versus ex-post impact evaluation.
• Chapter 3 focuses on the experimental design of an impact evaluation, discussing its strengths and shortcomings. Various non-experimental methods exist as well, each of which are discussed in turn through chapters 4-7.
• Chapter 4 examines matching methods, including the propensity score matching technique.
• Chapter 5 deals with double-difference methods in the context of panel data, which are designed to relax some of the assumptions on the potential sources of selection bias.
• Chapter 6 reviews the instrumental variable method, which is, according to the authors, designed to further relax assumptions on self-selection.
• Chapter 7 examines regression discontinuity and pipeline methods, which exploit the design of the programme itself as potential sources of identification of programme impacts.
• Chapter 8 presents a discussion of how distributional impacts of programmes can be measured, including new techniques related to quantile regression.
• Chapter 9 discusses structural approaches to programme evaluation, including economic models that can lay the groundwork for estimating direct and indirect effects of a programme.
• Chapter 10 discusses the strengths and weaknesses of experimental and non-experimental methods and highlights the usefulness of impact evaluation tools in policy making.

For researchers interested in learning how to use these models with statistical software, Part II of the handbook explores Stata in depth, providing exercises in the context of evaluating major microcredit programmes in Bangladesh, such as the Grameen Bank.