Three Approaches to Evaluation: Evaluation Evolution?

This seven-page article from The Broker magazine aims to help policymakers, practitioners, and scientists make better use of research and evaluation (specifically in the fields of poverty reduction and international development) and uses relatively advanced language. It consists of three sections: evidence-oriented evaluation, realistic evaluation, and complexity evaluation. The introduction explains the important point of how evidence-oriented evaluation has been the dominant approach to date but theorizes a trend away from that modality towards realistic and complexity evaluation. Section 1 defines evidence-oriented evaluation as an approach that "aims to find measurable changes that can be directly attributed to specific policies… and uses 'experimental' research methods." It describes the shortcomings of this approach, particularly the lack of reconstructing and testing policy theories and their underlying assumptions. Section 2 defines realistic evaluation as "assess[ing] how policy is 'received' under certain social and cultural conditions and how it triggers a response from people." It outlines a two-step process for realistic evaluation and explains how the exploration of contexts and mechanisms undertaken in this kind of evaluation does not easily lend itself to policy making because of its indirect nature. Section 3 presents complexity theory as "view[ing] policy as a dynamic system and try[ing] to evaluate how policymakers respond to complex problems. This approach is a reaction to realistic evaluation." It goes on to explain three dimensions of complexity theory. The conclusion makes a convincing case for complexity theory by pointing to the flaws in a linear cause-and-effect approach and how those models do not work well for areas such as civic engagement. Two helpful graphics are included: one on causal chains and the other summarizing the relationships of the three evaluation approaches.