CECAN Fellowship: How To Better The Evaluation of Complex Systems to Improve Policymaking – Learning from Evaluating DEFRA’s Reward & Recognition Fund (RRF)
Mentors: Ben Fagan-Watson, Ben Shaw, Candice Howarth
DEFRA’s RRF (evaluation report) tested multiple, diverse schemes in a complex macro system, with 31 schemes addressing a wide variety of challenges, behaviors, audiences, stakeholders, outputs and outcomes in the energy-environment-food nexus, with an overarching evaluation. The focus was on food waste, recycling & reuse, waste prevention, and the circular economy. The 31 pioneering schemes also operated in a complex micro context as they implemented a variety of schemes that variously assessed impact over long time periods, operated in open, unbounded system with multiple overlapping policy initiatives and substantial background ‘noise’.
The challenges with the evaluation of this scheme were:
- The disconnect between each scheme’s delivery needs versus those of evaluation;
- Assessing impact, additionality and attribution of change to rewards and recognition alone;
- The ‘tug-of-war’ between policy, research and delivery at the programme level, and
- Lack of joined-up thinking between commissioning, designing and implementing evaluation.
This fellowship aims to address these challenges specifically and, more broadly, to unpack how to better conduct evaluations of complex systems and improve evidence-backed policymaking by exploring RRF data2 and conducting interviews with practitioners, policymakers and researchers. This will result in guidance on improving the commissioning and design of evaluation, particularly those testing multiple interventions across case studies.
