Wednesday 5th September, 12:45 – 14:00, BEIS, 1 Victoria Street, London SW1H 0ET
Presenter: Adam Stiles, BEIS
Seminar Overview:
This seminar, led by Adam Stiles, will examine the application of process tracing methods within a realist evaluation.
The evaluation of the ‘Transitional Arrangements (TA) for Demand Side Response’ scheme was commissioned by the Department of Business, Energy and Industrial Strategy and has been led by CAG Consultants in partnership with Charles Michaelis (CECAN).
Our approach to process tracing has been informed by the work of Dr. Barbara Befani (CECAN). When evaluating the first TA scheme, we applied ‘process tracing with Bayesian updating’ to test the additionality of outcomes generated by the scheme – we described this as ‘contribution tracing’ to reflect the fact that we were testing for the contribution of the first TA to observed outcomes. This involved testing additionality hypotheses for each organisation participating in the first TA scheme, using a set of evidence tests for each hypothesis.
This process tracing contributed to our overall assessment of additionality, which used realist contribution analysis. However, the additionality hypotheses and evidence tests did not directly relate to the ‘Context-Mechanism-Outcome’ (C-M-O) configurations within our realist theory. When evaluating the second TA scheme, we attempted to use process tracing to test the individual ‘C-M-O’ hypotheses in our realist theory. This was challenging because of the large number of possible C-M-O configurations that could be tested. We have therefore applied process tracing in a more limited way when evaluating the second TA, using process tracing concepts (e.g. ‘hoop tests’, ‘straws in the wind’, ‘smoking guns’) to assess the weight of evidence supporting our C-M-O analysis for particular cases, but without assigning numerical probabilities or using Bayesian updating.
This seminar will discuss learning from these two approaches to undertaking process tracing within a realist evaluation.
How to Join:
You can register for free via our Eventbrite page.
*Please arrive 12.45 for registration and bring your own lunch and refreshments. There will be time for Q&A at the end of the session.