The Rise of AI Tools in Evaluation – Challenges and Opportunities
This blog summarises key points discussed by delegates in the session “How might the rise of AI tools affect future approaches to evaluation?” as part of the CECAN Conference held at the Royal Academy of Engineering on 4 June 2024.
Can we generate value profiles from policy documents?
What kinds of value do policies seek to deliver? A trivial answer would be “value for money”, but this doesn’t say anything about the kinds of value that we seek to maximise with respect to expenditure.
Health CASCADE Workshop: Co-producing complex systems interventions for public health
By Brian Castellani, Professor of Sociology (University of Durham).
It’s time for the UK government to embrace systems mapping
By Pete Barbrook-Johnson, co-author of Systems Mapping: How to build and use causal models of systems (Palgrave, 2022)
Pluralistic evaluation framework published in Ecological Economics
Last month saw the publication of our paper outlining a new tool for evaluating policies and projects. This tool has been under development since my fellowship with CECAN in 2017-18, and its publication in the journal Ecological Economics is a milestone for our team.
What would a complexity-appropriate evaluation framework for England’s action on nature over the next 10 years look like?
This blog provides a discussion of the potential role for evaluation in supporting and assessing progress on action for nature and biodiversity.
Evaluation as a Pathway to Transformation for a Sustainable Future
In June 2021, the Sustainability Research & Innovation Congress 2021 (SRI2021) held a series of online workshops as part of its “Transformations Conference 2021: Enabling Positive Tipping Points”. This post shares the learning from one workshop, “Evaluation as a Pathway to Transformation for a Sustainable Future.”
Part 5 – Why we Need Locally/Nationally Interdependent Models to Successfully Exit COVID-19 Lockdown
This post is the 5th of several devoted to addressing the complex challenges of modelling the coronavirus as a public health issue. It is also about clarifying for a wider audience how and why such modelling is important, as well as the value and power of complex systems thinking and computational modelling for public health policy.
Part 4 – Social Networks and the Coronavirus: The Importance of Complexity Science for Public Health
This post is the 4th of several devoted to addressing the complex challenges of modelling the coronavirus as a public health issue. It is also about clarifying for a wider audience how and why such modelling is important, as well as the value and power of complex systems thinking and computational modelling for public health policy.
Part 3 – Simulation and the Coronavirus: The Importance of Complexity Science for Public Health
This post is the 3rd of several devoted to addressing the complex challenges of modelling the coronavirus as a public health issue. It is also about clarifying for a wider audience how and why such modelling is important, as well as the value and power of complex systems thinking and computational modelling for public health policy.
Part 2 – Modelling COVID-19: So, What Does an Effective Approach to Simulation Look Like?
This post is the 2nd of several devoted to addressing the complex challenges of modelling the coronavirus as a public health issue. It is also about clarifying for a wider audience how and why such modelling is important, as well as the value and power of complex systems thinking and computational modelling for public health policy.
Part 1 – Modelling the Coronavirus: Why All Public Health Models are Not the Same
In response to these advances, the current post is meant to be the first of several addressing the complex challenges of modelling the coronavirus as a public health issue. It is also about clarifying for a wider audience how and why such modelling is important, as well as the value and power of complex systems thinking and computational modelling for public health policy. Still, this does not mean modelling will answer all of our questions; nor does it mean that all models are equally useful!
CECAN Contributes to Norway Anniversary Evaluation Conference
Picking up on the CECAN webinar last year on the topic ‘How to evaluate – or commission – an evaluation when everything is messy’, Dione Hills (Tavistock Institute and CECAN Associate) was asked to give a key note speech last month at the Norwegian Evaluation Conference.
Complexity and Evaluation Failure
People attending CECAN training and events often want to hear about evaluations that ‘failed’ because the wrong (i.e. not complexity appropriate) methods or approach were used. Providing such examples is not easy: accounts of evaluations that ‘fail’ are rarely published, and when we have examples from our own practice, confidentiality – or embarrassment – can make it hard to talk about these in public.
Value, Values and Valuing Nature
Artists and academics and everyday people who care about the health of the planet have been doing the calculus for a long time. There is a new urgency to these calculations now, partly thanks to the School Strikes and Extinction Rebellion, and underpinned by a wave of action-oriented estimates about the state of the global ecosystem and its capacity to carry all life, including a human race projected to reach nearly 10 billion people by 2050 (UN 2017).
Slipping into Simplicity
We and our colleagues often talk a good game when it comes to complexity. We want to understand nuance and context, and we know things are messier than we realise. Yet, we still struggle to grapple with complex issues and can feel frustrated with our lack of progress, or others’ seemingly foggy thinking.
How to Evaluate Complex Research Impact
We know research impact unfolds in complex and unpredictable ways, so how on earth do we learn from and evaluate it? In this blog, I will take a look at some of the approaches we have been developing and using in CECAN – a research centre set up to tackle the issue of complexity in evaluation. I will explain how you can use these approaches to do a quick and effective evaluation of complex research impacts, helping you understand what works and why.
Complexity and Evaluative Thinking
There has recently been an upsurge of interest in what constitutes ‘Evaluative Thinking’ (ET). One frequently quoted definition of this term (Buckley 2015) refers to ET being “critical thinking applied to contexts of evaluation”[i]. This blog reflects on ways in which ‘evaluative thinking’ and the application of an understanding of complexity to evaluation can be mutually supportive.
What has Economic Pluralism Got to Do With Understanding Complexity?
To understand the complex systems we inhabit, surely, we must recognise a plurality of perspectives. After all everyone gets a different view. However mainstream economists, the dominant social scientists, only accept one perspective on how economies work drowning out others.
The Story of the Pluralistic Evaluation Framework
The pluralistic evaluation framework is a new tool for policymakers that has gradually taken shape during the last 12 months of my CECAN fellowship. It is now ready to be presented at a webinar on 20 November, where I will be explaining the rationale in the space of 45 minutes. Here I want to share a little of the journey that it has been on, building on what I wrote here last January and June.
Power and Hope: Using Evaluation in Volatile Contexts
By Caroline Oliver, Reader in Sociology, Roehampton University and Research Affiliate at the Centre on Migration, Policy & Society, University of Oxford
The Nexus: A New Approach to Sustainability Transformations – What, Why and How
This blog reports on recent presentations and discussions held at the Royal Geographical Society annual conference in Cardiff, at a special session on adaptive management and governance of the food-energy-water-environment nexus. The session was jointly organised by researchers from CECAN and the James Hutton Institute.
CECAN Fellowship Blog: Evaluative Methodologies for the Transition Towards an Institutional Ecosystem Approach ‘System-Cultures’ Within Natural Resources Wales and CECAN
Many public natural resource management organisations are trying to institutionalise integrated approaches to natural resource management. These new approaches hold the promise of allowing them to better meet the complex institutional, policy, and natural environments in which they operate.
A Pluralistic Evaluation Framework
I wrote at the start of this year about my work on articulating values in evaluation. This has crystallised into a pluralistic evaluation framework, which I will shortly be presenting to policymakers from Defra and other agencies (in London on 12 June and in York on 3 July). This workshop is intended to offer fresh ideas and a simple template to assist at all stages of the policy cycle – from appraisal and impact assessment through to full evaluation.
The Defiance of Global Commitment
The following interview was between Phil Haynes (Professor of Public Policy, Brighton University) and Brian Castellani (Professor of Sociology, Kent State University) regarding Professor Castellani’s latest book, The Defiance of Global Commitment: A Complex Social Psychology, which is part of the series he edits, complexity in social science at Routledge.
Looking Through a Glass Darkly: Public Goods and Agricultural Policy
This blogpost highlights some of the points raised in a policy brief, New Directions: A public goods approach to agricultural policy post-Brexit. Following the Brexit referendum, the mantra “public monies for public goods” has been increasingly heard, especially in relation to agricultural and environmental policy.
Humility and Courage: Strategy and Evidence in Our Complex World
Here in CECAN, a lot of time is spent considering and analysing policies, projects and interventions. We also think about methodologies, approaches and systems that we can adopt to do this analysis in the best way possible.
Dynamic Pattern Synthesis for Modeling Complex Systems. An Interview with Phil Haynes
Phil Haynes is Professor of Public Policy and researches and teaches public policy and management, as applied to a variety of contemporary circumstances. His research focuses on the application of complex systems theory, often using applied statistical methods. His research has been funded by the ESRC and the government and voluntary sector.
Finding The Common Ground
Along with several lead and co-authors, as part of a British Ecological Society Agricultural Ecology Group convened workshop held in December 2017, a report has been produced entitled “Finding the Common Ground”, which sets out an ecological perspective on how future agricultural policy should develop as a consequence, and in relation to, Brexit.
Carillion May Have Collapsed, But Public-Private Partnerships Can Be So Much More…
Last month, Carillion, one of the largest companies in the UK which regularly entered into contracts with government to deliver public infrastructure and services, went into liquidation. Since then, public-private partnerships (PPP), and their pantomime villain superstars – private finance initiatives (PFI) – have received an unprecedented level of criticism. The Guardian Opinion section – and my love-hate relationship with it – has gone into overdrive!