Promoting Inclusive Markets and Financial Systems
Letting go of causal theories – market systems and complexity
Author: Marcus Jenal, Consultant for Systemic Development and Complexity
As part of our Systemic M&E Initiative, we are releasing a series of podcasts. We start with an interview with a champion in applying complexity theory and the mental father of the Cynefin framework: Dave Snowden. Have fun listening. Your feedback is very welcome!
In December 2012 I had the pleasure to record a podcast with Dave Snowden, founder and chief scientist of Cognitive Edge, a research network that focuses on the development of the theory and practice of sense-making. The podcast was followed by an interesting and very engaged discussion on MaFI where members contributed with questions, examples, and experiences. Listen here: http://www.seepnetwork.org/systemic-m-e-interview-1--dave-snowden-pages-20182.php
The podcast: no linear-causality in complex systems
For Snowden, development is much about managing systems where you don’t know in advance what a good outcome would be. Local people often have different views on what’s needed from development actors or funding agencies than people at head offices. It is difficult to determine in advance exactly what success would be.
Current monitoring and evaluation frameworks are built around predefined outcomes and based on assumptions of linear causal connections between a project's activities and changes at its outcome and impact levels.
According to Snowden this outcome-based approach has not only lead to poor results in industry and government, but it is also based on bad science. Conflating measures and targets leads to non optimal solutions, as expressed in Goodhart’s Law in economics and its simplification by Marilyn Strathern: “When a measure becomes a target, it ceases to be a good measure”. This was also confirmed by a review of all studies on human motivation conducted by the New Scientist who shows that extrinsic rewards destroy intrinsic motivation. If you define specific outcomes and reward their achievement, people will achieve the outcome no matter the consequences.
Based on current research, the assumption of linear connections between cause and effect do not reflect the reality of complex systems, such as markets or communities. To measure changes in complex human systems, Snowden proposes to use narrative. Narrative is a way to get into the underlying reasons why people do things rather than relying on stated orthodoxies as in traditional research. Using people’s narrative and their own interpretation of it allows for the quantitative and qualitative analysis of a large body of data. Consequently, Snowden proposes projects that focus on changing the underlying narratives of people’s behaviors.
The discussion: bottom-up action guided by a clear logic
This discussion on MaFI that followed the release of the podcast picked a couple of points Snowden made and added interesting aspects to them. For example, while people appreciated that it is important to use people’s own interpretation of their stories, reality shows that in field research it is not easy to rely on local narratives without providing your ‘expert interpretive spin’ to it.
Anuj Jain picked up the perverse incentives that current outcome based reward systems establish for development projects. As he pointedly put it: “From my own personal experience in the last 25 years programs that really 'worked' often hardly got the recognition they deserved and the those [sic] which hardly worked but helped 'tick' all the boxes; received great attention and reviews!“
James Delaney pointed out the importance of not forgetting lessons we have learned in other projects, rather than re-starting experimentation for every project: “We have good lessons and bad, and we really need to design projects that take these into account. Taking a systems approach, I think, does not excuse you from sketching out a viable theory of change that draws on some of these lessons (or multiple competing theories of change) and articulating and testing our assumptions. This may be a little linear, but it need not be thought of as existing in opposition to a systems approach to development learning.”
James and Mary Morgan discussed the challenges of starting a project in a complex system when, as Snowden pointed out, we don’t know what a good outcome should look like. As James put it: “I think that we all end up falling into causal thinking whenever we work to change something. I am doing X because I hope that Y will result. If we have no built in theories regarding what our actions will bring about, then how do we chose actions in the first place? The systems challenge is to ensure that we do not assume linear relationships, and that we postulate that there may be multiple and divergent outcomes from our actions. In this manner, we can learn more about the system (in a structured manner) through the course of action.”
Mary also pointed to the importance of using good research practices and tools when doing market analyses. She particularly pointed out that it is dangerous to just rely on the views of the individual actors and that institutional ethnography could add significant value to get a systemic view on the market. Also, she points out that she likes Snowden’s Cynefin framework and how the line between simple and chaotic is like a cliff – meaning that that if we impose a (simple) causal outcome, the whole facilitation effort can end up in chaos.
The debate since has not changed much: while practitioners appreciate the lessons and insights from complexity theory, there is still a reluctance to step away from developing linear chains of cause and effect to formulate program logic. Personally, I think that we should continue using a theory-based approach to programming when it helps us conceptualize our activities. But we should also continue experimenting with other approaches like the one introduced by Snowden where change emerges from our interaction with the system rather than being planned in advance. We can plan only as far as our imagination goes while change that achieves real impact at scale often comes about in unanticipated ways.
Other contributors to the discussion were Ekanath Khatiwada, Jayantha Gunasekera and myself. Thank you again for making this a great discussion.