There are two major forms of stupidity on the various debates that sprint on social media from time to time. The first is the assertion that because complex systems have no linear material causality that all forms of root cause analysis should be abandoned. the second is to assert that root cause analysis (RCA) is sufficient unto the day and will actually discover ‘causes’ in such a way as to prevent future failure. There is a variant on the second of these which insists that if you can’t prevent failure it is a result of a failure to implement proper discipline in programme execution. It could also be argued that it is stupid to engage in debates on this subject with engineers whose life work has been massive, money no object, government procurement projects and think their experience of that restricted field is or should be universal. It might also be argued that to so engage is to demonstrate noble purpose and to date, I am not sure which is correct.
Now I am not going to get into a detailed analysis of the various types of root cause analysis and or their underlying ontological assumptions. I am concerned here with practice and utility. Many methods and techniques that evolved based on poor ontological or epistemology assumptions may continue to have high utility, and getting the science right may extend that. The late great Terry Pratchett received various forms of advice when he was dealing with the horror of early-onset Alzheimer’s said: Some of them wanted to sell me snake oil and I’m not necessarily going to dismiss all of these, as I have never found a rusty snake. I hope that I would maintain that sense of humour in the same circumstance. I also love his quote about hope, one we all need in these difficult times: You can’t out-run the train but you can run and, who knows, if you can keep running for long enough, someone might find a way of blowing up the train.
I should also confess to, or advise readers that over the years I have engaged in multiple RCA sessions as auditor, manager, guilty/innocent party, and Director. All of those give you different perspectives on the process. But at its heart, the process is pretty clear. First, you define the problem, then you create a timeline showing connections between events. Once you have that you seek to establish the difference between cause and coincidence remembering at all times the danger of the Post hoc ergo propter hoc fallacy of believing that because something followed something else there is a causal relationship. The final representation can take various forms and most of my experience is in path diagrams, but I know that the statistical side of this had advanced since I was last involved in the practice.
Now that is when something as gone wrong, but there is always a more proactive approach where side casting techniques such as our The future, backwards come into play along with Pre-mortums, Positive Deviance and other techniques. Preventing failure before it happens is as, if not more, important than preventing the repetition of failure. Here a lot of my work in Cognitive Edge over the last few decades has focused on attitude mapping in health and safety as well as project management and the like. Attitudinal measures (and with SenseMaker® we can make those quantitative and longitudinal) are lead indicators while compliance and RCAs are lag indicators. Remember my work here came from the highly mission-critical and consequential area of counter-terrorism (DARPA’s Genoa II programme) where you really want to avoid retrospectives. Our focus in that work was on the human factors involved in weak signal detection and as most readers know we took a natural science approach to the problem rather than simply working on cases and experience.
Now while something can have utility, we need to be aware of its limits. The narrowing down of pathways through analysis contains some of the problems of Delphi techniques, where weak signals can be eliminated early. There is also a danger of not spotting a catalyst in the ‘causal’ chains as you construct them There are also problems with compliance, where new rules and practices often deal with the centre of a normal distribution when the real experiences are in the tails of a Pareto distribution where the tails are a lot fatter. We’ve shown in work with Blue Light Services in the UK that the regulations to prevent failure may actually be the root cause of mental breakdown in the emergency service staff. The only thing we know for certain about a complex system is that whatever we do will have unintended consequences. That doesn’t mean that we don’t legislate for the norm, but we also (as in the case of military combat) have to have a rule about rule-breaking and the ability to exercise heuristic-based control under conditions of extreme uncertainty
The reality is that the word root is very important in RCA and we don’t talk about cogs. The point of my banner picture with the entangled root system of a mangrove swamp is that it is not the neat clockwork of the opening picture. Life in human systems, especially those with large numbers of human interactions is deeply entangled and our ability to sort out the entanglement is limited. A path diagram is hugely valuable as long as you realise that it is a partial abstraction and cannot represent the totality of the system in practice. In Cynefin terms the ordered domains are like clockwork, RCA is sufficient unto the day; the complex is entangled and you need to do more, but RCA still has value. I’ll come onto liminal aspects of Cynefin later.
So let’s look at some of the science. and I want to look at two key facts and I will do that with two, the first from Brian Arthur is a good summary of complexity:
..Complexity is looking at interacting elements and asking how they form patterns and how the patterns unfold. It’s important to point out that the patterns may never be finished. They’re open-ended. In standard science, this hit some things that most scientists have a negative reaction to. Science doesn’t like perpetual novelty.
or, as I put it a complex system is dispositional and has propensities, but it does not have linear material causality. Again to be clear RCA has a role and I will come to that in a minute.
The second and key quote really hits the whole idea of retrospective analysis as being in any way complete (which does not mean that diligently practiced it is not both valuable and essential):
What about expert searchers who have spent years honing their ability to detect small abnormalities in specific types of image? We asked 24 radiologists to perform a familiar lung nodule detection task. A gorilla, 48 times larger than the average nodule, was inserted in the last case. 83% of radiologists did not see the gorilla. Eye-tracking t that the majority of the those who missed the gorilla looked directly at the location of the gorilla. Even expert searchers, operating in their domain of expertise, are vulnerable to inattentional blindness.
Drew, Vo & Wolfe (Psychol Sci. Sep 2013; 24(9): 1848–1853)
Partly in response to all of this I created some years ago, when working with Gary Klein in Singapore the SEE-ATTEND-ACT model which says that the fact we see the data does not mean that we will pay attention to it, and even if we pay attention we may not be willing or able to act on it. Whether this is right or wrong is another matter, but it reality and if we want to reduce, and mitigate the effects of, failure then dealing with reality is a part of things.
It is also important to realise that even in a deeply entangled system there are some cause and effect pathways – we need to be flexible here. Learning can be achieved in many ways and excluding more traditional and structured approaches is almost as bad as claiming those approaches are all that is needed. We also need to establish, where possible facts. As some of my colleagues know I get fed up to the back teeth with dealing with feelings where no examples can be quoted to support the validity of said feelings – I am making this statement in a business context to be clear. But feelings and attitudes, whatever the frustration, are a part and parcel of project management and more or less all human endeavor.
So what can we do differently, or rather in conjunction with RCA and also with the various anticipatory techniques?
- Constraint mapping can be carried out in conjunction with RCA and like techniques, in fact, it is better done first. Map what constraints were in play and how they changed over the course of the lifetime of the project before you even think about attribution of cause and effect. The constraint map will tell you what affordances were available in context for the decision-makers and changing those may give you more success in the future.
- Shift to real-time lessons learning capture during projects to avoid the danger of retrospective coherence when the analysis is done with the benefits of hindsight. That will also give you the means to train algorithms to spot similar patterns in the early stage of their emergence in live projects and intervene accordingly
- Dark constraint mapping is a risk measurement technique in which you look at what percentage of actual events can be accounted for fully by actual constraints or variations there in – the balance is a measure of risk and also a measure of the level of redundancy you need to build in going forwards to handle uncertainty.
- Understanding RCA in terms of ASHEN shifts you to the right level of granularity in understanding what did or didn’t work
- Attitude mapping is key throughout and gives you a lot of the missing elements in the most formal system and also opens up vector theory of change
Yesterday’s post has links to ASHEN and Constraint Mapping by the way. Attitude mapping is a SenseMaker® capability that uses fitness landscapes from complexity theory. I haven’t directly linked but happy to talk people through this
Now in addition to this, we have a range of techniques that surround the liminal and aporetic aspects of Cynefin. I am going to come back to that in future posts linked to the about to be published Field Book to managing in Complexity (and Chaos) a shared effort between the Joint Research Centre (JRC), the European Commission’s science and knowledge service, and the Cynefin Centre. One of the most exciting events in the last few years so I am simply trailing it here.
And a closing quote of relevance from Lewis Carroll:
It’s no use going back to yesterday, because I was a different person then
Just think about that ….
Also, this is the first post on the subject, I plan more including specific mapping onto Cynefin domains and in particular the liminal and aporetic aspects
The banner picture of mangrove roots in my favorite countries, Colombia is cropped from an original by Maria Jesus Errazuriz obtained from Unsplash
The ‘golden cogwheels’ opening picture is cropped from an original by Miguel Á. Padriñán obtained from Pexels