Causal risk models of air transport – Comparison of user needs and model capabilities

Aviation safety is so well developed that individual organisations cannot rely on the number of accidents as useful indicators of the safety level of their operation. Adequate control of risks requires the availability of a method to determine the level of safety as a function of the current status and of proposed or expected changes to the aviation system. Aviation safety policy plans have therefore proposed the development of causal risk models. Unfortunately however, they failed to specify or even describe such models other than in the most general of terms. Causal model development was stated as a goal in itself, without consideration of how such a model should be used. The objective of this thesis is to clarify these issues by comparing user requirements with the performance that can be delivered by various modelling techniques. The thesis answers the question what causal risk modelling adds to current safety management approaches and what the criteria are for ensuring it makes a successful contribution to safety. Experience gained in several causal model development projects (particularly for the Federal Aviation Administration and the Dutch Ministry of Transport and Water Management) are used to illustrate how a causal model should and should not be developed and used.

Chapter 2 describes what we mean with the term ‘safety’ and in what ways it can be expressed. The most relevant theories on accident causation are described, emphasizing the role of ‘normal’ occurrences in the accident sequence. These normal occurrences should therefore be correctly represented in a causal model. The influence of human behaviour and of organisations is also important and should be represented adequately.

Chapter 3 delves deeper into causality and the difference between causality and association. From this it follows that cause-effect relations can only be substantiated from specific knowledge on the mechanisms of the system. This means that substantial knowledge of the aviation system is a prerequisite for the development of a causal risk model for air transport. To avoid confusion between singular and generic causal relations, a causal risk model should represent success scenarios as well as failure scenarios. Causal chains never really end, in practice a useful criterion for the extent of a causal risk model is that it will have to include decisions and actions up to the highest managerial level of the actors that
are directly involved.

Requirements from potential users of a causal risk model for air transport are the subject of Chapter 4. A short overview of the history of aviation safety shows that the current level of safety is so high that the mechanisms that previously resulted in continuous safety improvement do not work anymore. Because each of the potential users of a causal risk model (airlines, air navigation service providers, airports organisations, maintenance and repair stations, aircraft manufacturers and aviation authorities) can influence safety in a different way and each has different drivers for improving safety, some of the user requirements are not compatible. The most important general requirements are that the model should represent to complete air transport system including dependencies, that the model should properly represent the influence of the human operator and of organisations, that the model is validated and produces quantitative and reproducible results, that the model representation is transparent and that the model is able to represent current as well as future accident scenarios.

Chapter 5 demonstrates with some practical examples how safety analyses are currently conducted. The approach followed in each example is compared to the user requirements that were derived in Chapter 4. This provides an even better insight into the performance that is required for a causal model in order to have added benefits compared to current approaches. From this chapter it follows that currently there is no standardised procedure for conducting safety analyses, each time it is performed differently. Available results from previous analyses are not fully utilised and results rely heavily on expert judgement. None of the examples fulfils all the user requirements that have been derived in Chapter 4. An effort to develop a method that meets these criteria therefore seems fully justified.

Causal risk modelling is applied for management of safety in industries other than air transport. Chapter 6 gives an overview of how causal risk models are being applied in the nuclear power industry, manned spaceflight, offshore industry, process industry, rail transport and health care and it discusses the lessons than can be learned with respect to development and application of similar models in air transport. Initially there was resistance against the application of risk models in the nuclear power industry, manned spaceflight, offshore industry and process industry, but catastrophic accidents led to regulatory changes that forced the industry to use probabilistic risk methods. Development of models and methods that were considered acceptable required a lot of time. Based on these experiences it can be expected that the introduction of causal risk model in air transport will be similarly jerkily and it could take decades before the industry has fully accepted the use of quantitative risk models. It is essential to state from the outset that the models should be used as a complement rather than as a replacement of current practice.

Several techniques are available for model representation and calculations. The most commonly applied techniques (fault trees, event trees, Bayesian belief nets and Petri-nets) and their characteristics are described in Chapter 7. Fault trees and event trees have the important advantage of simplicity. This makes them transparent but the areas of application are limited. Bayesian belief nets are better suited for the representation of ‘soft’ influences, but as a drawback are less transparent. Petri-nets can also represent dynamic situations but are even less transparent. The conclusion of this chapter is that a combination of techniques should be applied, e.g. with fault trees and event trees for the representation of the uppermost level of the model (the part which is directly visible to the user), while Bayesian belief nets and perhaps also Petri-nets should be applied, where necessary, for the correct representation of details.

The ability to provide quantitative results is one of the requirements of a causal risk model. Quantification is the subject of Chapter 8. One of the problems in modelling is finding the right units, specifically if the influence of complex factors like safety culture, safety management or non-technical pilot skills must be expressed. The credibility of the model depends on the model structure and the sources used for quantification. Important sources of information are accident investigation reports and incident reporting systems of airlines. Results from empirical studies can sometimes also be used. If correctly elicited, expert opinion can offer a quite acceptable accuracy of quantification. In addition to information on accidents and incidents, information on normal operations is required to determine the relative influence of causal factors. Flight data recorders can be an important source of this information. To use the information from existing data bases as efficiently as possible, it is important that definitions and descriptions of model elements match with those used in existing data bases. Preferably the model should conform to the definitions used in ECCAIRS, the European standard for incident reporting, but this also requires that some of the current problems of ECCAIRS are resolved.

The biggest difficulties are to be expected in representing the influences of the human operator, management and the organisation, and the many dependencies between model elements. These problems and possible solutions are discussed in Chapter 9. Solutions are the application of Bayesian belief nets for representing ‘soft’ influences and dependencies and also the definition of a limited number (33) of archetype accidents. These archetype accidents can be the backbone of a causal risk model. The human operator can be represented in the model by means of a Bayesian belief net, while the influence of management on human performance can be represented by means of generic ‘delivery systems’.

Chapter 10 addressed the possibilities for validating a causal risk model. Full validation is not possible due to the complexity of the air transport system, but limited validation of model components is feasible. Some examples are provided in the chapter.

Conclusions of this thesis, presented in Chapter 11, are that current drivers for safety improvement fall short to bring about further safety improvements. A method for safety assessment and safety analysis is required that is not solely based on accident frequencies, provides a quantified accident probability and is able to properly represent the complex arrangements within the air transport system, including human and organisational influences. The method should be transparent and yield reproducible and demonstrable valid results. Using a causal risk model as described in this thesis is able to meet those requirements and therefore is a method to identify safety improvement opportunities that will not be obtained with the current methods. Developing a causal risk model for aviation is not overly difficult; it is just a lot of work. And the majority of this work will have to be conducted by people with sufficient background knowledge of air transport. Representing the complexity of the air transport system in a causal risk model requires numerous assumptions, each of which can be challenged. The causal risk model can only be a success if these assumptions are agreed upon by all parties involved, and the air transport regulator is the appropriate lead in this process. Preferably this should be an international regulator like EASA or ICAO. They should initiate and lead the process to come to an industry standard model and to agree on how and when to apply this model. For further development of a causal risk model of air transport, it is essential to link the model directly to an accident and incident data system, preferably ECCAIRS. Feedback from the modellers to the
potential users and vice versa is required to get a better grip on user requirements. The modellers should indicate possibilities and limitations of the model, and should come-up with real-life examples of cases in which a causal risk model would be a helpful tool. This will have to be an iterative process. Support from the industry is essential to provide data
and insight, to the model developers, into operational aviation processes.

Download entire PhD thesis here: Alfred_Roelen_Causal risk models of air transport_PhD_Thesis_2008