In designing and implementing safety systems, risk assessments are made to identify and mitigate unsafe situations so as to ensure a certain level of safety is achieved (a level of risk is not exceeded). For traditional railroad signaling systems, each supplier in the North America has developed its individual qualitative approach referred to as V & V (validation & verification) for evaluating their respective systems. That is, the V&V process is meant to validate that the right thing is being done, and then verify that it was done correctly. For electrical / mechanical components and systems, such an approach makes sense. But, when a most complex and highly unpredictable variable such as the human is introduced as part of the system, then the V&V process is not sufficient; the risk assessment process becomes much more risqué.
The design and implementation of Positive Train Control (PTC) has taken the traditional signaling suppliers outside of their comfort zone for risk assessment. With PTC designed to prevent the failures of humans to operate their trains within the limits of the active movement authorities, means that a qualification process has to be complimented with a quantitative process as well. But, if humans are so unpredictable as to both the types and occurrence of errors that can be made, then how can even a quantification process be established? Actually, the process is quite straightforward. It’s a matter of simulating the environment to be evaluated over an extensive period of time and/or iterations, and to use historical data as to the type and degree of threats that may occur. The reason for the extensive time period and/or iterations is to provide for the randomness of events so as to ensure a statistically sound analysis.
Risk relative to evaluating PTC was defined by the Railroad Safety Advisory Committee (RSAC) to be the severity multiplied by the likelihood of the train being coincident in time and space with an unsafe condition. RSAC was composed of a mixture of regulators, rail management, labor, and supplier personnel, and one of their responsibilities was to evaluate a risk assessment process that was being specifically designed for PTC. Referred to as the Axiomatic Safety Critical Assessment Program (ASCAP), this tool was to be a very straightforward simulation program that could have readily provided a more than adequate analysis of PTC reducing risk – which everyone already intuitively understood anyhow. I mean, if PTC eliminates the most dangerous source of train accidents, again human errors, then it’s a winner (assuming it doesn’t introduce any significant risk – and it doesn’t). Of course, the regulators can’t accept intuitive analysis. They need the mathematical proof, and hence ASCAP.
You noticed that I said that ASCAP could have been a great tool. But, it failed to be delivered due to extremely poor management of resources. I am not referring to ASCAP’s developers, but rather to involvement by the RSAC participants that continuously battered the developers with “insights” and additional requirements of how to make the ASCAP simulate a railroad to the greatest exactness possible. What they failed to understand was that the error associated with simulating human-based events was much greater than correcting for the acceleration of a sample train from a railroad yard, for example.The bottom line here is that the RSAC advisors who were lacking in sound mathematical principles, including Operations Research (OR), and simple pragmatic analytical tools turned a straightforward simulation tool into an unachievable, complex quagmire of code. What was missing was a manager experienced in OR with railroad domain knowledge that could have separated the RSAC’s advisors appropriate advice from the fatuous comments.
ASCAP failed due to poor management and not due to its concepts or principles. Simulation is a quantification risk assessment approach that eliminates the risqué-ness in risk assessment processes involving humans.