Safety Culture: The First Pillar of Autonomy Assurance

My answer:“Unless people trust the technology, it doesn’t matter.” People won’t adopt technologies that they don’t trust.

Trust is central to safety. Without trust, even a proven “safe” technology will be viewed with skepticism. This is particularly true for software-intensive systems. Ask yourself: of the safety-critical software that you encounter every day, how much do you really know about it? Did you code-review that flight control software before you purchased your airline ticket? Or inspect the fault tree of that elevator before you stepped inside? Of course not. You trusted it to be safe.

Trust enables autonomy… trust of the vehicle buyer, of the passengers, and the public at large. We build that trust by the way we do things. In safety, this means a culture of rigorous development, systematic analysis and test, and safety-first mentality. These things are supported by an organization’s safety culture.  Safety culture is the wellspring of trust.

Safety Culture is a set of policies and strategies used by an organization to support safety-related developments. Safety Culture is a mindset of responsibility to the public who is ultimately exposed to the vehicle. It is deeply influenced by corporate culture and values. And in the long-term, safety culture (or the lack of it) accrues to the brand and tells the customer who can be trusted.

Safety culture means…

• making a decision to delay vehicle release, even if it means a short-term financial loss
• executing reviews of deep design details, even when “it never gave us problems before”
• using conservative assumptions whenever clear data-driven choices aren’t available
• constantly asking “what if?” in order to anticipate situations where safety could be compromised, and pro-actively addressing potential hazards.

How do we build safety culture? It’s certainly supported by good processes… verification reviews, traceable systems, standards adherence, and so on. More critically, it’s built from the top down. For employees to value safety culture, top management must value and enforce safety culture.

The inverse question is also interesting: How would we create the wrong safety culture if we tried to?  Upton Sinclair said, “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” Let’s be honest: the autonomous driving industry is growing rapidly. More and more people’s salary depends on making autonomy work.  And when judgment calls arise, we can expect pressure to claim “it works” as a natural outcome of human nature.  Fast-paced innovative developers can always point to the safety people as slow, cumbersome, not innovative, too process-driven, etc. They may say things like “Maybe that safety stuff worked before… but this is way different.  We just can’t do safety the old way. The old ways don’t work for this.” Sound familiar? There’s some truth to these statements… autonomy is different in some important ways. But to a safety professional, this sounds like a red flag.

So how do we reconcile safety culture with the need to innovate on a sweeping scale? Part of the solution is to put great people in the right places within the organization. The best people bring life to important ideas…. ideas like a safety culture. For executive managers, my message is clear: when you find those great people, put them in your safety office. Great safety people are those with overlapping skill-sets, who can simultaneously serve in the role of:

• communicator: to describe what’s needed, what is done, and what needs to be done
• technologist: to be credible with the innovators driving autonomous driving forward
• problem-solver: to find ways to “get it done,” IF they exist and can be proven safe
• integrity: to stand firm under pressure and say “No” when necessary.

People make culture. Safety culture is no different. The #1 way to drive safety culture is to empower great individuals with safety authority, and to support them in their interactions with the overall organization. In time, their example sets the standard at the rest of the organization. Ultimately their high standards are reflected in the brand.  So the winning autonomy brands must first build a strong and durable safety culture…. not as an afterthought, but as a necessary condition for success. Without this, nothing else will matter. That’s why safety culture is the first pillar of autonomy safety assurance.