An Introduction to Verification and Validation Testing for ADAS
An Introduction to Verification and Validation Testing for ADAS
The Safety Of The Intended Functionality (SOTIF) conveys a specific view on how automotive systems should be verified and validated as being functionally safe. It is not enough to just claim a system is safe. You must also:
This must be accomplished for every system on the vehicle that can impact safety in any way.
As stated in International Standard ISO/PAS 21448, Road vehicles — Safety of the intended functionality, for a road vehicle to achieve an acceptable level of safety, unreasonable risk must be avoided—not diluted, not minimized, but avoided entirely. This includes every hazard associated with both the intended functionality of the vehicles and any unintended functionality resulting from its use in the real world. The difference between intended functionality and unintended functionality can be significant.
The following is a real-life example that illustrates the difference between intended functionality and unintended functionality, and how safety can be radically impacted:
It is a warm summer day, perfect for yard work. A vehicle owner loads the bed of his mini truck with grass clippings to transport them to a community recycling center on the other side of town for processing into mulch. He has a full load but has not exceeded the weight limit of his vehicle. The load is balanced evenly across the bed. Because the clippings are fresh, they are sticking together and are not likely to blow out of the vehicle, so the load is left uncovered for the stop-and-go trip across town. He believes that he has done everything reasonably expected to be done to safely transport the load. He departs.
Because the truck is not being pushed beyond its design parameters, it handles as expected. On the way to the recycling center, it begins to rain softly in a steady summer drizzle. The driver switches on his intermittent wipers and slows down a bit for the conditions but notices no changes in the vehicle’s handling that would raise a concern. He continues his journey, driving the truck as he normally would in this type of weather.
Unbeknownst to the driver, the conditions are changing rapidly for both the road surface and the vehicle.
This is the first rain in a while; aided by the warm summer temperatures, road oil has been working its way up to the surface of the asphalt for weeks. With the application of a steady mist of rain, that oil has lifted from the road surface and now sits atop billions of beaded water droplets, acting as a lubricant and creating a deceptively slick surface. Onboard the vehicle, the grass load is soaking up rainwater like a sponge and becoming very heavy, especially towards the back of the bed where a concentrated amount of water is being blown onto the load by the curling slipstream of wet air flowing over the cab.
A quick glance in the rearview mirror does not indicate any visible change in the load; after all, damp grass looks the same no matter how much moisture is in it. But the driver is focused on what might be blowing out of the bed, rather than what is blowing in. Water is heavy. Across the span of only a few miles, dozens of gallons of water are soaked up and the weight at the rear of the bed increases by hundreds of pounds, especially right against the tailgate. Without the movement of a single blade of grass, the center of gravity of the vehicle has been radically shifted backward, beyond the safe design limits of the mini truck.
An unsafe amount of weight is now on top of and behind the rear axle at the point where the vehicle is the least stable, while the front wheels now have a fraction of their original weight load pressing down on them and are barely maintaining a grip on the road. A reduction of traction on the front wheels is particularly dangerous, as the front brakes typically do most of the work of stopping a vehicle. And, of course, the front wheels are also used to steer the vehicle. No traction—no braking. No traction—no steering. No traction—no control.
These changes are imperceptible to the driver while he is proceeding smoothly through town on a flat highway in a straight line. There are no flashing warning lights, no buzzers, no vibrations, and no telltale sensations of reduced traction being fed back to the driver through the pedals or steering wheel. The light application of the brakes at the slower in-town speeds as he lazily coasts up to an occasional stop light does not reveal that anything is amiss. The driving characteristics of the vehicle and the conditions of the road have radically changed, but none of this is apparent to the driver. He has no idea that he is now operating a highly unstable and dangerous vehicle under hazardous road conditions.
The driver approaches the recycling center and needs to slow his speed in order to turn right at the intersection. As he applies the brakes and just begins to make the turn, the front wheels—which are now barely in contact with the slippery road—lock up in a front-end skid. As the vehicle begins to rotate to the right, the front tires become oblique to the path of travel for a brief fraction of a second, then regain just enough sideways grip to drag a bit and create a pivot point slightly to the right of the highway lane. In an instant, the driver immediately loses control of the vehicle and becomes a helpless passenger. The heavily laden truck, traveling at speed, is no longer under anyone’s control.
The large heavy wad of sodden grass clippings at the very back of the cargo bed adheres to Newtonian physics, i.e., it tries to keep the rear of the vehicle moving forward down the road in its previous direction of travel. As the wet clippings represent the most weight at the furthest point from the pivot, they exert maximum leverage. The back of the vehicle performs a whiplash gate spin around the front pivot point, aided by the slick road conditions. It is like being on ice, but worse; the momentum of the heavy load is pulling the now backward-facing vehicle down the road, and the steerable wheels are now trailing, barely in contact, and essentially useless. As the rear of the vehicle passes its front end, the tenuous grip of the front tires breaks loose, and the vehicle continues down the slick wet road and into the intersection in a sliding donut spiral, completely out of control.
In the blink of an eye, the function of the vehicle has unintendedly and radically changed:
After many revolutions, the vehicle eventually slides to a stop awkwardly sprawled across multiple lanes. After a moment, the driver is able to gather himself and ease out of harm’s way, creep the truck into the recycling center, and have the load removed. Thankfully, no other vehicles or pedestrians were in the intersection when the truck spiraled through it out of control and into the opposing lanes of traffic. Had there been any other vehicles, any pedestrians, and/or any other obstacles or hazards, there would have been little or nothing the driver could have done to avoid them, and this story would have had a very different ending.
This is a true story. It actually happened—a safe state, neutralized in the blink of an eye and replaced with the unintended functionality detailed above. No sane person would design a vehicle to function that way. And yet, thousands of times a year, the above list is typical of the unsafe conditions other drivers suddenly find themselves dealing with when control of their vehicle is lost under unsafe conditions. Yes, safety was eventually restored when the truck was unloaded. But risk was not avoided by design; it was evaded by random chance. It was not prevention; it was sheer luck. And too often in these instances, the participants are not so lucky. Nothing about this is acceptable.
This example offers much to unpack. We will refer to it often throughout the remainder of this series. Let’s proceed by examining the scope of SOTIF itself, and within that scope, examine the influences that can be traced back to real-world examples like this.
The first and most basic principle of understanding the scope of SOTIF is realizing that its scope is not defined by malfunctions. Rather, the scope of SOTIF defines properly working equipment that has been designed, built, and tested to fulfill the equipment’s given requirements. At its most basic, SOTIF is scoped to specific intentions manifested through thoughtful and deliberate engineering.
As we work through the SOTIF process, we don’t constantly move the borders of the scope, reacting haphazardly to every new and unexpected ad hoc test result in a patchwork of duct tape and bandages. Instead, we try to figure out whether the vetted and approved requirements for a given system are good enough for what it was designed to manage. In other words, the SOTIF acronym might be expanded to be thought of as “safety of the intended functionality of a particular system.” Were the requirements of that particular system properly specified? Did we capture the intent of that particular system? The team works its way down the list and asks the same questions, over and over, for each safety-related system.
The true heart of the scope of SOTIF is the safety of the intended functionality. Reviewing these requirements and applying corrective action is how we also make safe the unintended functionality. And the scope of the SOTIF is defined by a system that is engineered to a purpose and functioning properly.
Overarching the scope of SOTIF is the realization that it does not exist and function in isolation. SOTIF is a process that is defined by a standard (ISO 21448) that is also closely linked to other standards and their processes (ISO 26262). The product produced as the output of properly applying these standard processes is the Advanced Driver Assistance Systems (ADAS) that are engineered into the vehicle. It is also ADAS that protect the drivers and with which they interact. To better understand how SOTIF and ADAS work together, let’s examine some of those relationships.
To illustrate how SOTIF and ADAS work together to address both the intended and unintended functionality, let’s start at the end goal and work backward. The overarching goal of functional safety is to achieve a functionally safe driving experience, in every scenario (intended or unintended), every time. To accomplish this, many elements must contribute, including having systems and components that themselves are certified as being functionally safe.
But how do you know if your system is functionally safe? You follow a defined process which, greatly simplified, looks like this:
This process is itself a process:
These steps, performed in this order, enable the systems to be created and deployed in a consistent, trustworthy, and cost-effective manner.
Before any design work can be completed, one must define the risks. After all, you must quantify the risks that you are trying to mitigate before you can design and build systems to mitigate them.
Risks are identified and sorted into one of four categories:
Because each type of risk carries its own idiosyncrasies, each category is prioritized and dealt with in a slightly different way. These are called SOTIF Scenarios. The scenarios themselves are a big topic, large enough to warrant their own article in this series.
Defining the scope of SOTIF involves multiple steps, each with a purpose. Further, they must be performed in the proper order. And they must address both the intended functionality of the system as well as any unintended functionality that may arise. The scope is defined, tested, and refined through deliberate intent. This work is guided by both the SOTIF and ADAS standards working together. And the risks drive the work. Identifying and categorizing those risks as SOTIF scenarios, is what we will examine next.
What is SOTIF(Safety of the Intended Functionality) ISO/PAS 21448
An Introduction to Verification and Validation Testing for ADAS
What is SOTIF? (Safety of the Intended Functionality) ISO/PAS 21448 The practical operation of automated vehicles in a real-world environment...
How Does SOTIF Address Safety Design Vulnerabilities?