Intro

Tool Confidence

Our ATE falls within scope of software tool confidence discussions because it consists of software that automates the execution of test procedures. So, let us briefly discuss what is meant by tool confidence of software tools. Tool confidence is the combination of two separate considerations. The first is called Tool Impact (TI) and is defined as the possibility that an erroneous output (of the tool) will introduce an error or fail to detect an error in the safety-related product. This first consideration applies to our ATE because it will be used to test the safety-related features and reactions of our device under test (DUT). We are relying on the ATE’s ability to not only properly simulate the DUT, but also to properly detect if the DUT fails to react appropriately. The second consideration is called Tool Error Detection (TD), and it is a measure of our confidence in the software tool’s ability to detect any (of its own) erroneous outputs. We can take credit here if our ATE has built in features that allow it to detect errors it may make while testing a DUT. Tool Impact and Tool Error Detection each have different levels that can be assigned based on criteria given in the ISO 26262 standard. Once we assign a TI and TD level, we can derive a Tool Confidence Level (TCL) for our ATE software.

With a tool confidence level determined, we can now look to the standard once again for guidance on what kind of qualification activities should be performed. The stringency of required qualification activities will be based on the determined TCL of our ATE and the Automotive Safety Integrity Level (ASIL) rating of the DUT.

Qualifying Our ATE

If our ATE is determined to have a TCL1, then no further action is required, and we are ready to use our ATE system. If we end up with a TCL2 or TCL3, we must qualify our ATE for testing safety-related DUTs. As we explore methods for qualifying our ATE, it is important to note that ISO 26262 Parts 8-11 are intended to provide guidance for all software-based tools used during the development lifecycle. With that in mind, there is a difference between a development tool, such as a compiler, and a test tool. A compiler can introduce an error even if the logic written in a text-based language is error free. If the compiler produces machine instructions that do not fulfill the intent of the text-based logic, then it could introduce an error that impacts safety. Our ATE software cannot directly introduce errors in the DUT because its output is not used in the DUT. Our ATE software can only fail to detect an error during testing of the DUT. It is a subtle difference, but an import one when considering how stringent a qualification needs to be performed. Our ISO 26262 development lifecycle should include the validation of test results. This step is intended to catch anomalies such as test tool errors.

As we approach qualification of our ATE, we must keep in mind that the goal here is to establish that our ATE can properly verify safety-related features on DUT and apply sound principles with that goal in mind.

Existing ATE

If we are planning on using an existing ATE system for functional safety testing, then we have some options for qualifying our system. The ISO 26262 standard provides us with four methods that can be used to qualify our ATE.

As can be seen from the table above, our ATE can be qualified based on “Increased confidence from use” if it will be used to verify safety-related DUTs with a lower ASIL rating. Applying the increased confidence from use method is not as easy as it may seem though. The following criteria must be met to apply the increased confidence from use method.

1. Our ATE has been previously used to test DUTs in similar use cases
2. There are no new specifications being placed on the ATE
3. We have maintained a systematic approach to capturing and fixing any malfunctions
4. We have captured sufficient data to show that we meet the above three criteria

The main thing that makes accomplishing increased confidence from use is the diligence that must have been maintained in documenting and maintaining data that can be used as evidence.

Similarly, the “Evaluation of the tool development process” method can also be used for verifying lower ASIL rated DUTs. To meet this method our ATE must have been developed following an appropriate standard. This method also implies that we have evidence to show that this occurred and implies that verification was performed, recorded, and results are available for review.

If we cannot apply the first two methods, or if our ATE is going to be used to test higher ASIL rated DUTs, we must then apply “Validation of the software tool.” This method is a formal approach to verify our ATE can properly exercise the DUT’s safety-related features and catch any errors. This does imply a development-like process needs to occur where we take the DUT’s safety requirements and create ATE system requirements that focus on enabling our ATE to properly test the DUT’s safety features. The best approach here is to tailor a validation process that is based on an established safety process within our organization.

New ATE

If we are developing a new ATE system for testing, then we are only left with the final method provided by the standard: “Development in accordance with a safety process.” At first this may sound like it could be an overburdening requirement, but there are a few things to keep in mind. The goal of our ATE is to verify the DUT’s safety-related functionality. We do not need to handle topics like safety mechanisms, redundancy, and FIT rates. What we need to do is apply a quality development process, where we start with the DUT’s safety requirements and create ATE system requirements that fulfill the intent of verifying the DUT’s safety requirements. From there we implement our requirements in our ATE design and properly verify our ATE system. We should also include a procedure to verify our ATE system is functioning properly throughout its lifetime.

Conclusion

New and existing ATE systems can be used for functional safety testing but do require a level of formality that we may have previously gotten away with not doing for testing DUTs without functional safety requirements. With some careful thought and planning, our ATE systems can be used for functional safety testing with a minimal amount of extra processes and paperwork.

Interested in learning more about functional safety for your organization? Contact our team today!