The testing of components, systems, and software is an integral part of automotive development. Manufacturers have been testing their vehicles for as long as there have been vehicles. As engineering knowledge about the automotive realm matured into formal disciplines, automotive testing likewise evolved into methodical and formal processes. The goal was the acquisition of accurate, complete, and trustworthy data. What was lacking, was the capacity for comprehensive system-wide testing.
Almost all these tests were performed on tangible hardware via time-consuming and expensive manual processes. Humans manually designed the tests, fabricated the test articles, and calibrated and validated the test equipment. People set up the tests, performed the tests, wrote down the results, and shared them. The task was made manageable by limiting the scope of testing to only the most critical tasks, and by applying sheer manpower.
As the twentieth century progressed, testing processes became more refined. Proving grounds, professional journals, deliberate car crash tests, wind tunnel testing, and all manner of scientific measurements were introduced. The formation of multiple automotive organizations resulted in standardized systematic testing and safety measures. Still, almost all these activities were conducted, and their results shared, using manual processes. The old constraints still applied: there was not enough time or money to test every scenario.
As computers entered the labs and began appearing on the vehicles themselves, some of the data gathering became more automated. And yet, the design and set up of tests largely remained a manual reactive process. One problem, one question, one test, one result. Fabricate a test part and test it to failure. Record the results. If fixing Problem A broke something in System B, react and try again. The investigation would typically end when A and B were resolved, but without methodically examining their potential effects across the entire system. It remained cost prohibitive to test new changes across all existing systems.
The widespread adoption of computers changed this scenario, with computer modeling introducing sweeping changes in both the design and testing processes. However, as computers were introduced onboard the vehicles themselves, they became an integral part of the vehicle, tightly woven into the fabric of the overall vehicle system. Vehicles evolved into comprehensive mechatronic systems. The testing of software became just as important as the testing of hardware, and this introduced a paradigm shift in the nature and complexity of testing.
In a mechatronic vehicle, changes to software can affect both the operation of hardware and the operation of other software. The potential ripple effect becomes much greater. As on-board software is updated, changed, or the target system is modified, vehicle systems must be tested to ensure that they still perform as intended after changes are implemented. Regression testing is the rerunning of select test cases to establish that previously developed and tested software still functions as intended after it has been changed. Likewise, full regression testing is the selection of all previously executed test cases, to help ensure that the software changes do not have unintended consequences anywhere else in the system.
Where full regression testing was impractical in the old analog hardware vehicles, it has become possible in the tightly interlaced software and hardware systems of today’s mechatronic vehicles. And yet, today, manual-type testing is still the norm. There is still a tendency to treat software components as if they are individual elements. For the most part, in the automotive industry, there has not been a strict effort to do full regression testing on every single software release or calibration change.
Traditionally, engineers sit down at a bench with a module connected to a test system, and they test the inputs and outputs coming into the module by simulating an operating engine environment, to make sure that their components are functioning properly. They adjust the system as required and build specific functional elements into the system. Then, a different team of more system-oriented engineers test the entire system just to make sure it functions properly as a whole.
However, that process is slow and labor-intensive. It can be narrow in scope, and typically does not take into account how changes today could affect existing work outside the immediate focus. As the complexity of the systems grow exponentially, the complexity of the tests must in turn grow exponentially, and the amount of effort required to complete those tests must in turn increase dramatically. That is a significant issue for the industry just based on sheer capacity restraints alone. Additionally, the testing burden is growing due to the more stringent nature of functional safety requirements.
In automotive, there is a huge gap between the level of testing today, and the level that testing will have to achieve in the very near future. For an advanced driver assistance system (ADAS)-oriented autonomous vehicle, full regression testing will be absolutely necessary for a system to become certified as being functionally safe. Right now, companies are not equipped with the technology solutions that they need, and they are not equipped with the number of test engineers that they need to manually perform that work. It is wholly impractical to attempt to do this work manually. There is a significant gap that needs to be filled, and that’s where LHP comes in.
The answer to this dilemma is automated full regression testing conducted to rigorous standards with full visibility. Once implemented, these tests will be much broader and more extensive than they have been in the past. Automated full regression testing will transform the way the automotive industry conducts business. And at LHP, we know how to get there.
LHP delivers solutions step-by-step, both with our services and with our hardware solutions. Our testing starts with the software itself. We apply extensive testing to the software at the unit test level. Then we test at the next level, the simulation of the controls. We apply component test imaging, we test the feature components, and then we model the simulation of the whole system.
We test the software, executing the basic choices of the software to make sure that the logic and the functions are proper, making sure that there are no overruns on arrays, and other similar checks. Then, we evaluate the actual simulation itself to make sure the dynamics of the entire system are operating properly. We also test the hardware interactions, and then, the interaction of the system with the whole vehicle. It is a stage-by-stage incremental process that must be successfully achieved before the vehicle can be certified functionally safe and released into production.
Part of our solution involves implementing remote access to test systems to increase the utilization of resources. To perform bench-level testing, the test systems could cost an organization $120,000 or more per system. One group might need half a dozen of them. That is a significant investment just for test systems. One service we provide to companies is to help them maximize the utilization of their test systems. For example, many companies have global engineering organizations. Instead of just shutting the test systems down for the night, we show them how they can be used overnight during the off-peak hours by another of their organizations overseas. We show them how to maximize the utilization of their test assets, step-by-step. We also show companies how to leverage automated test scripts, so that the system is running through tests and doing the regression tests in an automated fashion.
LHP provides value through our services. We have extensive knowledge and expertise with control software, sensors, everything that goes into a system, and the integration into applications. We understand implementation. We also understand the requirements for Functional Safety and the whole development workflow from concept to production. We have the depth and breadth of knowledge and experience to help a company successfully navigate this complex environment.
At LHP, we also have groups that build the test systems. We know how to build the test sets, and we know how to generate the automated test scripts that link the appropriate test to a requirement. We possess key knowledge of the testing process, and we build the test system platforms that enable the testing.
In the past, automotive testing has largely been a manual process. At LHP, we are stepping boldly into the future by driving more automation into the process of testing today’s modern mechatronic vehicles. Our solutions empower organizations to maximize the use of test equipment through broader access and cloud-based test services. The future of the automotive testing landscape is one that globally optimizes the use of test equipment to implement automated full regression testing. LHP has the skills and experience to successfully lead your organization confidently into this autonomous future, step-by-step.