Although the amount of software in vehicles is soaring, many companies’ testing strategies continue to follow traditional, hardware-centered procedures. According to Steffen Schmidt, President & CEO of IPG Automotive, this approach on its own is no longer viable. Talking to AEEmobility, he explained why conventional testing strategies are reaching their limits and why simulation and software-in-the-loop will have to be the backbone of functional validation in the future.
Mr. Schmidt, the transformation towards software-defined vehicles has been creating considerable challenges. Apart from the rapidly expanding role of software, the industry is also having to deal with a shift toward more centralized electronic architectures. This begs the question about the impact these developments have on the testing strategy.
Steffen Schmidt: Now that we are at the transition to software-defined vehicles, I can see three key challenges. One is the growing role of software. Modern vehicles already contain more than 100 million lines of code, and by 2030 we expect 500 million or even up to one billion lines, depending on the scenario. The second challenge is that the architecture is changing fundamentally – central high-performance computers and service-oriented structures are becoming more important.
And the third challenge refers to recall costs. In 2011, only 5 to 10 percent of recalls were caused by electronic or software problems. By 2022, the ratio had already risen to more than 65 percent – and recent data suggests it is heading towards 80 percent. The more software a vehicle contains, the higher the share of software-related errors is bound to rise.
Current testing strategies still involve a large part of validation taking place in field tests or on the proving ground. This will not be viable anymore. Frequently, errors are identified at a point when a lot of time and money has already been invested, making the necessary return loop to development time-consuming and expensive. The fact that rare but critical situations occur randomly during test drives and are almost impossible to reproduce makes this particularly difficult. The increasing complexity of modern vehicle systems will make this procedure more and more inefficient. Ultimately, the late, hardware-centered validation incurs high costs, leads to lengthy development times and limits test depths. These problems can be tangibly reduced by employing an early, software-oriented testing strategy. The first steps towards achieving this have been taken, but it will take a few more years before software-in-the-loop-based approaches will have become established everywhere.
The transformation of the automotive industry toward software-defined vehicles requires tailored testing strategies: The goals are shorter development cycles, lower development and warranty costs, and higher software quality and functional safety throughout the entire development process. (Picture: IPG Automotive)
Does the Kraftfahrt-Bundesamt (German Federal Motor Transport Authority) already accept such simulation-based procedures, or is a complete validation through real-world tests across millions or even billions of kilometers driven still necessary?
OEMs, vendors, tool providers and homologizing authorities are called upon to pave the way for this. Euro NCAP is already taking the first noticeable steps by no longer prescribing that all tests must be performed completely on the proving ground, but allowing a much larger share to be simulated. This approach shows that things are moving in the right direction, and it supports the necessary coordination between the industry and regulatory authorities. However, it is difficult to say how long it will take for this procedure to become fully established.
Sensor technology is at the center of the changes. More and more customers are inquiring about how accurate and realistic sensor models are, calling for proof of validation for different sensor technologies, which are usually provided in collaboration with the sensor producer. The aim is to prove that virtual sensors reliably simulate real-world behavior within defined tolerances.
How will industry and regulatory authorities have to develop their approval processes to help new technologies become series-ready more quickly – in order to remain competitive with China?
The causes for the slow progress are not so much regional differences than existing structural framework conditions. Key is the early, and more intensive, involvement of the approval authorities. At the same time, trust in the new technologies must be built. Validation projects play a central role in this, because they verifiably prove that these new approaches are effective and reliable, allowing regulatory validation.
The dynamic of developments in the European – and some of the North American – automotive industry is relatively slow. Other markets are often faster to implement new technologies because they don’t come from historical structures and are less caught up in organizational silos and red tape. Their decision paths are shorter, which allows them to focus more on the speed of development.
When it comes to scenario-based testing, time and computing capacity are key. CarMaker enables millions of simulations; but how do you ensure that such a testing scope is scalable and economical? What role do software-based testing strategies play compared with conventional hardware-based approaches?
The economic efficiency of a testing strategy largely depends on the testing strategy. Conventional, hardware-focused approaches are expensive and time-consuming. Scaling is only possible to a certain extent. Tests on the test bench or the proving ground are practically impossible to parallelize; the diversity of functions in modern vehicles means that they quickly reach their limit.
Steffen Schmidt, President & CEO of IPG Automotive: „A particular challenge is that rare but critical situations occur randomly during real-world driving tests and are virtually impossible to reproduce. As modern vehicle systems become increasingly complex, this approach is becoming less and less efficient. Ultimately, late, hardware-centric validation leads to high costs, long development times, and limited test coverage.“ (Picture: IPG Automotive)
Software-based testing strategies have an obvious advantage. If CarMaker is used in combination with a virtual ECU in a cloud environment, the number of tests that can be run in parallel is almost unlimited. Reaching the kind of test depth that would have taken months to reach, can thus be achieved within just a few days or weeks. Already, customers are managing to run up to one million simulated tests per week. The short development and update cycles we are seeing today can only be validated with strongly parallelized virtual tests. The scope needed here can neither be provided by laboratories nor by proving grounds.
„Rare, safety-critical situations are difficult to reproduce in real-world driving tests, whereas in virtual environments they can be simulated in a targeted, repeatable, and highly detailed manner.“
Steffen Schmidt, CEO of IPG Automotive
From a financial point of view, working in the cloud – either a company cloud or a public one such as AWS or Azure – brings marked benefits. The decisive factor is scalability: If a customer needs to run one million tests in five days instead of seven, obtaining additional computing capacity at short notice is not a problem.
Fully embedding tests in CI, CT or CD pipelines can considerably speed up the process. Continuous development and testing, large-scale automation and automated test analysis reduces run times to a level that conventional static test approaches can no longer achieve.
Software-centric end-to-end testing strategy for software-defined vehicles: Testing is increasingly shifted to earlier stages of development (front-loading) in order to achieve maturity levels sooner, detect defects more quickly, and resolve them more efficiently. At the same time, virtualization enables greater scalability and test coverage throughout the entire development process. (Picture: IPG Automotive)
Where does simulation reach its limits still, and how are such limitations addressed in practice?
In our opinion a large part of what is being tested in the real world can be simulated. Different driving styles, traffic situations and many complex phenomena can be simulated, with the decisive advantage of being reproducible. In a virtual test, a scenario that would be random in real world can be repeated indefinitely under the exact same conditions. That being said, it is not an “either” “or” question. Simulation cannot completely replace real-world tests because new, previously unknown phenomena always occur first in the real world. You can only model something that you know or that you have observed. This is why real-world testing and field data remain indispensable. It is the combination of the two worlds that creates value –simulation increasingly determines the prioritization of real-world tests. As soon as a relevant event has been identified in the real world, it can be simulated and validated efficiently. Instead of driving millions of real-world kilometers before a rare scenario occurs again, it can be cost-effectively tested with simulation.
What role will the proving ground play in the future?
In virtual environments and vehicle-in-the-loop setups, sensors always work with synthetic environments. Modeling the complete physical chain of effects allows the efficient development and validation of functions. This is why proving grounds will continue to play an important role, albeit with a very different function. They will no longer have the purpose of racking up as many kilometers as possible. They will serve the targeted validation of key functions under real-worlds physical conditions. Once you reach the end of the development process, experiencing and testing the complete vehicle system with its final hardware configuration in real world remains crucial.
There is a distinct shift in focus from the quantity of kilometers driven to the quality and diversity of test cases. Driving a million kilometers under uniform conditions without critical scenarios will hardly provide any insights. And that’s where modern simulation and software solutions add value: They allow end-to-end traceability from defining a use case, its virtual and real-world validation all the way to the recorded test result. Proving grounds will be part of a hybrid test concept that logically combines virtual and real-world testing. The real challenge lies in consistently shaping the transition from the current, mostly real-world-based approach to the integration of both worlds.
„The safety of highly complex vehicle systems can now only be verified statistically. Neither one million nor one hundred million test kilometers driven can guarantee safety.“
Steffen Schmidt, CEO of IPG Automotive
What would companies have to change in order to speed up the change?
The key question is how to manage the transition from the current situation to the desired goal. It all depends on how much time companies have to spend on development and standardization, and how established a software-centered mindset already is. In many organizations, a hardware-centered approach still prevails. This leads us to three essential questions: How can you get there? How long will it take? Is there any will to accelerate the process? Faster transformation is bound to require processes, structures and – above all – mindsets to undergo fundamental changes. I think that’s where the real challenge lies. The technology itself is not that much of a problem. It is available, and companies can get support with implementing it.
Homologation, safety verifications and scenario-based testing generate a surge in required data volumes. How do you ensure that your customers don’t drown in scenario or data chaos?
That’s a justified question. The rise of simulation, particularly compared with real-world development and testing methods, allows us to translate many tried-and-tested principles from conventional vehicle development. To help with this, we created VIRTO.
Before we can start running more simulations, we have to have precise processes for the setup of virtual prototypes. These prototypes are based on data, which is often distributed across a company. As soon as these virtual vehicles have been created and the next step is testing, vast numbers of scenarios are needed. VIRTO addresses both of these aspects, by managing virtual fleets of vehicles and by managing scenarios. Both aspects are collated and embedded into an end-to-end process. The key advantage is that every step is fully trackable. Requirement specifications, data, variants and test results can be consistently linked. The simulation is based on a unique single source of truth, which ultimately makes the data much more usable.
ASAM OpenX – Standard ecosystem for simulation-based development and testing
ASAM OpenX refers to a family of open standards developed by the Association for Standardization of Automation and Measuring Systems (ASAM e.V.) that focuses on simulation-based development, testing, and validation in the automotive industry. The goal is to enable a uniform, vendor- and tool-agnostic description of roads, scenarios, environments, and interfaces. OpenX standards thus form a common language for virtual test environments and lay the foundation for reproducible and interoperable simulations. They are used in particular for the validation of driver assistance and automated driving functions, where rare and safety-critical scenarios must be systematically tested. In practice, vehicle developers use OpenX standards to model Euro NCAP scenarios for simulated test drives, for example, or to standardize traffic networks and motion sequences in simulation environments. As software and system complexity grows, OpenX standards are becoming increasingly important, as they facilitate the exchange of scenarios and models between different tools and support end-to-end, scalable test processes.
Are your tools exclusively meant for the late-stage validation phases? Or can they be used during earlier conception and architecture development stages?
Our tools are also used during early development stages. In the development of powertrains, it starts with fundamental conceptual decisions, such as the target range and the resulting battery size. The position of the battery and its impact on vehicle handling and lap time for sporty electric vehicles can be tested during an early development stage. When it comes to driver assistance, the concept phase often concentrates on the ideal position of the sensors to ensure reliable environment sensing in all relevant driving situations. Some actual customer projects and references for the use of our tools at an early planning stage already exist. This shows that our tools don’t just play to their strength during final validation, but that they make an important contribution to substantiating the vehicle concept design.
How deep is the integration of ASAM-OpenX standards into your tools?
The ASAM-OpenX family is key when it comes to open standards. It includes established OpenDRIVE, OpenSCENARIO and Open Simulation Interface, and the more recent addition, OpenMATERIAL. OpenDRIVE has been a recognized and stable standard for years. OpenSCENARIO has also reached a high level of maturity, despite the fact that there are still different “dialects” in the market that sometimes make its end-to-end utilization difficult.
In principle, we think that the establishment of open standards is necessary to ensure the interoperability of different tools and development environments. That is why our products supports these standards, and their ever-deeper integration into our existing tool chains is being promoted. More specifically, the next CarMaker generation – which will be launched this year – will present a leap forward in terms of supporting OpenSCENARIO and OpenDRIVE by offering native support for these standards. (oe)
Thank you for talking to us!
(The interviewer was Klaus Oertel, editor-in-chief at AEEmobility)
