When applying new vehicle functions, the engineers have to put in an ever-greater amount of work, because with the increasing range of functions and the growing system connectivity of modern vehicles, the number of tests and the number of relevant parameters required for the calibration also increases significantly. In order to be able to cope with the higher complexity in the future while also increasing development efficiency, Porsche Engineering uses artificial intelligence (AI) to calibrate the control units and input data. “With Porsche Engineering Reinforcement Learning—PERL for short—we are turning the calibration of driving functions into a smart decision-making process,” says Matteo Skull, Lead Engineer at Porsche Engineering.
PERL is based on deep reinforcement learning, a self-learning AI process. The basic idea is that instead of optimizing individual parameters, the AI develops a strategy that leads to the best possible calibration result for an entire function. The advantages are the high degree of process efficiency of the methodology, as it is self-learning, and the universal applicability to many areas of vehicle development (see also Porsche Engineering Magazine 1/2021 on the principle of PERL). “Porsche Engineering recognized at an early stage the potential of deep reinforcement learning for the automated calibration of control units. Since 2017, we have been working on PERL together with AI experts from the Porsche Engineering sites in Cluj and in Timisoara, Romania, and have been continuously developing the methodology ever since,” says Dr. Matthias Bach, Senior Manager HV Battery Calibration and Diagnostics and Diagnostics at Porsche Engineering.
Porsche Engineering has registered more than 50 patents for PERL.
More than 50 patents have been registered for PERL in the time since. Thanks to this expertise, Porsche Engineering is one of the first companies worldwide to have validated have integrated it into the development process for new vehicle systems. In the meantime, PERL has already been used for the calibration process in two customer projects. In the first, Porsche Engineering has been cooperating with the FZI Research Center for Information Technology in Karlsruhe and with Porsche for about three years. In this project, PERL is used to coordinate the fuel mixture formation for a new gasoline engine for hybrid vehicles. In the second project, PERL is used for the calibration of vibration damping in the powertrain of a Porsche electric vehicle.
Calibration of controllers
In both cases, AI is used to calibrate controllers, which poses a particular challenge. Thomas Rudolf, PhD student at Porsche Engineering and the FZI, explains: “The calibration of control functions is a complex task, because we need to control highly dynamic processes precisely. In the mixture formation for an engine, for example, the injection quantity must be correctly adjusted for each speed and torque combination using parameter maps so that the lambda value—the control variable for optimum operation of exhaust gas aftertreatment—matches the target.”
The main challenge this involves is the dead time resulting from the spatial distance between the engine and the sensor system at the end of the exhaust tract, coupled with the high speed at which the control system must work, say, during a load change. If the control system reacts too slowly, the function of the emission control system will be impaired and emissions will increase. If regulation is too aggressive, the system may see-saw. It is therefore crucial to find a balance between the two extremes. “Customized parameterization of complex and varied control systems in the automotive industry continues to be a major challenge. Modern learning methods can significantly accelerate the extensive, cost-intensive, and sometimes manual calibration and make it more efficient,” says Prof. Sören Hohmann, Head of the Institute of Control Systems at Karlsruhe Institute of Technology (KIT) and Director at the FZI, summarizing the advantages of AI use.
“Modern learning methods can significantly accelerate the extensive, cost-intensive, and sometimes manual calibration and make it more efficient.” Prof. Dr.-Ing. Sören Hohmann, Head of the Institute for Control Systems at KIT and Director at the FZI
PERL is able to align the wide range of control parameters for highly dynamic engine operation in a way that sets the optimum mixture. “This makes PERL an indispensable development tool—especially for future, stricter emission standards that will require even more precise lambda control in all operating areas,” adds Bach. Dr. Galabina Aleksieva-Rausch, who is responsible for processes, quality, and method development at Porsche, shares his assessment: “As part of a study of the potential offered by PERL, we used it in parallel with conventional calibration to ensure comparability of the results. The result was much better than expected expected: Calibration using the AI approach was already almost as good as that of the series calibration, even without any fine adjustment.”
The maturity of the calibration data generated in the computer using AI depends heavily on the task in question, but is usually between 80 and 90 percent. Fine tuning and validation of the raw calibration are then carried out by means of bench tests and test drives, which must remain an integral part of the calibration in the future due to the requirements for quality control and assurance. PERL also supports the applicator with these tasks, because the program can run in the background during tests and uses the data obtained to make suggestions for further optimization of the calibration.
At present, the efficiency gained by using PERL for series development cannot yet be quantified. However, it is already clear that AI-supported calibration methods such as PERL can significantly accelerate the overall development process: “Thanks to PERL, we can run the calibration at a much earlier stage and get valid results more quickly. We then take those results and approach the subsequent development steps in a much more focused and therefore more efficient way,” says Stefano Chini from Aleksieva-Rausch’s team.
The second application case using PERL is vibration damping in the powertrain of an electric vehicle. In this instance, the engineers are trying to attenuate a disruptive vibration in the powertrain with the input of a specific countermeasure—the same principle is used in headphones, say, and known as noise canceling. This calibration mainly concerns excitations in the range of 1 to 15 hertz, which are often perceived by the occupants in the vehicle as vibrations and, in the worst case, can even damage the powertrain. As with the mixture formation in the engine, optimal control is crucial to the functioning of the system, because for effective damping, the vehicle’s electric motor must be actuated at the right time and correctly synchronized. In addition, the counter-vibration must not be too strong, because the motor torque required to generate the counter-pulses is not available for propulsion. This means that, when this concept is applied, comfort and driving dynamics interact directly with one another.
Finding the optimum balance
Applicators must therefore find an optimum balance between comfort and sportiness when designing the vibration damping. In order to achieve the best possible result—the result typical for Porsche—three maps per electrically driven axle are simultaneously supplied with data to achieve vibration damping. Due to the high level of complexity, a robust initial input of data to the control units requires a great deal of effort if done manually. Only when the control units have received the data can the applicators start the fine calibration work. “One of PERL’s main objectives was to shorten this period. From the very beginning, the focus was on a universal application for AI for very different vehicle platforms and derivatives,” says Maurice Hauß, software engineer at Porsche.
In order to be able to use PERL, the development teams at Porsche Engineering and Porsche first used individual test bench data to compile a model of the chassis physics of real-life vibration profiles; this model was expanded using neuronal networks. “The hardware model simulates the physics while the artificial neuronal network bridges the accuracy gap between the real world and the simulation,” explains Skull. The PERL core methodology was then applied. The AI agent was trained on the hybrid model using a large dataset and randomly initialized maps; this ensures a good generalization of the strategy.
After the end of the extensive training, it took only a few seconds in target deployment for the agent to complete the vehicle-specific calibration adaptation. In addition, it was possible to transfer the AI strategy from the original vehicle to other derivatives—including ones with different powertrains—without modification and in some cases without the need for vehicle-specific measurements The expectations associated with the use of PERL were thereby far exceeded. “With new variants, thanks to PERL, we can start the calibration with precalculated data; other test car users can work on the vehicle calibration in parallel during this time and do not have to wait for us,” explains Tobias Roulet, Manager E-Vehicle Calibration at Porsche. “In total, we can save one to two weeks of calibration time. What’s more, PERL enters valid data in advance, so there is no risk of damage due to incorrect calibration data when the powertrain is first commissioned.”
“PERL has proven that the approach is universally applicable for all types of data entry and that AI can significantly reduce the work and time required in the calibration.” Dr. Matthias Bach, Senior Manager Battery Calibration and Diagnostics at Porsche Engineering
After the successful completion of both customer projects, the customer assessment has been entirely positive. “PERL has proven that the approach is universally applicable for all types of data entry and that AI can significantly reduce the work and time required in the calibration. This means that the newly developed PERL methodology makes a major contribution to a strategic goal of Porsche Engineering: To ensure a short delivery time of high-quality solutions for complex tasks—for the benefit of our customers,” according to Bach. An expansion of PERL usage at Porsche is already in the planning stage. The program is to be used, for example, to optimize the camshaft bearing control calibration.
Vibration damping calibration
Neuronal network improves simulation accuracy
For effective vibration damping, the motor in an electric vehicle must be actuated at the right time and correctly synchronized. In addition, the counter-vibration must not be too strong, because the motor torque required to generate the counter-pulses is not available for propulsion. In order to be able to use PERL, the developers first used test bench data to compile a model of the chassis physics of actual vibration profiles. The model was then expanded using neuronal networks. The hardware model simulates the physics while the artificial neuronal network bridges the accuracy gap between the real world and the simulation.
PERL Policy Training
Cloud Computing accelerates calculations
The agent was trained using a large dataset and randomly initialized maps to ensure a good generalization of the strategy. 64 CPU cores in the cloud processed the simulations in parallel, which resulted in a significant acceleration of calculations compared to the use of a local machine. The strategy has been optimized with a graphics processor in the cloud.
Summary
Porsche Engineering uses reinforcement learning (RL) for the calibration of driving functions. The new approach means that a large part of the data input is performed by AI; only at the end is manual fine-tuning carried out on the test track. In addition, a single trained RL agent can handle the calibration of several vehicle derivatives. As a method, RL significantly reduces the time and costs involved in the calibration.
Info
Text first published in the Porsche Engineering Magazine, issue 1/2024.
Text: Richard Backhaus
Copyright: All images, videos and audio files published in this article are subject to copyright. Reproduction in whole or in part is not permitted without the written consent of Dr. Ing. h.c. F. Porsche AG. Please contact newsroom@porsche.com for further information.