Category Archives: Software & Testing Solutions

Future Mobility

The Balance Between Complex Testing Tasks and Effective Development in Testing Environments

11. September 2017 | Software & Testing Solutions

The Balance Between Complex Testing Tasks and Effective Development in Testing Environments

The areas of mobility and transportation are undergoing a heavily accelerated process of change. Traditional global mega-trends encompass widely differentiated user-specific and customer-specific requirements, different laws, ever stricter environmental regulations, limited resources and the electrification of drive systems. In addition, issues such as autonomous vehicles and customized, on-demand mobility solutions are growing in importance. Coming up with solutions and products requires completely new approaches on top of the established technologies, where constant advancements are needed. Examples of such approaches include digitization, information technology and networking.

The increasing complexity and differentiation result in stricter requirements that cannot be met in acceptable time frames at an acceptable cost level using established methods and processes. New, customized and specifically designed solutions and products must be developed to overcome these challenges.

Over the past years, advanced and high-performance simulation processes have become established as a main pillar in the portfolio of vehicle and drive system development. Numerous conventional methods of testing and proving have been partially or completely replaced by computer simulation. Increasing system complexity and validation requirements today and in the future, however, require specifically tailored testing solutions to validate functional reliability, quality, etc.
The suitably powerful testing capacity available provided in today’s cutting-edge development environment is no longer limited to mere logistical and technological solutions such as test rigs and measurement systems. For services to be highly efficient, all stakeholders need to be appropriately involved in the entire process of development. Besides designing and equipping the testing environment, this includes areas such as personnel structure and expertise, methods, operational organization, highly efficient logistics, information networking, and more.

Future Testing Environment Strategies

The design of future testing environments must be based on medium-term and long-term strategic product development planning and the testing requirements derived from it. They must do more than meet the simple engineering, operational, and logistical requirements. In particular, they need to be cost-effective, have a balance between operating capacity and personnel resources (and ensure they are utilized to the greatest possible extent), and assign work sensibly between what covers their own needs and what is commissioned by customers.

Testing Environment Organization and Processes

The organization and processes of modern testing environments have changed fundamentally in recent years. In the past, workers from development departments were often very extensively involved in the testing environment operations, sometimes even having a say in how they were conducted. The testing environment staff essentially represented the facility’s capacity, resources, and operators. Responsibility for defining the testing program and performing evaluations rested with the engineering department.

Over the past few years, various tasks have been shifted from engineering to the testing environments. Today, many testing environment crews are, for the most part, independently in charge of generating all test results. It takes adequate human resources and engineering capacity to perform these additional duties. That includes making the testing environment responsible for all processes and their design as well as medium-term and long-term structural orientation and investment budgeting.

Staff: Structure and Authority

The increasing transfer of numerous duties from engineering to the testing environment’s area of responsibility is creating a need for a greater number of engineers in current personnel structures and ranges of expertise. In the past, the largest share of a testing environment’s staff by far consisted of mechanics and some electricians and foremen with very few engineers. Today, the percentage of technicians and engineers has risen sharply. Added to that are IT and other specialists of various disciplines, who take care of the sophisticated test rig automation systems, measurement systems, and various software tools.

Logistics, Plus Flows of Information, Materials, and Data

Organizing modern testing environment operations to be efficient and economical requires the implementation of processes structured to be efficient and flexible from end to end and which can be adapted to meet changing requirements. This includes information management, which encompasses the handling and distribution of all incoming, internally circulating, and outgoing information, the management of experimental and measurement data plus material flows and logistics, quality management, and more. All primary and support processes need to mesh with each other smoothly and require continuous assessment, adjustment, and optimization.

Graphic - FEV Testcenter efficiency




Working and Testing Methods

Today’s advanced testing environments already use IT-based methods and tools to a large extent for organizing processes and performing work. Examples include databases used to aid in the preparation of project-specific test rig set-up and program plans, largely standardized test rig and measurement systems, highly automated running of test programs, integrated computer simulation tools, automated evaluation of test runs, and databases used to file test results.

The FEVFLEX information management software offered by FEV is a powerful solution for managing tasks, procedures, devices, media, test objects, test rigs, measurement data, and test projects, thereby contributing sustainably to a testing center’s efficiency. In addition, FEV MORPHEE significantly lowers the variety of software applications conventionally needed on test rigs. No matter if ECU (HIL), component, engine, powertrain, vehicle, or others: MORPHEE adapts to any kind of test environment.

Further reductions in time and costs of the development process can be achieved with Online and Offline-DoE tools for virtual calibration. FEV xCAL combines best-in-class modeling algorithms with an intuitive, workflow-based interface, thus enabling virtual and efficient calibration of a wide variety of powertrains and other applications.

Structure and Equipment

Today’s high-tech testing environments provide environmental simulation systems as well as traditional equipment such as test benches for engines, transmissions, vehicles, system components, and measuring equipment. In the future, there will be more new testing systems for conducting every test necessary in the field of autonomous vehicles. In recent years, high-performance computer simulation tools have replaced conventional testing methods, with new methods and procedures for testing being created. One example of this is the real-time networking of various subsystem test rigs with the built-in simulation of the system components of an entire powertrain that are not available in physical form.

For instance, FEV and the Institute for Combustion Engines (VKA) have developed the “virtual shaft” as an important tool. The test environment consists of physically separate test benches that are linked by a real-time EtherCAT connection. Thanks to the virtual shaft, the dynamometers in both component test benches are controlled in such a way that system behavior matches that of a real mechanical shaft. This enables us to recreate interactions – such as between an engine and a transmission – as early as the prototype stage before the two components can be physically adapted. That saves valuable time in development. Other benefits mainly include a protected test environment and a high number of options for monitoring individual test objects. This way, damage to prototypes can be effectively prevented. In addition, the virtual shaft allows the testing of hybrid powertrain combinations that are not yet mechanically compatible and would otherwise have to undergo extensive adaptation.

Graphic - FEV Testcenter efficiency

Virtual wave between two testing facilites






FEV Makes Software Quality Measurable

Metrics-based Strategies for Quality Assurance of Automotive Embedded Software

6. August 2017 | Software & Testing Solutions

Metrics-based Strategies for Quality Assurance of Automotive Embedded Software

Digitalization and high innovative demand drive future key technologies in the automotive industry. Increasing software complexity is met by rigorous quality and safety standards, but strict resource constraints on projects are limiting time spent on quality assurance (QA). Therefore, FEV has developed a tailored, optimized QA strategy that is scientifically sound to improve efficiency and quality in software projects. Based on data derived from 13 customer projects, the impact of strategic decisions on the quality of the resulting software was examined, thus leading to a better understanding of how high quality can be achieved and quantified appropriately.


Focus of the Study

Two major subjects are the primary focus of FEV’s research. First, the aim is to analyze, formalize and then capture the quality of a software product using the appropriate metrics. The second goal is to identify how specific properties of the quality assurance strategy influence the outcome of a project.

A pragmatic approach to defining software product quality is its segmentation into intrinsic and extrinsic facets. Extrinsic quality is most often judged on the basis of the customer’s satisfaction, whereas intrinsic quality can best be quantified using established product quality models, such as that found in the ISO 25010.

Key of the Study

For the purpose of the initial study, FEV limited the scope of investigation to certain properties proposed by the ISO 25010. Relevant quality characteristics were then quantified using a set of metrics. Most notably, the requirements conformity and residual defect ratio were considered.

The data from 13 automotive software projects was gathered and analyzed in order to validate the correlation between metric and de facto product quality and to identify any relationships to specific QA strategies. Statistical models, such as correlation coefficients and ANOVA (Analysis of Variance), were considered for analysis purposes.


The research showed that customer satisfaction is strongly correlated with both requirements conformance (positive correlation) and residual defect ratio (negative correlation), which indicates a strong linear relationship between these metrics.
We make the reasonable assumption that intrinsic and extrinsic product quality strongly correlate with each other. Then, both metrics – requirement conformance and residual defect ratio – appear to be suitable indicators for intrinsic product quality, since customer satisfaction is regarded as the major marker for extrinsic quality in literature.


Lastly, the impact of different quality assurance strategies on a project’s outcome was analyzed by first surveying which test methods were applied (respectively), and then examining the relationship between these properties and the identified quality indicators. First results here are promising, and FEV is confident that it will enable the profound derivation of an effective and efficient QA strategy in upcoming projects.

Graphic - automotive software

Relevant software products characteristics and metrics evaluated.









Focus on Reusability

Enhancing Agile Automotive Software Product Line Development by Similarity Analysis

3. August 2017 | Software & Testing Solutions

Enhancing Agile Automotive Software Product Line Development by Similarity Analysis

In the automotive domain, the complexity of software functions and the demanded quality standards, e.g. CMMI, ISO 26262, are still growing, while in the meantime the expected release cycles get shorter and shorter. In addition, the vehicle starts to turn into a smart device which is able to interact with its environment and to react autonomously. As a consequence, further aspects, such as security or privacy, receive a higher prioritization.

This up-to-date scenario is a great example for the frequently changing and hardly predictable demands in today’s automotive industry.

  • Who can ensure that features which are identified as necessary today will be required in a similar form tomorrow?
  • What is the adequate time frame for long-term strategies in the context of shorter and shorter release periods?

FEV’s PERSIST (powertrain control architecture enabling reusable software development for intelligent system tailoring) was established to give first answers to these questions. This approach introduces agile methods to be able to react flexibly on requirement changes and to reduce the duration of one development cycle and project quality risks due to continuous integration. Nevertheless, the establishment and maintenance of a Software Product Line (SPL) in the context of the daily work of a supplier has to face several ongoing projects in parallel. As a consequence, an approach is required which allows the project teams to focus on the implementation of the required product, while it is possible to continuously establish and maintain a SPL in parallel with minimized effort. Therefore, it is necessary to further extend the collaboration between Agile Software Development (ASD) and Software Product Line Engineering (SPLE), resulting in Agile Software Product Line Engineering (APLE).

Software Product Line Development Focused on Application Engineering

FEV searched for a SPL development method which provides essential feedback for the SPL during daily project work but reduces the additional effort for the projects to a minimum. The method does not require long-term decisions, keeps the SPL up-to-date and identifies new potential for reusable components is needed.

The main important development artifacts used during this process are the reference architecture, project architecture, component specification, test cases and the component implementation. The proposed method follows a component-based project-first approach: This means that the main item of the software architecture is a component and any specification for a component is first raised during project development. A component will only be considered to be redesigned in a more general way, if it is proven that a corresponding demand is given and a general component is a realistic opportunity within several projects. In addition, AE shall gain most benefit from the already established product line and components which are currently developed in different projects, without being slowed down by the burden of being dependent of generalized components.


Step by Step Process

1 In the first step, a new component is specified. This is done by performing a first draft of the interface with a related functional description.

2 In addition, its position in the software architecture is estimated. This is done by considering the reference architecture of the SPL. In the context of PERSIST, components are hierarchically arranged in a set of compositions. If a suitable component can be identified, which seems to be similar or identical to the specified component, the name and location of the component will be adapted to the reference architecture. This is the first step, where the SPL supports the project development. Complex architectural decisions can often be supported by experiences from the past which are stored in the reference architecture.

3 In case the component cannot be mapped, a specific position in the project’s software architecture needs to be defined.

4 In a second step the decision made in step 3 is reevaluated by the SPL team to avoid any false negatives.

5 If the component cannot be matched, the complete implementation will be performed from scratch. The new component is added to the reference architecture after its location has been agreed on.

6 If the component specified in the context of the project can be mapped to a component an extrinsical equality regarding the reference architecture can be established. This connection cannot only be used to compare the component under development with general components of the SPL but also to compare it with other individual, but extrinsical, equal components from different projects.

In addition further analysis can be performed on structural (interfaces) and semantical (test cases, function models) level.

7 If a candidate with a similarity lower than 100% is identified, this candidate can be used by the project team to finalize their own specification and the implementation can be based on the provided development artifacts.
The project team does not spend any additional effort on implementing a reusable component which is able to fulfil the requirements of both or more similar components. The identified similarities are only used to speed up the project development.

8 In a parallel step the SPL team evaluates if the identified similarities provide enough potential for a general reusable component. Based on the amount of similar components and the degree of similarity on structural and semantical level a decision has to be made whether to implement the component based on a similar component [7] or to proceed to step [9]: If the potential is high enough, the SPL team will implement a general component which can then be reused in further projects [9].

9 The proposed steps ensure that the project teams always specify components which are as close as possible to the already established reference architecture and available components. In the best case general components can directly be reused. Additionally all derivations of the reference architecture can automatically be spotted and the degree of variation can be evaluated. Therefore the potential degeneration of an established product line can be observed continuously and a synchronization between product and product line can be performed iteratively.


Activities 1 to 5 are already well established, while for activity 6 and 8 the necessary automatization is currently being finalized. General components, which are used in several projects, have already been established, but are always based on intensive manual reviews or a proactive approach. The possibility to investigate a given reference architecture during the specification of a new component is not noticed as an additional effort. Instead the reference architecture provides helpful information to perform architectural decisions.

In the current status the reference architecture consists of 219 components, whereas 125 (57%) components have no extrinsically equal match. 94 (43%) components of the reference architecture are part of more than one project and 61 (28%) of these components are also part of at least three projects.



Method for agile soft ware product line development: The colored activities represent activities which are performed by the SPL team, while the white ones are performed during projects.










The proposed approach provides the possibility of realizing the project-driven implementation work directly in the project without losing the benefits of a corresponding SPL. Each project team can benefit from the established reference architecture to speed up architectural decisions, while the similarity analysis can provide additional foundations for the actual implementation.