The tests must be in line with the driving scenarios that the ADF will encounter while operating in real traffic. Therefore, it is necessary to investigate the driving scenarios as well as their parameters before defining the test parameters.
A general concept for determining relevant test cases has been developed for instance by the German research project PEGASUS (Mazzega, J. et al., 2019) and other research projects (such as V&V Method, SAKURA, HEADSTAR, CCAM Sunrise). The general concept relies also on deriving test cases from a database, which contains several real-world driving scenarios, and which is manufacturer independent. Such a database – if available – would be of course very helpful to ensure the correct definition of the test space and the test cases. However, it must also be noted that such database could miss some rare event that are of importance for a certain ADF. In the latter case the “injection” of expert defined test cases could be an approach to fulfil these gaps in the database.
Since the scenarios to be tested depend strongly on the ODD of the ADF as well as the technical capabilities of the ADF, first a description of the intended ODD and the function are required. In the second step the test space and individual test scenarios can be defined. Note: The test space is described by parameter ranges covered by the individual test scenarios. The entirity of all test scenarios define the overall test space. The test space should cover the ODD sufficiently.
The selected tests should not only cover scenarios that occur frequently, it is also necessary to test the ADF in rare scenarios – in particular if these rare scenarios could lead to serious consequences. The test scenarios shall cover all operation conditions of the ADF. These include scenarios, in which the function is not operating (ADF not available, ADF ready to be activated, activation) as well as those in which the function is operating (ADF is operating, ADF is deactivated). Within these conditions different modes or sub-conditions could exist (e.g. deactivation by the user, deactivation by the function). If this is the case, the sub-condition must also be covered by the tests.
It must also be ensured that all (relevant) requirements of the ADF are covered by tests and checked whether the function fulfils them. Here, it is important to note that function’s requirements can change in the course of development. These changes must be considered in the test plan continuously during the development process. Therefore a process shall be established that ensures that updates to the requirements leads also to an update of the test plan.
Additional input for this question is the NATM discussion on UN ECE level (UNECE -NATM, 2021).
Main Question
Is the test space defined according to the function design and the intended ODD? (Test planning)
Sub-Questions
- Are the relevant driving scenarios defined covering the entire ODD?
- Is a process established that ensures that the right relevant scenarios are selected?
- Are relevant rare driving scenarios taken into account?
- Are relevant critical scenarios taken into account?
- Are scenarios taken into account that cover the entire operation of the ADF (not available, ready, activation, active and operating, deactivation)?
- Are all (relevant) requirements of the ADF tested?
References
- Mazzega, J. et al. (2019) PEGASUS METHOD. Zenodo. doi: 10.5281/ZENODO.6595201.
- UNECE -NATM (2021) or (UNECE -NATM, 2021)