Once the tests have been executed, the question, whether the test plan is correctly implemented and followed, becomes relevant. While testing, different limitations or constrains can occur that lead to intended or unintended deviations from the test plan. Intended deviation might be necessary to overcome detected issues. In contrast the unintended deviations might not be noticed. Therefore, it is strongly recommended to check during the test execution as well as afterwards, whether the tests have been carried out according to plan.
In the end it must be ensured that the required data for the sign-off, homologation or certification process are available at required quality. If this is not the case, the tests need to be repeated. The answer to the question, what the required data quality is, depends strongly on the purpose of testing. To a certain extend the requirements need to be defined by the (external) stakeholders. In case there are no clear requirements set by the external stakeholders, it is important to ensure that the data provide clear and not contradicting results, for instance a high deviation between single tests with identical input parameters could be an indicator for a deficient data quality.
Finally it shall be checked whether all relevant information has been documented and stored correctly. If a deviation from the test plan has occurred, it should be documented. The documentation should also cover the reasons for this deviation from the test plan.
Main Question
Is the test plan implemented and followed correctly? (Test execution)
Sub-Questions
- Are any deviations from the test concept / plan documented?
- Are any reasons for the deviation from test concept / plan documented?
- Are all required data for the sign-off, homologation or certification process available?