View Categories

AI – Adversarial Examples and Robustness in AI Systems

In order to solve the problem of ROB in AI systems, many publications on this aspect deal with the so-called adversarial examples that cause malfunctions of a DNN model, although the respective input patterns are very similar to valid data (Bastani et al., 2016). 

Main Question

Have the main aspects characterizing Robustness of AI been taken into account? (Becker et al., 2020), (Belkin et al., 2019), (Duthon et al., 2018), (Gama et al., 2014)

Sub-Questions

  1. Has the Adherence to certain thresholds on a set of statistical metrics been considered (to hold on the validation data)?
  2. Has the Invariance of the functional performance, w.r.t. certain types of data perturbations, been considered?
  3. Has the Invariance of the functional performance, w.r.t. systematic variations in the input data, been considered? (E.g., measurement drifts or operating conditions).
  4. Has the Stability of training outcomes been considered? (Under small variations of the training data and with different training runs under stochastic influences)
  5. Has the Consistency of the model output been considered? (For similar input data, resistance to adversarial examples). See also AI_9. (Bastani et al., 2016)

References

  • Bastani, O., Ioannou, Y., Lampropoulos, L., Vytiniotis, D., Nori, A.V., Criminisi, A. (2016) ‘Measuring neural net robustness with constraints’. Proc. 30th Intern. Conf. on Neural Inform. Process. Systems, pp. 2621.  Available at: https://trustml.github.io/docs/nips16.pdf (Accessed 22 May 2024).
  • Becker, M., Lippel, J., Stuhlsatz, A. and Zielke, T. (2020) ‘Robust dimensionality reduction for data visualization with deep neural networks’. Graphical Models, 108, p.101060. doi:https://doi.org/10.1016/j.gmod.2020.101060.
  • Belkin, M., Hsu, D., Ma, S. and Mandal, S. (2019) ‘Reconciling modern machine-learning practice and the classical bias–variance trade-off’. Proceedings of the National Academy of Sciences, 116(32), pp.15849–15854. doi:https://doi.org/10.1073/pnas.1903070116.
  • Duthon, P., Bernardin, F., Chausse, F. and Colomb, M. (2018) ‘Benchmark for the robustness of image features in rainy conditions’. Machine Vision and Applications 29(5), pp.915–927. doi:https://doi.org/10.1007/s00138-018-0945-8.
  • Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M. and Bouchachia, A. (2014).’ A survey on concept drift adaptation’. ACM Computing Surveys, 46(4), pp.1–37. doi:https://doi.org/10.1145/2523813.