Sensitivity Analysis of DL Perception Models to Sensing Parameters

Sensitivity Analysis of Deep-Learning Perception Models to Sensing Parameters (Sielemann, Bitzer et al., 2026)

In the study of deep learning-based models and deep neural networks (DNN), measuring their sensitivity or robustness against perturbations has been a key focus. This is especially relevant for safety-critical applications, such as fully-automated vehicles on public roads, which need to demonstrate-besides robustness against intentional adversarial manipulations-the robustness under all conditions of their operational design domain (ODD). In this context, the specific understanding of how deep learning models are affected by ODD aspects is crucial to assure safe and reliable operation.

We present an approach to systematically evaluate the sensitivity of DNNs to variations in environmental conditions through the use of physically-based synthetic data, demonstrated on the task of traffic sign recognition. For this task, we leverage the data synthesis pipeline from our previous work on the Synset Signset Germany dataset, for which the comparability to real-world data was demonstrated. This pipeline is used to create a large-scale systematic dataset of perturbations with physical, exposure-related meaning, including 2,196 subdatasets, accumulating to 9,442,800 test images. This dataset is provided as open data to serve as a basis for further in-depth investigations into the performance of DNNs under challenging conditions.

The application of our testing approach indicates that the performance of DNNs can be related to the physical parameters of input image data and that the sensitivity to these parameters can be measured to a significant and reproducible degree; however, the results show high variation and complexity in the behavior of the evaluated DNNs that extend beyond simple data quality models.

Synset Signset Sensitivity Analysis Datasets

Field	Task	License	Samples	Key Annotations	Institutions
Computer Vision	traffic sign recognition	Creative Commons Attribution 4.0	4,300 independent images per subdataset, 9,442,800 images in total	Per frame: Class (sign type), environment conditions (e.g., time of day, contrast), imaging artifacts (noise, motion blur, chromatic aberration), semantic segmentation, binary mask.	Fraunhofer IOSB, KIT-IES

Paper and reference

This work is licensed under Creative Commons Attribution 4.0.

To cite this work in your scientific publication, please use the following bibliography entry:

Sielemann, A., Bitzer, N., Roschani, M., Ziehn, J., and Beyerer, J. (2026). Sensitivity Analysis of Deep-Learning Perception Models to Sensing Parameters. In 2026 IEEE 29th International Conference on Intelligent Transportation Systems (ITSC).

@inproceedings{sensitivity_analysis_sielemann_2026,
  title={{Sensitivity Analysis of Deep-Learning Perception Models to Sensing Parameters}},
  author={Sielemann, Anne and Bitzer, Niklas and Roschani, Masoud and Ziehn, Jens and Beyerer, Juergen},
  booktitle={2026 IEEE 29th International Conference on Intelligent Transportation Systems (ITSC)},
  year={2026}
}

In case of copying and redistributing, or publishing an adapted version of our datasets, please provide the name of our dataset, the creator names, a copyright notice, a link to this website, a license notice with link to the license, and, if changes were made, a disclaimer notice, and a short description of the applied changes. For example, as follows:

This work is based on Sensitivity Analysis of Deep-Learning Perception Models to Sensing Parameters
by Anne Sielemann, Niklas Bitzer, Masoud Roschani, Jens Ziehn, and Juergen Beyerer,
© 2026 Fraunhofer IOSB, All rights reserved.
Link: https://synset.de/datasets/synset-signset-ger/sensitivity-analysis/
Licence: CC BY 4.0
Disclaimer: The original authors are neither affiliated nor responsible for any applied changes.

Paper for Personal Use

External platforms
Dataset (Hugging Face)

Credits

This work was funded in part by the German Federal Ministry for Economic Affairs and Energy (BMWE) within the program “New Vehicle and System Technologies” as part of the AVEAS research project, and conducted within the scope of the KAMO: Karlsruhe Mobility High Performance Center.