ICML 22 Workshop on NEW FRONTIERS IN ADVERSARIAL MACHINE LEARNING · 2022

Robust Models are less Over-Confident

Julia Grabinski, Paul Gavrikov, Janis Keuper, Margret Keuper

Why this publication matters

An incorrect prediction is especially troublesome when a model presents it with great confidence. This work finds that training models to resist adversarial changes can also make their confidence more cautious on ordinary inputs. It highlights a useful connection between resisting attacks and communicating uncertainty, while showing that network design matters too.

Abstract

Despite the success of convolutional neural networks (CNNs) in many academic benchmarks for computer vision tasks, their application in the real-world is still facing fundamental challenges. One of these open problems is the inherent lack of robustness, unveiled by the striking effectiveness of adversarial attacks. Current attack methods are able to manipulate the network’s prediction by adding specific but small amounts of noise to the input. In turn, adversarial training (AT) aims to achieve robustness against such attacks and ideally a better model generalization ability by including adversarial samples in the trainingset. However, an in-depth analysis of the resulting robust models beyond adversarial robustness is still pending. In this paper, we empirically analyze a variety of adversarially trained models that achieve high robust accuracies when facing state-of-the-art attacks and we show that AT has an interesting side-effect: it leads to models that are significantly less overconfident with their decisions, even on clean data than non-robust models. Further, our analysis of robust models shows that not only AT but also the model’s building blocks (like activation functions and pooling) have a strong influence on the models’ prediction confidences. Data & Project website: https://github.com/GeJulia/robustness_confidences_evaluation

Abstract source ↗

Figures

Prediction confidence distributions for standard and robust classifiers. — **Figure 1.** Mean model confidences on their correct (x-axis) and incorrect (y-axis) predictions over the full CIFAR10 dataset (top) and CIFAR100 dataset (bottom), clean (left) and perturbed with the attacks PGD (middle) and Squares (right). Each point represents a model. Circular points (purple color-map) represent non-robust models and diamond-shaped points (green color-map) represent robust models. The color of each point represents the models accuracy, darker signifies higher accuracy (better) on the given data samples. The star in the bottom right corner indicates the optimal model calibration and the gray area marks the area were the confidence distribution of the network is worse than random, i.e. more confident in incorrect predictions than in correct ones. View in source ↗

**Figure 2.** Overconfidence (lower is better) bar plots of robust models and their non-robust counterparts trained on CIFAR10. Non-robust models are highly overconfident, in contrast, their robust counterparts are less over-confident. View in source ↗

Cite this paper

Download .bib

@inproceedings{grabinski2022robustmodelsare11,
  title = {{Robust Models are less Over-Confident}},
  author = {Julia Grabinski and Paul Gavrikov and Janis Keuper and Margret Keuper},
  booktitle = {ICML 22 Workshop on NEW FRONTIERS IN ADVERSARIAL MACHINE LEARNING},
  year = {2022},
  url = {https://advml-frontier.github.io/pdf/12/CameraReady/ICML_Workshop_Adv_2022.pdf}
}

Figures and abstract are reproduced from the linked research sources. Credit remains with the authors and publishers.