Keuper Labs
← All publications

ICML 24 · 2024

CosPGD - an efficient white-box adversarial attack for pixel-wise prediction tasks

Shashank Agnihotri, Stefen Jung, Margret Keuper

Adversarial RobustnessOptical FlowSemantic SegmentationImage Restoration

Why this publication matters

Systems that estimate motion or label every pixel need stress tests that go beyond ordinary image classification. CosPGD provides a shared way to construct challenging inputs across these tasks. It helps researchers compare reliability and uncover weaknesses that clean-image accuracy alone can conceal.

Abstract

While neural networks allow highly accurate predictions in many tasks, their lack of robustness towards even slight input perturbations often hampers their deployment. Adversarial attacks such as the seminal projected gradient descent (PGD) offer an effective means to evaluate a model’s robustness and dedicated solutions have been proposed for attacks on semantic segmentation or optical flow estimation. While they attempt to increase the attack’s efficiency, a further objective is to balance its effect, so that it acts on the entire image domain instead of isolated pointwise predictions. This often comes at the cost of optimization stability and thus efficiency. Here, we propose CosPGD, an attack that encourages more balanced errors over the entire image domain while increasing the attack’s overall efficiency. To this end, CosPGD leverages a simple alignment score computed from any pixelwise prediction and its target to scale the loss in a smooth and fully differentiable way. It leads to efficient evaluations of a model’s robustness for semantic segmentation as well as regression models (such as optical flow, disparity estimation, or image restoration), and it allows it to outperform the previous SotA attack on semantic segmentation. We provide code for the CosPGD algorithm and example usage at https://github.com/shashankskagnihotri/cospgd.

Abstract source ↗

Figures

Optical-flow predictions before and after PGD and CosPGD attacks.
Figure 1. Optical flow predictions using RAFT (Teed & Deng, 2020) on Sintel (Butler et al., 2012; Wulff et al., 2012) validation. (a) and (b) show two consecutive frames for which the initial optical flow in (d) was predicted. The results of attacking the model with target −→0 (c) are depicted in (e) for PGD and (f) for CosPGD. For the same perturbation magnitude and number of iterations, the proposed CosPGD alters the estimated optical flow more strongly and brings it closer to target (c). View in source ↗
Change in pixel-wise image gradients over attack iterations on DeepLabV3 performing semantic segmentation on PASCAL VOC 2012 validation subset.
Figure 2. Change in pixel-wise image gradients over attack iterations on DeepLabV3 performing semantic segmentation on PASCAL VOC 2012 validation subset. We observe that the absolute difference between gradient values (top) is larger for PGD and increasing for SegPGD, while being stable for CosPGD. Further, CosPGD has fewer changes in gradient direction over attack iterations (bottom) compared to PGD and SegPGD. This shows CosPGD is more stable during optimization compared to PGD and SegPGD. View in source ↗

Cite this paper

Download .bib 
@inproceedings{agnihotri2024cospgdanefficient48,
  title = {{CosPGD - an efficient white-box adversarial attack for pixel-wise prediction tasks}},
  author = {Shashank Agnihotri and Stefen Jung and Margret Keuper},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=CXZqGJonmt}
}

Figures and abstract are reproduced from the linked research sources. Credit remains with the authors and publishers.