NeurIPS 21 Workshop - Distribution shifts - connecting methods and applications (DistShift) · 2021
An Empirical Investigation of Model-to-Model Distribution Shifts in Trained Convolutional Filters
Why this publication matters
We usually study AI by looking at its answers, but its learned filters also tell a story. This work compares hundreds of millions of these small pattern detectors across models and tasks. The shared dataset opens a practical route to understanding what transfers between applications and what changes when the training data changes.
Abstract
We present first empirical results from our ongoing investigation of distribution shifts in image data used for various computer vision tasks. Instead of analyzing the original training and test data, we propose to study shifts in the learned weights of trained models. In this work, we focus on the properties of the distributions of dominantly used 3 × 3 convolution filter kernels. We collected and publicly provide a data set with over half a billion filters from hundreds of trained CNNs, using a wide range of data sets, architectures, and vision tasks. Our analysis shows interesting distribution shifts (or the lack thereof) between trained filters along different axes of meta-parameters, like data type, task, architecture, or layer depth. We argue, that the observed properties are a valuable source for further investigation into a better understanding of the impact of shifts in the input data to the generalization abilities of CNN models and novel methods for more robust transfer-learning in this domain. Data available at: https://github.com/paulgavrikov/CNN-Filter-DB/.
Figures
Cite this paper
@inproceedings{gavrikov2021anempiricalinvestigation6,
title = {{An Empirical Investigation of Model-to-Model Distribution Shifts in Trained Convolutional Filters}},
author = {Paul Gavrikov and Janis Keuper},
booktitle = {NeurIPS 21 Workshop - Distribution shifts - connecting methods and applications (DistShift)},
year = {2021},
url = {https://arxiv.org/abs/2201.08465}
}
Figures and abstract are reproduced from the linked research sources. Credit remains with the authors and publishers.