Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRw 2024) · 2024
Ambiguous Annotations - When is a Pedestrian not a Pedestrian?
Why this publication matters
Some images do not have an obvious single correct label, even for human annotators. This work examines what that ambiguity means for detecting people in street scenes. Recognizing such cases can improve the use of training data and make dataset quality a more thoughtful question than simply counting labeling errors.
Abstract
Datasets labelled by human annotators are widely used in the training and testing of machine learning models. In recent years, researchers are increasingly paying attention to label quality. However, it is not always possible to objectively determine whether an assigned label is correct or not. The present work investigates this ambiguity in the annotation of autonomous driving datasets as an important dimension of data quality. Our experiments show that excluding highly ambiguous data from the training improves model performance of a state-of-the-art pedestrian detector in terms of LAMR, precision and F1 score, thereby saving training time and annotation costs. Furthermore, we demonstrate that, in order to safely remove ambiguous instances and ensure the retained representativeness of the training data, an understanding of the properties of the dataset and class under investigation is crucial.
Figures
Cite this paper
@inproceedings{schwirten2024ambiguousannotationswhen45,
title = {{Ambiguous Annotations - When is a Pedestrian not a Pedestrian?}},
author = {Luisa Schwirten and Jannes Scholz and Daniel Kondermann and Janis Keuper},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRw 2024)},
year = {2024},
url = {https://arxiv.org/pdf/2405.08794}
}
Figures and abstract are reproduced from the linked research sources. Credit remains with the authors and publishers.