Keuper Labs
← All publications

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2023) · 2023

Fine-Grained Product Classification on Leaflet Advertisements

Daniel Ladwig, Bianca Lamm, Janis Keuper

RetailFine-Grained RecognitionMultimodal LearningDatasets

Why this publication matters

Two product packages may look almost identical even though their labels identify different items. This work combines the picture with its printed text to recognize fine-grained product differences. Its dataset and experiments show why using both sources of information can be valuable for processing retail leaflets.

Abstract

In this paper, we describe a first publicly available fine-grained product recognition dataset based on leaflet images. Using advertisement leaflets, collected over several years from different European retailers, we provide a total of 41.6k manually annotated product images in 832 classes. Further, we investigate three different approaches for this fine-grained product classification task, Classification by Image, by Text, as well as by Image and Text. The approach ”Classification by Text” uses the text extracted directly from the leaflet product images. We show, that the combination of image and text as input improves the classification of visual difficult to distinguish products. The final model leads to an accuracy of 96.4% with a Top-3 score of 99.2%. https://github.com/ladwigd/Leaflet-Product-Classification

Abstract source ↗

Figures

Product crops from retail leaflets grouped across prices and retailers.
Figure 1. Price monitoring based on printed leaflets is a key data analysis task in retail, which technically can be defined as a finegrained, multi-modal classification problem. We provide a first public dataset with 41.6k annotated samples for this task. View in source ↗
Training set (top) and test set (bottom) of a class.
Figure 2. Training set (top) and test set (bottom) of a class. View in source ↗

Cite this paper

Download .bib 
@inproceedings{ladwig2023finegrainedproduct28,
  title = {{Fine-Grained Product Classification on Leaflet Advertisements}},
  author = {Daniel Ladwig and Bianca Lamm and Janis Keuper},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2023)},
  year = {2023},
  url = {https://arxiv.org/pdf/2305.03706}
}

Figures and abstract are reproduced from the linked research sources. Credit remains with the authors and publishers.