ICML 24 · 2024
Implicit Representations for Constrained Image Segmentation
Why this publication matters
When part of an object is hidden, the visible pixels may not be enough to determine its boundary. This method builds known shape properties directly into the representation of a segmentation. It lets useful geometric knowledge guide the result instead of hoping that a model learns every constraint from examples.
Abstract
Implicit representations allow to use a parametric function that maps (spatial) coordinates to the value that is traditionally stored in each pixel, e.g. RGB values, instead of a discrete grid. This has recently proven quite advantageous as an internal representation for images or scenes for deep learning models. Yet, its potential to ensure certain properties of the solution has not yet been fully explored. In this work, we demonstrate that implicit representations are a powerful tool for enforcing a variety of different geometric constraints in image segmentation. While convexity, star-shape, path-connectedness, periodicity, or symmetry of the (spatial or space-time) region to be segmented are very challenging to enforce for pixel-wise discretizations, a suitable parametrization of an implicit representation, mapping spatial or spatio-temporal coordinates to the likeliness of a pixel belonging to the fore- or background, allows to provably ensure such constraints. Several numerical examples demonstrate that challenging segmentation scenarios can benefit from the inclusion of application-specific constraints, e.g. when occlusions prevent a faithful segmentation with classical approaches.
Figures
Cite this paper
@inproceedings{schneider2024implicitrepresentationsfor50,
title = {{Implicit Representations for Constrained Image Segmentation}},
author = {J. P. Schneider and M. Fatima and J. Lukasik and A. Kolb and Margret Keuper and M. Moeller},
booktitle = {International Conference on Machine Learning},
year = {2024},
url = {https://openreview.net/forum?id=IaV6AgrTUp}
}
Figures and abstract are reproduced from the linked research sources. Credit remains with the authors and publishers.