Keuper Labs
← All publications

AutoML 2026 Methods Track · 2026

IBUS: Overcoming Structural Biases in Hierarchical NAS with Iterative Bottom-Up Sampling

Abay Artykbayev, Martin Rapp, Benedikt Staffler, Margret Keuper

Neural Architecture SearchSampling

Why this publication matters

A flexible search space is only useful if sampling can reach its important architectural patterns. This study shows that top-down sampling in einspace rarely produces skip-connections or convolutions, then introduces IBUS to build larger networks from compatible smaller blocks. Across seven image, text, and other classification tasks, IBUS achieves higher mean accuracy than einspace with the same evolutionary search settings, with gains of up to 13.7 percentage points. It can also combine blocks from ResNets, vision transformers, and MLP-Mixers to explore hybrid architectures.

Abstract

Most search spaces for neural architecture search (NAS) rely on a fixed macro-structure limiting their expressiveness. Recent works utilize context-free grammars (CFGs) to design expressive hierarchical search spaces that contain multiple architecture families. However, we found that such search spaces underrepresent certain crucial architectural patterns, e.g., skip-connections, due to inherent structural biases. We propose a new search space with a novel iterative bottom-up sampling approach, IBUS, to resolve such structural biases. IBUS also naturally enables using parts of existing architectures, e.g., ResNets and ViTs, to sample hybrid architectures. Our experiments show that IBUS consistently outperforms the current state-of-the-art CFG-based search space on all evaluation tasks, with final accuracy increases of up to 13.7%. Our code is available at https://github.com/boschresearch/ibus-nas.

Abstract source ↗

Figures

IBUS module types and the iterative process of selecting, validating, and adding building blocks to a growing pool.
Figure 1. Overview of our iterative bottom-up sampling IBUS. Starting from primitives, each sampling step composes existing building blocks using a random module type to create the next building block. This is repeated iteratively to sample complex architectures from bottom-up. View in source ↗
Counts of uniquely sampled convolutions and skip-connections in 2,000 architectures, comparing IBUS with einspace.
Figure 2. Numbers of uniquely sampled convolutions and skip-connections across 2000 randomly sampled architectures for IBUS and einspace. Overall, IBUS samples significantly more skip-connections and convolutions than einspace, ensuring that these crucial architectural patterns are not underrepresented. View in source ↗

Cite this paper

Download .bib 
@inproceedings{artykbayev2026ibus,
  title = {{IBUS}: Overcoming Structural Biases in Hierarchical {NAS} with Iterative Bottom-Up Sampling},
  author = {Abay Artykbayev and Martin Rapp and Benedikt Staffler and Margret Keuper},
  booktitle = {AutoML 2026 Methods Track},
  year = {2026},
  url = {https://openreview.net/forum?id=AI5qzCymtR}
}

Figures and abstract are reproduced from the linked research sources. Credit remains with the authors and publishers.