A robot that learns from demonstrations should not just imitate what it sees---it should understand the high-level concepts that are being demonstrated and generalize them to new tasks. Bilevel planning is a hierarchical model-based approach where predicates (relational state abstractions) can be leveraged to achieve compositional generalization.
However, previous bilevel planning approaches depend on predicates that are either hand-engineered or restricted to very simple forms, limiting their scalability to sophisticated, high-dimensional state spaces. To address this limitation, we present IVNTR, the first bilevel planning approach capable of learning neural predicates directly from demonstrations. Our key innovation is a neuro-symbolic bilevel learning framework that mirrors the structure of bilevel planning. In IVNTR, symbolic learning of the predicate "effects" and neural learning of the predicate "classifiers" alternate, with each providing guidance for the other.
We evaluate IVNTR in six diverse robot planning domains, demonstrating its effectiveness in abstracting various continuous and high-dimensional states. While most existing approaches struggle to generalize (with less than 35% success rate), our IVNTR achieves an average success rate of 77% on unseen tasks. Additionally, we showcase IVNTR on a mobile manipulator, where it learns to perform real-world mobile manipulation tasks and generalizes to unseen test scenarios that feature new objects, new states, and longer task horizons. Our findings underscore the promise of learning and planning with abstractions as a path towards high-level generalization.
Bilevel-learning during training and bilevel planning during inference.
(a) During training, IVNTR leverages bilevel learning to invent neural predicates, neural samplers, and symbolic operators
(b) During inference, IVNTR leverages bilevel planning. The planner first uses neural predicates to abstract the high-dimensional states into discrete symbols, and then alternates between search and sampling to find a plan that achieves the goal.
We evaluate IVNTR in six diverse robot planning domains, demonstrating its effectiveness in abstracting various continuous and high-dimensional states. While most existing approaches struggle to generalize (with less than 35% success rate), our IVNTR achieves an average success rate of 77% on unseen tasks.
@inproceedings{Li2025RSS,
author = {Li, Bowen and Silver, Tom and Scherer, Sebastian and Gray, Alex},
title = {{Bilevel Learning for Bilevel Planning}},
booktitle = {Proceedings of the Robotics: Science and Systems (RSS)},
year = {2025}
}