Bridging the Theory and Practice of Interactive Imitation Learning

Abstract

Imitation learning (IL) is a general learning paradigm for tackling sequential decision-making problems, leveraging offline expert demonstrations, interactive annotations, or both, with the aim of learning a policy that has performance competitive with the expert, using as few annotations as possible. This thesis focuses on interactive imitation learning, bridging theory and practice by developing provably efficient algorithms together with practical methods. Our contributions, organized by Chapters 4, 5, 6, and 7, are as follows: • First, under the realizable setting, we revisit a recent conclusion that Behavior Cloning (BC)—which relies solely on offline demonstrations—cannot be improved in general, and show that when annotation cost is measured per state, algorithms with interactive annotations provably outperform BC with improvements by up to a horizon factor. • Beyond the realizability assumption, we propose oracle-efficient algorithms for restricted policy classes that achieve improved sample and interaction-round complexity guarantees, complemented by a separate computational lower bound. • Next, we address computational efficiency for general policy classes by designing a new oracle-efficient algorithm and a practical variant, both achieving preferable performance on continuous control benchmarks. • Finally, we introduce hybrid imitation learning (HyIL), where the learner leverages both offline demonstrations and interactive queries, and propose an efficient algorithm with both theoretical guarantees and empirical gains over purely offline or interactive methods. Together, these contributions substantially advance the statistical efficiency and computational tractability of interactive imitation learning, laying a strong foundation for both theoretical progress and practical impact.