Model-Free Inverse H-Infinity Control for Imitation Learning

Abstract

This paper proposes a data-driven model-free inverse reinforcement learning (IRL) algorithm tailored for solving an inverse H-infinity control problem. In the problem, both an expert and a learner engage in H-infinity control to reject disturbances and the learner's objective is to imitate the expert's behavior by reconstructing the expert's performance function through IRL techniques. Introducing zero-sum game principles, we first formulate a model-based single-loop IRL policy iteration algorithm that includes three key steps: updating the policy, action, and performance function using a new correction formula and the standard inverse optimal control principles. Building upon the model-based approach, we propose a model-free single-loop off-policy IRL algorithm that eliminates the need for initial stabilizing policies and prior knowledge of the dynamics of expert and learner. Also, we provide rigorous proof of convergence, stability, and Nash optimality to guarantee the effectiveness and reliability of the proposed algorithms. Furthermore, we show-case the efficiency of our algorithm through simulations and experiments, highlighting its advantages compared to the existing methods.