Parameterized modular inverse reinforcement learning

Abstract

Reinforcement learning and inverse reinforcement learning can be used to model and understand human behaviors. However, due to the curse of dimensionality, their use as a model for human behavior has been limited. Inspired by observed natural behaviors, one approach is to decompose complex tasks into independent sub-tasks, or modules. Using this approach, we extended earlier work on modular inverse reinforcement learning, and developed what we called a parameterized modular inverse reinforcement learning algorithm. We first demonstrate the correctness and efficiency of our algorithm in a simulated navigation task. We then show that our algorithm is able to estimate a reward function and discount factor for real human navigation behaviors in a virtual environment, and train an agent that imitates the behavior of human subjects.