Inverse problems in non-cooperative games : learning and designing multi-agent interactions

Access full-text files

Date

2023-05

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Non-cooperative games offer a formal way to model the interactions of multiple autonomous players. This thesis investigates two inverse problems in non-cooperative, general-sum, multi-player games. The cost design problem aims at designing costs to achieve socially beneficial interactions among players in atomic routing games. We use an entropy-regularized Nash equilibrium to solve a given atomic routing game and devise an approximate projected gradient method to design the cost that minimizes social costs by leveraging the implicit function theorem. Numerical results on a GridWorld example show the effectiveness of the proposed algorithm. The inverse learning problem aims to infer reward parameters from observed interactions of the players in affine Markov games. We formulate affine Markov games as a class of Markov games where each player has independent dynamics and actions coupled by an affine reward function. We then introduce the soft-Bellman equilibrium as a novel solution concept and establish the conditions for the existence and uniqueness of such equilibria in affine Markov games. We propose an algorithm to solve this inverse problem and show that it outperforms a baseline algorithm that ignores the coupling between the players in a predator-prey OpenAI Gym environment by reducing the Kullback-Leibler divergence between the equilibrium policies and observed policies by at least two orders of magnitude.

Description

LCSH Subject Headings

Citation