Engineering artificial intelligence systems for privacy

This dissertation addresses the increasing need for privacy-aware artificial intelligence (AI) systems and investigates engineering approaches that strike a balance between maximizing performance and minimizing potential privacy threats. The privacy analysis methodology followed throughout this study uses the Contextual Integrity Theorem's framing of privacy as "appropriate information flows" to find a common vocabulary between the social concept of privacy and mathematical AI algorithms. The analysis begins with a technical privacy policy that describes the information flows that are appropriate for a given AI task. Once these permissible information flows are determined, the next step is to develop engineering approaches to optimize performance within the confinements of the technical privacy policy, which we refer to as privacy engineering. We study privacy engineering for AI systems in two sequential decision-making domains, namely policy synthesis and reinforcement learning. In both domains, agents must strategize their decisions to achieve an ultimate long-term goal, such as a robot navigating to a predetermined location. For policy synthesis, we study privacy engineering in the context of Markov decision processes (MDPs), which are abstract models of an environment consisting of states, actions, transition probabilities, and rewards. The privacy priority considered for this problem is to protect the confidentiality of the MDP's transition probabilities. Such a privacy priority is particularly relevant if the disclosure of the environment model to unauthorized parties could be harmful, such as the models that businesses develop to predict competitive market trends. For this problem, we identify differential privacy as an appropriate technical mechanism to achieve the set privacy goals and make two main contributions. First, we introduce the Dirichlet mechanism for enforcing differential privacy on simplex-valued data—which includes transition probabilities in MDPs with finite states and actions. Then, we use the Dirichlet mechanism to develop a differentially private policy synthesis algorithm. For the reinforcement learning problem, we study privacy engineering in the context of cooperative multi-agent reinforcement learning in which a team of agents must learn a common task through trial and error. For this problem, we assume that information disclosures about the agents' individual interactions with the environment violate privacy. We demonstrate that numerous existing algorithmic solutions rely on sharing environment interactions. Consequently, we introduce alternative privacy-engineered algorithms that establish permissible data-sharing frameworks according to the set technical privacy policy. The contributions of this dissertation demonstrate that privacy and AI can indeed be reconciled via privacy engineering. The findings highlight future research opportunities to design and implement AI algorithms with privacy as a priority.