Browsing by Subject "Sequential decision-making"

Now showing 1 - 3 of 3

Algorithms for cooperative and competitive autonomous systems
(2022-04-26) Savas, Yusuf Yagiz; Topcu, Ufuk; Tanaka, Takashi; Bakolas, Efstathios; Fridovich-Keil, David; Zhu, Yuke
Autonomous systems no longer operate in isolation. In many applications, ranging from ride-hailing to on-demand delivery and surveillance, they carry out tasks in the presence of other autonomous systems and humans. For successful operations in these applications, autonomous systems need to reason about uncertainty to co-exist alongside other agents, understand motivations to cooperate with friends, and strategically manipulate information to compete against adversaries. In this dissertation, we present novel cooperation and competition capabilities for autonomous systems to accomplish tasks with theoretical performance guarantees in the presence of other agents. First, we focus on cooperation and develop algorithms for an autonomous agent to influence the behavior of another agent through sequential incentive offers. Second, we consider an autonomous agent operating in adversarial environments and develop algorithms for the agent to compete against adversaries by minimizing the predictability of its goal-directed behavior. Third, we develop an algorithm for an autonomous agent to deceive outside observers regarding its intentions while carrying out tasks in stochastic environments. For each of these capabilities, we model the agent behavior via Markov decision processes and present a comprehensive theoretical analysis that establishes conditions for the existence of optimal solutions and the complexity of computing those solutions. Furthermore, we contribute to the theory of Markov decision processes by presenting an analysis for a new formulation that combines total discounted cost criterion with a reachability constraint.
Efficient, reliable, and interpretable decision-making for human-autonomy co-existence
(2021-08) Ghasemi, Mahsa; Topcu, Ufuk; de Veciana, Gustavo; Dimakis, Alexandros; Niekum, Scott; Kira, Zsolt
The ever-increasing presence of autonomy in our lives calls for immediate and significant investment in the development of effective, safe, and trustworthy autonomous systems. In the real world, these systems need to make sequential decisions in complex and dynamic environments while interacting with human decision-makers. Furthermore, they may have access to an immense amount of heterogeneous data, which requires them to be selective about what data to gather due to their limited perception capability. I focus on developing theory and algorithms that enable autonomous agents to operate in our complex world. The algorithms I develop address the following fundamental challenges: 1) active identification and gathering of actionable information from large-scale, multi-modal, and noisy data, 2) planning and learning based on partial and evolving knowledge, in real time and with theoretical performance guarantees, and 3) interacting and collaborating with humans in a trustworthy manner. Given the multidimensionality of these challenges, the proposed algorithms rely on drawing new connections between the fields of control, learning, formal methods, and information theory. The first part of my contributions entails introducing joint active perception and planning algorithms in unknown or uncertain environments so that a decision-maker can efficiently find the most relevant data for its task. In the second part of my contributions, I propose online planning algorithms capable of integrating runtime data, from the environment or a human user, into sequential decision-making reliably. The third part of my contributions presents planning algorithms for tasks described in temporal logic languages such that their outputs are interpretable by human users. The proposed interdisciplinary ideas enhance the efficiency, reliability, and interpretability of automated sequential decision-making — taking a step toward designing competent, autonomous systems that co-exist with humans.
Sequential decision making in artificial musical intelligence
(2019-03-22) Liebman, Elad; Stone, Peter, 1971-; Dannenberg, Roger; Grauman, Kristen; Niekum, Scott; Saar-Tsechansky, Maytal
Over the past 60 years, artificial intelligence has grown from a largely academic field of research to a ubiquitous array of tools and approaches used in everyday technology. Despite its many recent successes and growing prevalence, certain meaningful facets of computational intelligence have not been as thoroughly explored. Such additional facets cover a wide array of complex mental tasks which humans carry out easily, yet are difficult for computers to mimic. A prime example of a domain in which human intelligence thrives, but machine understanding is still fairly limited, is music. Over the last decade, many researchers have applied computational tools to carry out tasks such as genre identification, music summarization, music database querying, and melodic segmentation. While these are all useful algorithmic solutions, we are still a long way from constructing complete music agents, able to mimic (at least partially) the complexity with which humans approach music. One key aspect which hasn't been sufficiently studied is that of sequential decision making in musical intelligence. This thesis strives to answer the following question: Can a sequential decision making perspective guide us in the creation of better music agents, and social agents in general? And if so, how? More specifically, this thesis focuses on two aspects of musical intelligence: music recommendation and human-agent (and more generally agent-agent) interaction in the context of music. The key contributions of this thesis are the design of better music playlist recommendation algorithms; the design of algorithms for tracking user preferences over time; new approaches for modeling people's behavior in situations that involve music; and the design of agents capable of meaningful interaction with humans and other agents in a setting where music plays a roll (either directly or indirectly). Though motivated primarily by music-related tasks, and focusing largely on people's musical preferences, this thesis also establishes that insights from music-specific case studies can also be applicable in other concrete social domains, such as different types of content recommendation. Showing the generality of insights from musical data in other contexts serves as evidence for the utility of music domains as testbeds for the development of general artificial intelligence techniques. Ultimately, this thesis demonstrates the overall usefulness of taking a sequential decision making approach in settings previously unexplored from this perspective