Modeling human motor learning traits using reinforcement learning

Access full-text files

Date

2021-08-12

Authors

Masetty, Bharath

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Learning a new motor skill is a complex process that requires extensive training and practice. Several theories from motor learning, neuroscience, education, and game design suggest that curriculum-based training may be the key to efficient skill acquisition. However, traditional methods for designing such training curricula often result in time-consuming, costly, and potentially ineffective motor skill learning. Systematizing and automating the curriculum generation process may improve humans' motor skill learning process. This work is a stepping stone towards the long-term goal of automating the curriculum generation process for human motor skill learning. Recent advances in artificial intelligence have introduced curriculum learning using reinforcement learning, which has enabled impressive speed-ups in artificial agents' abilities to learn complex tasks. This thesis draws its inspiration from a two-stage hierarchical model of curriculum learning consisting of two learning agents: a student agent that learns a given task and a teacher agent that learns the optimal curriculum for training the student agent. The core idea of this thesis is to bring the two-stage curriculum learning approach to design a curriculum for human motor skill acquisition. To accomplish this, we must replace the student agent with a model of human learning, which poses three main challenges: (1) it is not straightforward to accurately represent humans skill level or state of knowledge; (2) unlike artificial agents, limits exist on human training time and repetitions; and (3) human learning cannot be paused or externally controlled. In this thesis, we address these challenges by creating an artificial representation of human motor learning behavior. Our model of human motor learning is developed in the context of a specific motor task called Reach Ninja. We first model Reach Ninja as a Markov decision process (MDP) to enable RL agents to learn the Reach Ninja task. Using human demonstrations, we then identify the necessary constraints to limit the performance of the RL agent on the Reach Ninja MDP, which brings the learning behavior close to that of humans. The resultant approximate model demonstrates pre-training and post-training performance similar to that of humans. We then design a static curriculum capable of effectively training the artificial agent in our approximate model and test whether the same static curriculum can induce a similar learning behavior in humans. Preliminary tests with human subjects show that training with the same static curriculum did not improve learning efficiency compared to training directly on the target task. Finally, we discuss the methodology for learning a dynamic curriculum based on our model of Reach Ninja and human motor learning

Description

LCSH Subject Headings

Citation