Endowing human-centered behaviors to single and multiple robots for safe, robust, and efficient operation in human environments

Kim, Minkyu, Ph. D.
Journal Title
Journal ISSN
Volume Title

Future tasks for robots to support humans in complex, dynamic, or human-populated environments will increasingly require more awareness of human behavior to operate effectively and safety. Human-centered behaviors are behaviors that seek to improve the interaction between humans and robots. For example, predicting the awareness of humans on nearby robots allows robots to be more "respectful" to their surroundings when navigating near them. Or another example, predicting where humans are heading to or what rooms they might be occupying can allow robots to follow and find people more effectively. In order for robots to be safe one of the top priorities is to know if nearby humans or pedestrians are aware of their presence. As such, in this dissertation we develop techniques to detect the gaze of humans within the robot's field of view and use this sensing modality to reason about awareness. For example, if a person is looking directly at a robot for a few seconds we assume that the person is aware of the robot moving nearby. We exploit this capability to plan paths for robots that are both safe – i.e. robots stay away within a few meters if humans are not looking – and respectful – i.e. by staying away from unaware humans they don't interrupt the tasks that humans might be performing. These considerations are important for safe and socially respectful navigation of mobile robots in human environments. Another important behavior of mobile robots is their ability to follow humans around in a "human-like" manner. By human-like we mean that we use inspiration about how people follow others in cluttered and dynamic environments. For instance, when a group of friends goes to a sports event some members of the group might temporally get lost. Those people may use inference regarding the heading direction of the group or contextual information – e.g. the group said they would head to a particular location – in order to rejoin them. As such, we have incorporate inference methods for robots to predict heading trajectories of humans as well as using contextual information of possible human locations in order to follow or relocate human teammates. A third important topic towards human-centered mobile robot behaviors is their ability to locate missing people, objects, or buildings in urban areas. This kind of capability is important for applications such as search and rescue or finding people in the crowd to deliver items, for instance. We have extensively explored such capability both in indoor and outdoor environments. Due to the complexity of searching in large spaces with many rooms, alleys, roads, buildings, etc, we've taken the approach of using multiple robots to accomplish such tasks more quickly. For indoor setups we've employed multiple robots including small (but fast) quadrupedal robots in coordination with wheeled robots. For outdoor setups we've employed golf-cart sized wheeled robots in coordination with Clearpath robots to sweep environments in search of missing objects. Overall, this dissertation has sought to improve the interaction and safety between mobile robots and humans. Within this area that I call human-centered mobile navigation my contributions include: (1) the integration of human awareness of robots in the state and reward models for POMDP optimization to improve social navigation in human environments; (2) exploring robust and autonomous person-following capabilities using active search in the sense that robots aim to recover individual people when they disappear from their field of view using multi-modal sensing and predictions; (3) using prior knowledge related to contextual information for statistical inference of possible object locations in case of occlusions or objects going missing; (4) realizing heterogeneous coverage path planning algorithms using clustering and information-theoretic optimization techniques that are faster than the state-of-the-art; (5) implementing all of these methods in a variety of hardware including wheeled and quadrupedal robots as well as teams of multiple robots working together.