Browsing by Subject "Eye movements"
Now showing 1 - 6 of 6
- Results Per Page
- Sort Options
Item A modular attention hypothesis for modeling visuomotor behaviors(2021-07-24) Zhang, Ruohan; Ballard, Dana H. (Dana Harry), 1946-; Hayhoe, Mary; Stone, Peter; Huth, Alexander; Dayan, PeterIn this dissertation, we explore the hypothesis that complex intelligent behaviors, in vivo, can be decomposed into modules, which are organized in hierarchies and executed in parallel. This organization is similar to a multiprocessing architecture in silico. Biological attention can be viewed as a "process manager" that manages information processing and multiple computations. In this work, we seek to understand and model this modular attention mechanism for humans in a range of behavioral settings. We explain this approach to understanding modular attention at three levels based on David Marr’s paradigm: the computation theory level, the representation and algorithm level, and the hardware implementation level. At the computation theory level, we propose that simple visuomotor behaviors can be broken down into modules that require attention for their execution. At the representation and algorithm level, we model human eye movements and actions in a variety of visuomotor tasks. We collect and publish a large-scale, high-quality dataset of eye movements and actions of humans playing Atari video games. We study the active vision problem by jointly modeling human eye movements and actions, and compare how humans and artificial learning agents play these video games differently. We then propose a modular reinforcement learning model for modeling human subjects’ navigation behaviors in a virtual-reality environment with multiple goals. We further develop a modular inverse reinforcement learning algorithm to efficiently estimate the subjective reward and discount factors associated with each behavioral goal. At the implementation level, we propose a theoretical neuronal communication model named gamma spike multiplexing that allows the cortex to perform multiple computations simultaneously without crosstalk. The model explains how the modular attention hypothesis might be implemented in the biological brain. The end goals of this work are to (1) build models to explain and predict observed human visuomotor behaviors and attention; (2) use these biologically inspired models to develop algorithms for better artificial learning systems.Item Decision-making in the primate brain : formation, location, and causal manipulation(2016-05) Katz, Leor Nadav; Huk, Alexander C.; Aldrich, Richard; Cormack, Lawrence K; Hayhoe, Mary; Priebe, Nicholas; Seidmann, EyalInteraction within the environment relies on the ability to accumulate sensory evidence in favor of a decision. Despite the paramount importance of decision-making to survival, the neural instantiations and computational principles governing the process have remained elusive. In this thesis I consider how sensory evidence is accumulated to guide decisions, and where in the primate brain this process takes place. I report the results of three main experiments. In the first, I test whether sensory evidence is accumulated differentially for motion in the frontoparallel plane (i.e. 2D motion; left/right) compared to motion through depth (i.e. 3D motion; towards/away). I show that integration of 3D motion is different than 2D and likely relies on a mechanism that is distinct. In the second experiment, I test an influential theory in cognitive neuroscience: that neurons in the monkey lateral intraparietal (LIP) cortex accumulate sensory information in favor of a decision communicated by an eye-movement. I found that despite strong correlations between LIP responses and decisions, reversible inactivation of neurons in LIP had no measurable impact on decision-making performance. More generally, I show that decision-related activity does not necessarily play a causal role in choices. In the final experiment, I test whether the process of making a decision stands to influence functions that are decision irrelevant. I found that causally manipulating the amount of sensory evidence available to human observers influenced decision-irrelevant oculomotor commands, suggesting that even during non- oculomotor decisions, oculomotor regions of the brain are recruited. Taken together, the experimental findings reported motivate new ideas about evidence accumulation and advance our understanding of the decision-making process in the primate brain.Item The effect of the length and structure of sentences upon the silent reading process(1931) Holland, Benjamin F.; Gray, Clarence Truman, 1877-1951Item Eye movements, visual search and scene memory in an immersive virtual environment(2014-08) Snyder, Katherine Lorraine; Hayhoe, MaryVisual memory has been demonstrated to play a role in both visual search and attentional prioritization in natural scenes. However, it has been studied predominantly in experimental paradigms using multiple two-dimensional images. Natural experience, however, entails prolonged immersion in a limited number of three-dimensional environments. The goal of the present experiment was to recreate circumstances comparable to natural visual experience in order to evaluate the role of scene memory in guiding eye movements in a natural environment. Subjects performed a continuous visual-search task within an immersive virtual-reality environment over three days. We found that, similar to two-dimensional contexts, viewers rapidly learn the location of objects in the environment over time, and use spatial memory to guide search. Incidental fixations did not provide obvious benefit to subsequent search, suggesting that semantic contextual cues may often be just as efficient, or that many incidentally fixated items are not held in memory in the absence of a specific task. On the third day of the experience in the environment, previous search items changed in color. These items were fixated upon with increased probability relative to control objects, suggesting that memory-guided prioritization (or Surprise) may be a robust mechanisms for attracting gaze to novel features of natural environments, in addition to task factors and simple spatial saliency.Item Modelling visually guided natural locomotion(2021-11-29) Muller, Karl Sungmin; Hayhoe, Mary; Cormack, Lawrence K; Huk, Alexander C; Huth, Alexander; Geisler, Wilson SVision is an active process where an organism must seek out and acquire the information necessary to support different behavioral goals. This makes understanding these behaviors important for understanding how visual processes unfold in the brain, which is adapted to perform the necessary computations for these behaviors. Bipedal locomotion is one such behavior, which is of particular importance in evolutionary history. In this work I examine locomotion over complex terrain using a mobile eye tracker and motion capture system. This allows for an integrated record of eye and body movements, as well as approximation of the retinal input image. Computer vision methods were applied in order to extract visual motion and to reconstruct environment geometry. This allowed an unprecedented opportunity to examine the visuo-motor decision processes controlling locomotion in natural terrain. Our results reveal the statistical regularities in motion signals that depend on gaze angle and terrain, and have implications for how the visual system might process this information. Gaze angle shapes the spatial distribution of both speeds and directions of visual motion, which has implications for how the visual system might account for this relationship. Terrain differences also manifest in motion signals as deviations from flat ground motion, the magnitude of which is correlated with proximity of gaze allocation to the walker. We also find that gaze is partly predictable on the basis of body orientation and image features. Finally we find that foot placement reflects the avoidance of height changes, with the degree of this influence being modulated by subject leg length. Walkers appear to factor this information into their decision making across multiple spatial scales. Thus foot placement reflects a complex interplay between energetic costs and the need for stable footholds, all taking place as walkers maintain their forward momentum. The conclusions drawn from this new dataset, as well as the novelty of the dataset itself are important contributions towards a deeper understanding of how vision is used to guide locomotion in the natural world.Item The role of uncertainty and reward on eye movements in natural tasks(2012-05) Sullivan, Brian Thomas; Hayhoe, Mary; Ballard, Dana Harry; Geisler, Wilson S.; Cormack, Lawrence K.; Pillow, Jonathan W.The human visual system is remarkable for the variety of functions it can be used for and the range of conditions under which it can perform, from the detection of small brightness changes to guiding actions in complex movements. The human eye is foveated and humans continually make eye and body movements to acquire new visual information. The mechanisms that control this acquisition and the associated sequencing of eye movements in natural circumstances are not well understood. While the visual system has highly parallel inputs, the fovea must be moved in a serial fashion. A decision process continually occurs where peripheral information is evaluated and a subsequent fixation target is selected. Prior explanations for fixation selection have largely focused on computer vision algorithms that find image areas with high salience, ones that incorporate reduction of uncertainty or entropy of visual features, as well as heuristic models. However, these methods are not well suited to model natural circumstances where humans are mobile and eye movements are closely coordinated for gathering ongoing task information. Following a computational model of gaze scheduling proposed by Sprague and Ballard (2004), I argue that a systematic explanation of human gaze behavior in complex natural tasks needs to represent task goals, a reward structure for these goals and a representation of uncertainty concerning progress towards those goals. If these variables are represented it is possible to formulate a decision computation for choosing fixation targets based on an expected value from uncertainty weighted reward. I present two studies of human gaze behavior in a simulated driving task that provide evidence of the human visual system’s sensitivity to uncertainty and reward. In these experiments observers tended to more closely monitor an information source if it had a high level of uncertainty but only for information also associated with high reward. Given this behavioral finding, I then present a set of simple candidate models in an attempt to explain how humans schedule the acquisition of information over time. These simple models are shown to be inadequate in describing the process of coordinated information acquisition in driving. I present an extended version of the gaze scheduling model adapted to our particular driving task. This formulation allows ordinal predictions on how humans use reward and uncertainty in the control of eye movements and is generally consistent with observed human behavior. I conclude by reviewing main results and discussing the merits and benefits of the computational models used, possible future behavioral experiments that would serve to more directly test the gaze scheduling model, as well as revisions to future implementations of the model to more appropriately capture human gaze behavior.