Belief-directed exploration in human decision-makers : behavioral and physiological evidence

Decision-making in uncertain environments poses a conflict between the goals of exploiting past knowledge in order to maximize rewards and exploring less-known options in order to gather information. The descriptive modeling framework utilized in previous studies of exploratory choice behavior characterizes exploration as the result of a noisy decision process, rather than a process reflecting beliefs and/or uncertainty about the environment. It stands to reason that people do not merely negotiate the exploration-exploitation dilemma by stochastically making choices, but rather, fully utilize their knowledge of the environment structure and integrate their trial-by-trial observations of choice in order to direct exploratory choice. The work presented in this dissertation evaluates this hypothesis. As the previous used tasks structures and descriptive models obfuscate this more sophisticated form of belief-directed exploration, I describe a novel exploration-exploitation task that affords disentanglement of reflective belief-directed exploration strategy from a reflexive and naïve exploration strategy. The former strategy is distinguished from latter by its ability to update its belief states in the absence of direct observations of choice payoff changes. Accordingly, we specify cognitive models instantiating these two choice strategies and in the first experiment, we find evidence that behavior is by and large better characterized by a reflective strategy, and further, that choice latencies appear to index value computations carried out in implementing such a strategy. In a second experiment, I reveal how physiological arousal (measured by Skin Conductance Responses) appears to index a form of value computation similar to what is prescribed this reflective model, and further, how individual differences in physiological response to these value signals bear on choice behavior. In a third experiment, I demonstrate how this sophisticated form of choice behavior carries cognitive costs, and following the contemporary model-based/model-free reinforcement learning framework, I show how placing concurrent decision-makers under cognitive load diminishes the contribution of the more sophisticated reflective exploration strategy, fostering reliance on stochastic, reflexive form of exploratory choice behavior.