Reinforcement learning strategies support generalization of learned hierarchical knowledge
| dc.contributor.advisor | Preston, Alison | |
| dc.creator | McKee, Connor | |
| dc.date.accessioned | 2021-12-09T14:09:03Z | |
| dc.date.available | 2021-12-09T14:09:03Z | |
| dc.date.issued | 2021 | |
| dc.description.abstract | In our everyday lives, we must learn and utilize context-specific information to inform our decision making. How do we learn what choices to make based on our memories? Prior rodent work has demonstrated that after learning, knowledge becomes organized hierarchically in a context-dependent manner. Here, we quantify the emergence of context-dependent hierarchical knowledge during learning and examine the flexible use of that knowledge to generalize across different scenarios. Participants learned about objects with context-dependent reward values in an X-shaped virtual environment consisting of an elongated, contextually-varying hallway with decision points on either end. First, participants learned the context-dependent object-reward pairings for one set of three objects. Next, they learned the context-dependent object-reward pairings for a new set of three objects. We hypothesized that prior knowledge of the hierarchical structure would generalize to the second set of objects as evidenced by a facilitation in learning rates. Participants gradually learned the context-dependent object-reward pairings during learning. When introduced to the new object set, learning rates did not significantly differ, indicating generalization of the hierarchical reward structure to the new object set. To further quantify how decision making unfolded, we applied three types of reinforcement learning (RL) models to our behavioral data: model-free, model-based (MB), and combination model-based model-free (MBMF). The MB model performed the best at using participants’ past selections to successfully predict future decisions and reward value expectations, indicating that current decisions were guided by prior selections. The MBMF model was best able to represent changes in participant learning across runs, possibly due to the model’s ability to assess different learning strategies. Overall, our results demonstrate that participants learned to flexibly decide which actions were the most adaptive, promoting correct decision-making in a given context. Furthermore, the structure of prior knowledge may support the generalization of learned experience. | en_US |
| dc.description.department | Neuroscience | en_US |
| dc.identifier.uri | https://hdl.handle.net/2152/90786 | |
| dc.identifier.uri | http://dx.doi.org/10.26153/tsw/17705 | |
| dc.language.iso | eng | en_US |
| dc.relation.ispartof | Honors Theses | en_US |
| dc.rights.restriction | Open | en_US |
| dc.subject | Neuroscience | en_US |
| dc.subject | psychology | en_US |
| dc.subject | learning | en_US |
| dc.subject | memory | en_US |
| dc.subject | generalization | en_US |
| dc.subject | navigation | en_US |
| dc.subject | reinforcement learning | en_US |
| dc.subject | computational modeling | en_US |
| dc.title | Reinforcement learning strategies support generalization of learned hierarchical knowledge | en_US |
| dc.type | Thesis | en_US |