Adaptive Decision Making in Dynamic Environments by Artificial and Biological Agents

• Author / Creator

  Wispinski, Nathan J

• The ability to adaptively respond to changing environments is a fundamental aspect of intelligent behaviour. From catching a ball in motion to changing one’s mind in the face of new information, adaptation requires several key cognitive mechanisms, such as the flexible integration of sensorimotor information and the ability to make predictions about the future. In this dissertation, I explore decision mechanisms underlying adaptive decision making in both artificial and biological agents, and the environmental pressures that may give rise to these mechanisms.
  In Chapter 2, we assessed how well human participants can plan an upcoming movement based on a dynamic, but predictable stimulus. Our results showed that how people moved during their decisions reflected information in the moment, despite known neural and movement delays. These results suggest that humans rapidly and accurately integrate visuospatial predictions and estimates of their own temporal limitations to adapt their behaviour to a constantly changing environment.
  In Chapter 3, we developed a deep reinforcement learning agent that learns via rewards to make adaptive decisions. In two tasks with different movement requirements, these artificial agents exhibit “changes of mind”—a behaviour thought to be a hallmark of flexible behaviour. Despite being trained solely with rewards in the absence of biological data, behaviour and neural mechanisms in these agents emerge during reward learning that closely resemble those in primates making similar decisions. These results suggest that the ability to make adaptive decisions similar to many biological agents emerges in artificial agents trained to maximize reward in the face of noisy, temporally evolving information.
  In Chapter 4, we investigated deep reinforcement learning agents in an ecological patch foraging task. Our results showed that these artificial agents learn to patch forage adaptively in patterns similar to biological foragers, and approach optimal patch foraging behaviour. When investigating the mechanisms underlying this behaviour, we find dynamics that closely align with those from foraging theory, and neural recordings from foraging primates thought to give rise to biological adaptation during foraging.
  Overall, I argue that the need to effectively act in dynamic environments contributes to the emergence of computational mechanisms in both artificial and biological learning systems that allow for adaptive behaviour. Further, as the ability to adaptively respond to changing environments is a fundamental aspect of intelligent biological behaviour, I discuss the implications of this work with respect to the emergence of human-like artificial intelligence.

• Subjects / Keywords

  • decision making
  • reinforcement learning
  • deep learning
  • neuroscience
  • foraging
  • motor control

• Graduation date

  Fall 2023

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/r3-2bwt-5e85

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.