Should Models Be Accurate?

• Author / Creator

  Saleh, Esraa M M

• Learning only by direct interaction with the world can be expensive in many real world applications. In such settings, Model-based Reinforcement Learning (MBRL) methods are a promising avenue towards data-efficiency. By planning with a model, a sequential decision making agent can decrease its reliance on direct interaction with the world. However, when the world is large, complex or seemingly changing, a learned model will be inevitably imperfect. Past work demonstrates that the effects of model imperfection can be difficult to avoid. In this thesis, we question the traditional objective of models that aims for accuracy in simulating the world. A model really only needs to be useful. Inspired by advances in meta-learning, we design a novel model learning loss. We show that a useful but inaccurate model can be learned with this loss so that it matches or surpasses accurate models in performance.

• Subjects / Keywords

  • reinforcement learning
  • planning
  • machine learning
  • model based reinforcement learning
  • meta-learning

• Graduation date

  Fall 2023

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-4rfg-xe29

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.