Using Prior Data to Facilitate Learning and Inference in New Environments

• Author / Creator

  Wen, Junfeng

• This dissertation demonstrates how to utilize data collected previously from different sources to facilitate learning and inference for a target task. Learning from scratch for a target task or environment can be expensive and time-consuming. To address this problem, we make three contributions in this dissertation: (1) improving the efficiency of classical domain adaptation methods, (2) developing a novel theory and algorithm for multi-source adaptation, and (3) proposing a theoretically sound approach to estimate the stationary distribution of a Markov chain from batch data. For (1), specifically in the covariate shift scenario, classical methods that compute importance weights suffer from computational issues when the sample size is large. We resolve such issues from an optimization perspective by applying the Frank-Wolfe algorithm. For (2), to fully utilize data from not one but multiple sources, we develop a theory and a corresponding algorithm that are suitable for selecting the most relevant sources for adaptation. Finally, for (3), we propose a method to estimate a target stationary distribution from batch data without interacting with the environment. It is a general method that can be applied to many use cases such as off-policy evaluation in reinforcement learning and post-processing MCMC samples. For all three contributions, we provide empirical studies on various tasks and environments, which show that utilizing prior data effectively can indeed improve learning for a target task.

• Subjects / Keywords

  • Transfer Learning
  • Domain Adaptation
  • Reinforcement Learning
  • Machine Learning

• Graduation date

  Spring 2021

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/r3-ay3z-s312

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.