An Exploration of Predictive Representations of State

• Author / Creator

  Ma, Chen

• The predictive representations hypothesis is that representing the state of the world in terms of predictions about the future will result in good generalization. In this thesis, good generalization is specifically quantified by good learning performance in both accuracy and speed when predicting future experiences of interest. We test the predictive representations hypothesis in different scenarios where predictions of interest vary. We observe that the predictive representations hypothesis does hold in specific scenarios. Inspired by this finding, we propose Predictive State Update (PSU), a state update rule that incrementally computes the next state from the current state, while being aware of current predictions of interest in addition to next increment of experiences. Any existing state representation approach can instantiate the PSU if it summarizes the past incrementally, updating the next state based on the current state and next increment of experiences. We empirically demonstrate that (i) the use of PSU can boost the generalization performance of existing state representation approaches, such as those based on simple recurrent neural networks, LSTM (Long Short-Term Memory) networks, and GRU (Gated Recurrent Units) networks, and (ii) these instantiations of PSU outperform approaches which represent states exclusively using predictions.

• Subjects / Keywords

  • Machine Learning

• Graduation date

  Spring 2020

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-3a2p-ym39

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.