Learning with Artificial Neural Networks

• Author / Creator

  Zhang, Shangtong

• In this thesis, we make two contributions in learning with artificial neural networks. Artificial neural networks have made great success in various challenging domains.

  Our first contribution is a new technique named cross-propagation that does cross-validation online. In cross-validation, hold-out training data (i.e., validation set) is used to tune hyper-parameters (e.g., step size) of an algorithm. The key idea of cross-propagation is to use the newly coming training example as a hold-out validation set to update parameters (e.g., weights of a neural network) of an algorithm. We propose three cross-propagation-based algorithms to train neural networks and show the advantage of the three algorithms empirically in both online and off-line setting.

  Our second contribution is a systematic evaluation of experience replay, a method that is commonly used in modern deep reinforcement learning systems to stabilize the training of neural networks. Experience replay involves storing transitions into a memory and training the agent with sampled transitions from the memory. In this thesis, we rethink the utility of experience replay. It introduces a new hyper-parameter, the memory size, which is a task-dependent hyper-parameter. We further propose a simple new experience replay method which requires only little extra computation and made the reinforcement learning system more robust to the selection of the memory size compared with the original experience replay.

• Subjects / Keywords

  • experience replay
  • cross-propagation

• Graduation date

  Fall 2018

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/R3057D80S

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.