- 90 views
- 100 downloads
Real Time Recurrent Learning with Complex-Valued Trace Units
-
- Author / Creator
- Elelimy, Esraa MMA
-
Recurrent Neural Networks (RNNs) are typically used to learn representations in partially observable environments. Unfortunately, training RNNs is known to be difficult, and the difficulty increases for agents who learn online and continually interact with the environment. Two common strategies to overcome this difficulty are to approximate gradient-based algorithms for learning the recurrent state or to find a recurrent architecture for which a computationally cheap gradient-based learning algorithm exists. Methods in the second category often limit representational capacity, just as using linear activations or diagonal weight matrices. In this work, we propose a novel recurrent architecture called Recurrent Trace Units (RTUs). RTUs expand representation capacity, but remain inexpensive to train. We derive RT2, a real-time recurrent learning algorithm for RTUs that is tractable, exact, and has linear compute and memory complexities. We investigate performance on a diagnostic benchmark inspired by animal learning and across several partially observable control environments. We show the agents that use RT2 achieve overall better performance when faced with long-term prediction tasks and reach their goals faster in control tasks.
-
- Subjects / Keywords
-
- Graduation date
- Fall 2023
-
- Type of Item
- Thesis
-
- Degree
- Master of Science
-
- License
- This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.