ERA

Download the full-sized PDF of Linear Least-squares Dyna-style PlanningDownload the full-sized PDF

Analytics

Share

Permanent link (DOI): https://doi.org/10.7939/R3QB9V881

Download

Export to: EndNote  |  Zotero  |  Mendeley

Communities

This file is in the following communities:

Computing Science, Department of

Collections

This file is in the following collections:

Technical Reports (Computing Science)

Linear Least-squares Dyna-style Planning Open Access

Descriptions

Author or creator
Yao, Hengshuai
Additional contributors
Subject/Keyword
Recursive least-squares
Gradient-descent
World model
Data efficiency
Reinforcement Learning
Linear Dyna
Computation complexity
Type of item
Computing Science Technical Report
Computing science technical report ID
TR11-04
Language
English
Place
Time
Description
Technical report TR11-04. World model is very important for model-based reinforcement learning. For example, a model is frequently used in Dyna: in learning steps to select actions and in planning steps to project sampled states or features. In this paper we propose least-squares Dyna (LS-Dyna) algorithm to improve the accuracy of the world model and provide better planning. LS-Dyna is a special Dyna architecture in that it estimates the world model by a least-squares method. LS-Dyna is more data efficient, yet it has the same complexity with existing linear Dyna that is based on gradient descent estimation of the world model. Furthermore, the least-squres modeling is computed in an online recursive fashion and does not have to record historical experience or tune a step-size. Experimental results on a 98-state Boyan chain example and a Mountain-car problem show that LS-Dyna performs significantly better than TD/Q-learning and the gradient-descent linear Dyna algorithm.
Date created
2011
DOI
doi:10.7939/R3QB9V881
License information
Creative Commons Attribution 3.0 Unported
Rights

Citation for previous publication

Source
Link to related item

File Details

Date Uploaded
Date Modified
2014-04-24T22:36:01.280+00:00
Audit Status
Audits have not yet been run on this file.
Characterization
File format: pdf (Portable Document Format)
Mime type: application/pdf
File size: 318631
Last modified: 2015:10:12 21:08:01-06:00
Filename: TR11-04.pdf
Original checksum: 5c7169cb13eecaf11ae366b5d6ef4be0
Well formed: true
Valid: true
File title: lsdyna.dvi
Page count: 10
Activity of users you follow
User Activity Date