ERA

Download the full-sized PDF of Automatic step-size adaptation in incremental supervised learningDownload the full-sized PDF

Analytics

Share

Permanent link (DOI): https://doi.org/10.7939/R31G9D

Download

Export to: EndNote  |  Zotero  |  Mendeley

Communities

This file is in the following communities:

Graduate Studies and Research, Faculty of

Collections

This file is in the following collections:

Theses and Dissertations

Automatic step-size adaptation in incremental supervised learning Open Access

Descriptions

Other title
Subject/Keyword
step size
supervised learning
stochastic gradient descent
Type of item
Thesis
Degree grantor
University of Alberta
Author or creator
Mahmood, Ashique
Supervisor and department
Sutton, Richard S. (Computing Science)
Examining committee member and department
Shah, Sirish L. (Chemical and Materials Engineering)
Sutton, Richard S. (Computing Science)
Schuurmans, Dale (Computing Science)
Department
Department of Computing Science
Specialization

Date accepted
2010-09-30T17:07:52Z
Graduation date
2010-11
Degree
Master of Science
Degree level
Master's
Abstract
Performance and stability of many iterative algorithms such as stochastic gradient descent largely depend on a fixed and scalar step-size parameter. Use of a fixed and scalar step-size value may lead to limited performance in many problems. We study several existing step-size adaptation algorithms in nonstationary, supervised learning problems using simulated and real-world data. We discover that effectiveness of the existing step-size adaptation algorithms requires tuning of a meta parameter across problems. We introduce a new algorithm - Autostep - by combining several new techniques with an existing algorithm, and demonstrate that it can effectively adapt a vector step-size parameter on all of our training and test problems without tuning its meta parameter across them. Autostep is the first step-size adaptation algorithm that can be used in widely different problems with the same setting of all of its parameters.
Language
English
DOI
doi:10.7939/R31G9D
Rights
License granted by Ashique Mahmood (ashique@ualberta.ca) on 2010-09-29T08:36:02Z (GMT): Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of the above terms. The author reserves all other publication and other rights in association with the copyright in the thesis, and except as herein provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.
Citation for previous publication

File Details

Date Uploaded
Date Modified
2014-04-30T22:57:00.453+00:00
Audit Status
Audits have not yet been run on this file.
Characterization
File format: pdf (Portable Document Format)
Mime type: application/pdf
File size: 4528454
Last modified: 2015:10:12 20:33:09-06:00
Filename: Mahmood_Ashique_Fall_2010.pdf
Original checksum: 25562a10dba175a14405ea79a0282d00
Well formed: false
Valid: false
Status message: Lexical error offset=4517817
Page count: 100
Activity of users you follow
User Activity Date