ERA

Download the full-sized PDF of Convex Latent ModelingDownload the full-sized PDF

Analytics

Share

Permanent link (DOI): https://doi.org/10.7939/R3BC3T87S

Download

Export to: EndNote  |  Zotero  |  Mendeley

Communities

This file is in the following communities:

Graduate Studies and Research, Faculty of

Collections

This file is in the following collections:

Theses and Dissertations

Convex Latent Modeling Open Access

Descriptions

Other title
Subject/Keyword
convex two-layer modeling
convex multi-layer modeling
convex optimization, convex deep learning, deep learning, robust regression
tractable robust modeling
Type of item
Thesis
Degree grantor
University of Alberta
Author or creator
Aslan,Ozlem
Supervisor and department
Schuurmans, Dale (Computing Science)
Szepesvari, Csaba (Computing Science)
Examining committee member and department
Schuurmans, Dale (Computing Science)
Szepesvari, Csaba (Computing Science)
Ray, Nilanjan(Computing Science)
Crammer, Koby (Electrical Engineering, The Technion)
Greiner, Russ (Computing Science)
Department
Department of Computing Science
Specialization
Statistical Machine Learning
Date accepted
2017-03-31T09:43:41Z
Graduation date
2017-06:Spring 2017
Degree
Doctor of Philosophy
Degree level
Doctoral
Abstract
Most machine learning problems can be posed as solving a mathematical program that describes the structure of the prediction problem, usually expressed in terms of carefully chosen losses and regularizers. However, many machine learning problems yield mathematical programs that are not convex in model parameters, forcing the consideration of heuristic optimization strategies that do not provide guarantees of solution quality. The main focus of this thesis is to develop convex approximations of important non-convex machine learning problems; in particular, new convex formulations for deep latent modelling and robust estimation are developed. Training deep predictive models with latent hidden layers poses a hard computational problem: since the model parameters have to be trained jointly with inference over latent variables, the convexity of the training problem is usually destroyed. This thesis first proposes a novel reformulation of supervised training of a twolayer architecture by introducing a latent feature kernel, which allows a rich set of latent feature representations to be captured while still allowing useful convex formulations via semidefinite relaxation. To tackle the resulting computational problem, efficient training algorithms are developed to exploit the specific structure of the problem and overcome the inadequate scaling of general purpose semidefinite solvers. Promising empirical results are obtained that show useful hidden structure can still be captured even in the presence of convex relaxations. The thesis then shows that the two–layer approach can be extended to handle an arbitrary number of latent layers. To achieve this extension, a novel layer loss is proposed that is jointly convex in the adjacent normalized latent feature kernels. An efficient algorithmic approach is then developed for this extended formulation. Again, promising empirical results are obtained that demonstrate improved capabilities over single latent layer models. These results demonstrate the first fully convex formulation of training a deep architecture with an arbitrary number of hidden layers. A final non-convex problem this thesis addresses is robust regression in presence of outliers. Although the field of robust regression is well established, standard estimators in the robust regression literature are not convex and pose intractable computational problems, while robust estimators proposed in the machine learning literature are only robust under unrealistic assumptions. To address these shortcomings, this thesis proposes a new formulation of robust regression that admits a convex relaxation and efficient training algorithm, while still satisfying nontrivial robustness and consistency guarantees.
Language
English
DOI
doi:10.7939/R3BC3T87S
Rights
This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for the purpose of private, scholarly or scientific research. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.
Citation for previous publication

File Details

Date Uploaded
Date Modified
2017-03-31T15:43:41.900+00:00
Audit Status
Audits have not yet been run on this file.
Characterization
File format: pdf (Portable Document Format)
Mime type: application/pdf
File size: 3367590
Last modified: 2017:06:13 12:13:41-06:00
Filename: Aslan_Ozlem_201703_PhD.pdf
Original checksum: b62af72ef14a76dcbb5afe0b51bbd3c5
Well formed: false
Valid: false
Status message: Invalid page tree node offset=1038747
Status message: Unexpected error in findFonts java.lang.ClassCastException: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject cannot be cast to edu.harvard.hul.ois.jhove.module.pdf.PdfDictionary offset=8117
Status message: Invalid object number or object stream offset=3335673
Status message: Invalid object number or object stream offset=3335673
Status message: Invalid object number or object stream offset=3335673
Activity of users you follow
User Activity Date