Advances in Probabilistic Generative Models: Normalizing Flows, Multi-View Learning, and Linear Dynamical Systems

• Author / Creator

  Karami, Mahdi

• This thesis considers some aspects of generative models including my contributions
  in deep probabilistic generative architectures and linear dynamical
  systems.
  First, some advances in deep probabilistic generative models are contributed.
  Flow-based generative modelling is an emerging and highly applicable method
  to construct complex probability density. Herein, I investigate a set of invertible
  convolutional flows based on the circular and symmetric convolutions
  with efficient Jacobian determinant computation and inverse mapping (deconvolution)
  in 𝒪(𝑁 log𝑁) time. Further, an analytic approach to designing
  nonlinear elementwise bijectors is proposed that induces special properties in
  the intermediate layers, by implicitly introducing specific regularizers in the
  loss. It is demonstrated that these transforms allow more effective normalizing
  flow models to be developed for generative image models.
  In the second part, a deep generative framework is expanded to multi-view
  learning. This model is composed of a linear probabilistic multi-view layer in the
  latent space in conjunction with deep generative networks as observation models
  where the variations of each view is captured by a shared latent representation
  and a set of view-specific factors. To approximate the posterior distribution
  of the latent probabilistic multi-view layer, a variational inference approach
  is developed that results in a scalable algorithm for training deep generative
  multi-view neural networks. Empirical studies confirm that the proposed deep generative multi-view model can efficiently integrate the relationship between
  multiple views.
  Finally, the thesis considers maximum likelihood estimation of linear dynamical
  systems (LDS) and develops an optimization based strategy for recovering
  the latent states and transition parameters. Key to the approach is a two-view
  reformulation of maximum likelihood estimation for linear dynamical systems
  that enables the use of global optimization algorithms for matrix factorization.
  It is shown that the proposed estimation strategy outperforms widely-used
  identification algorithms such as subspace identification methods, both in terms
  of accuracy and runtime.

• Subjects / Keywords

  • Machine learning
  • Multi-modal learning
  • Probabilistic generative model
  • Deep learning
  • Linear dynamical systems
  • Normalizing flow
  • Invertible Convolutional Networks
  • Multi-view learning

• Graduation date

  Fall 2020

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/r3-ppv8-jc75

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.