Theoretical and Computational Aspects of Mixture Models, with Applications to Empirical Bayes Methods

• Author / Creator

  Tao, Sile

• This thesis studies mixture models, in particular the estimation of mixing
  distributions and their applications to empirical Bayes prediction. The objectives
  are two-fold: to study the large-sample property of empirical Bayes
  estimators; to develop algorithms for the nonparametric estimation of mixing
  distributions as well as methods inspired by the Kiefer-Wolfowitz nonparametric
  maximum likelihood estimator.
  Asymptotic optimality of empirical Bayes estimators is a topic that has
  been in past studied by various authors, starting from Robbins (1956), and
  continued by Deely and Zimmer (1976), Robbins (1964), and Rutherford and
  Krutchkoff (1969). They all worked in somewhat different settings, focusing
  not only on mixture models but the general empirical Bayes methodology.
  Moreover, these authors considered exclusively the squared loss in predictions.
  In this thesis, we establish asymptotic optimality for the empirical Bayes estimators;
  the results apply not only for the squared loss, but for a large class
  of convex loss functions. A consistency result of Bayes estimators for mixture
  models for a large class of convex loss functions is provided under mild
  conditions. Nowadays, decision problems involving alternative loss functions
  other than the squared loss are becoming increasingly popular. For instance,
  Mukherjee, Brown, and Rusmevichientong (2015) have recently applied a parametric
  empirical Bayes method to the so-called newsvendor problem involving
  a piecewise linear loss function. The last chapter of this thesis compares their
  methodology with one that is based on mixture models, and discusses the
  potential of the latter in this field.
  The second part of the thesis is devoted to the estimation of mixing distribution
  in mixture models. Based on the breakthrough of Koenker and Mizera
  (2014), see also Dicker and Zhao (2014), Abadie and Kasy (2017), we propose
  four estimation methods/algorithms. Cutting-Plane Method, which for
  technical reasons comes last, is in fact an alternative algorithm for the Kiefer-
  Wolfowitz nonparametric maximum likelihood estimator studied by Koenker
  and Mizera. However, unlike their algorithm, the Cutting-Plane Method is
  also applicable in higher-dimensional parameter spaces. The same is true
  for the remaining three proposed methods. Projected Stochastic Gradient
  is capable of working in even higher dimensions but its convergence may be
  slow. Stochastic Average Approximation is generally much faster but in some
  versions, its estimation target differs from that of Kiefer-Wolfowitz nonparametric
  maximum likelihood estimator. This is even more true for Constraint
  Resampling, which is in fact an autonomous and novel estimation method;
  its properties, as well as those of other proposed methods are assessed via
  simulations and theoretical results. The penultimate chapter is devoted to
  facilitate the multivariate data-analytical applications of the developed algorithms.
  Nonparametric empirical Bayes methods are studied in the presence
  of explanatory variables. A nonparametric empirical Bayes regression model
  is later proposed. In contrast to some of the previous approaches, such a
  regression model has a very simple form and inherits most of theoretical properties
  of nonparametric empirical Bayes procedures. Unlike methods based on
  the partial linear model, the parameter estimation procedure is equivalent to
  solving a convex optimization problem in function space and can be efficiently
  solved by the proposed algorithms.

• Subjects / Keywords

  • nonparametric empirical Bayes
  • optimization

• Graduation date

  Fall 2018

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/R36T0HC23

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.