Theoretical and Computational Aspects of Mixture Models, with Applications to Empirical Bayes Methods

  • Author / Creator
    Tao, Sile
  • This thesis studies mixture models, in particular the estimation of mixing
    distributions and their applications to empirical Bayes prediction. The objectives
    are two-fold: to study the large-sample property of empirical Bayes
    estimators; to develop algorithms for the nonparametric estimation of mixing
    distributions as well as methods inspired by the Kiefer-Wolfowitz nonparametric
    maximum likelihood estimator.
    Asymptotic optimality of empirical Bayes estimators is a topic that has
    been in past studied by various authors, starting from Robbins (1956), and
    continued by Deely and Zimmer (1976), Robbins (1964), and Rutherford and
    Krutchkoff (1969). They all worked in somewhat different settings, focusing
    not only on mixture models but the general empirical Bayes methodology.
    Moreover, these authors considered exclusively the squared loss in predictions.
    In this thesis, we establish asymptotic optimality for the empirical Bayes estimators;
    the results apply not only for the squared loss, but for a large class
    of convex loss functions. A consistency result of Bayes estimators for mixture
    models for a large class of convex loss functions is provided under mild
    conditions. Nowadays, decision problems involving alternative loss functions
    other than the squared loss are becoming increasingly popular. For instance,
    Mukherjee, Brown, and Rusmevichientong (2015) have recently applied a parametric
    empirical Bayes method to the so-called newsvendor problem involving
    a piecewise linear loss function. The last chapter of this thesis compares their
    methodology with one that is based on mixture models, and discusses the
    potential of the latter in this field.
    The second part of the thesis is devoted to the estimation of mixing distribution
    in mixture models. Based on the breakthrough of Koenker and Mizera
    (2014), see also Dicker and Zhao (2014), Abadie and Kasy (2017), we propose
    four estimation methods/algorithms. Cutting-Plane Method, which for
    technical reasons comes last, is in fact an alternative algorithm for the Kiefer-
    Wolfowitz nonparametric maximum likelihood estimator studied by Koenker
    and Mizera. However, unlike their algorithm, the Cutting-Plane Method is
    also applicable in higher-dimensional parameter spaces. The same is true
    for the remaining three proposed methods. Projected Stochastic Gradient
    is capable of working in even higher dimensions but its convergence may be
    slow. Stochastic Average Approximation is generally much faster but in some
    versions, its estimation target differs from that of Kiefer-Wolfowitz nonparametric
    maximum likelihood estimator. This is even more true for Constraint
    Resampling, which is in fact an autonomous and novel estimation method;
    its properties, as well as those of other proposed methods are assessed via
    simulations and theoretical results. The penultimate chapter is devoted to
    facilitate the multivariate data-analytical applications of the developed algorithms.
    Nonparametric empirical Bayes methods are studied in the presence
    of explanatory variables. A nonparametric empirical Bayes regression model
    is later proposed. In contrast to some of the previous approaches, such a
    regression model has a very simple form and inherits most of theoretical properties
    of nonparametric empirical Bayes procedures. Unlike methods based on
    the partial linear model, the parameter estimation procedure is equivalent to
    solving a convex optimization problem in function space and can be efficiently
    solved by the proposed algorithms.

  • Subjects / Keywords
  • Graduation date
    Fall 2018
  • Type of Item
  • Degree
    Doctor of Philosophy
  • DOI
  • License
    Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.