Robust Gaussian Process Regression with a mixture of two Gaussian distributions as a noise model

• Author / Creator

  Daemi, Atefeh

• Increasingly many complex processes from the different fields of biological systems, engineering or econometrics are often required to be controlled. Hence, in such cases, we deal with identification of underlying complex processes which is essential for control design, optimization, and process monitoring. However, developing models for complex processes purely based on first principles is a tedious task and sometimes infeasible. Data driven modelling which makes inference about the underlying process based on observations has been considered as a promising alternative in such scenarios. In data driven modelling, a mathematical model describing the relationship between observed measurements is obtained and thus identified model can be utilized to derive equations for prediction of unobserved values. In this thesis, Gaussian process (GP) as a non-parametric model which is a powerful approach to modelling of complex datasets is investigated from a Bayesian point of view. One of the most important applications of Gaussian process models is in regression problems, wherein the output noise is commonly assumed to follow a normal distribution. However, in many practical problems, this assumption is not always realistic. Thus, we propose robust Bayesian methods to reduce the difference between the underlying process and the model, arising from outliers or other disturbances. We propose a mixture of two Gaussian distributions as a non-Gaussian likelihood for the noise model to capture both regular noise and irregular noise, so-called outliers, thereby making the regression model to be robust to the occurrence of outliers. We present an Expectation Maximization (EM) algorithm-based approach to making approximate inference possible for learning the proposed robust GP regression model. The proposed method is compared with other robust regression GP existing in the literature from a predictive performance perspective. In this thesis, we also explore the problem of a new robust GP regression in which the input presented to the model is noisy. To address this problem, we assume that the input noise is of an independently and identically distributed (i.i.d.) Gaussian noise and the output noise model is assumed to be distributed according to a mixture of two Gaussian distributions to capture both regular and irregular noises. We utilize the Expectation Maximization (EM) based algorithm that involves the errors-in-variables (EIV) to approximate the predictive distribution with a Gaussian process whose kernel function relies on both the input noise and the output noise hyper-parameters. Further, the improved performance of our proposed method is demonstrated by several illustrative examples. The proposed robust GP with a Gaussian mixture noise model is also utilized for modelling nonlinear dynamic systems. In time series models based on the robust GP, we assume that the underlying process maps past observations and external inputs to the current observation, wherein the proposed robust GP with noisy input is employed for the multiple steps ahead prediction. It means that the whole predictive distribution of the output at any time step is fed back into the model for the next time-step which is considered as a noisy input to the model. Thus, the proposed model for nonlinear functions with input and output noise is used to learn true dynamics of the system which has been corrupted by outliers and to predict the output for multiple steps ahead in time. The effectiveness of the proposed approach is illustrated on both synthetic data and simulated Mackey-Glass chaotic time-series.

• Subjects / Keywords

  • Mixture of Gaussian distributions
  • EM algorithm
  • Noisy input
  • Gaussian Process
  • Dynamic modeling

• Graduation date

  Spring 2018

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/R31V5BV6X

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.