Distributional Losses for Regression

• Author / Creator

  Imani, Ehsan

• In this thesis we introduce a new loss for regression, the Histogram Loss. There is some evidence that, in the problem of sequential decision making, estimating the full distribution of return offers a considerable gain in performance, even though only the mean of that distribution is used in decision making. A parallel line of research in classification has found that converting hard one-hot targets to soft targets, distributions that contain information about the relationship between classes or ambiguity in the label, can improve accuracy. These findings have given rise to questions about the underlying reasons that are still left unanswered. Our proposed loss function is influenced by these two ideas and involves learning the conditional distribution of the target variable by minimizing KL-divergence between a target distribution and a flexible histogram prediction. Experiments on four datasets show that the Histogram Loss often outperforms commonly used regression losses. We then design theoretical and empirical analyses to determine why and when this performance gain appears, and how different components of the loss contribute to it. Through this investigation we also provide additional insights about open questions and hypotheses posed in previous works.

• Subjects / Keywords

  • Neural Networks
  • Histogram Loss
  • Regression
  • Machine Learning

• Graduation date

  Spring 2019

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-2g1z-a587

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.