Fitting Sparse Hierarchical Models: Applications to Factorial Designs

• Author / Creator

  Nabipoor Sanjebad, Majid

• We study penalized fitting strategies aimed at sparse model selection of models satisfying certain hierarchical restrictions, in linear models arising from factorial experiments. After discussing various merits of existing approaches, we propose a modification and generalization of the approach of Bien, Taylor and Tibshirani, capable of handling also models with factors with possibly more than two levels. The approach is based on the modified constraint used in conjunction with the group LASSO. The effect of the modified constraint on the selection of main effects and pair interactions is explored. We characterize the solution for both quadratic and logistic loss and give an unbiased Stein-type estimate for the degrees of freedom, the quantity required as the key component for the selection among competing models in regularization. We compare the derived estimates of the degrees of freedom with the existing ones from the literature.\ \We also study properties of certain alternative approaches: for the so-called standardized group LASSO of Simon and Tibshirani, we show first that it remains unchanged under the transformation of Zhao et al., aimed at unifying group weights, and then we characterize the solution of the newly standardized group LASSO. Based on this characterization, we again derive the unbiased estimate of the degrees of freedom. We establish such an estimate of the degrees of freedom also for the overlapped group LASSO of Obozinski et al.\ \We after show that the derived estimates of the degrees of freedom converge, when the tuning parameter converges to zero, to the (true) degrees of freedom of the corresponding constrained least-squares estimator. We investigate certain particular properties of sparse fitting procedures in factorial designs. We establish the connection, for balanced designs, between penalized estimation and traditional constrained least-squares estimators. We also propose methods of selecting the regularization parameter selection based on AIC and BIC. Finally, we show how replications in factorial designs affect the selection process of standardized group LASSO.

• Subjects / Keywords

  • Regularization
  • Factorial designs
  • Group lasso
  • Hierarchy
  • Degrees of freedom

• Graduation date

  Fall 2016

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/R33X83T7H

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.