On Ensemble Models for Associative Classification

• Author / Creator

  Kabir, Md Rayhan

• Associative classifiers have shown competitive performance with state-of-the-art methods for predicting class labels. In addition to accuracy performance, associative classifiers produce human readable rules for classification which provides an easier way to understand the decision process of the model. Early models of associative classifiers suffered from the limitation of selecting proper threshold values which are dataset specific. Recent work on associative classifiers eliminates that restriction by searching for statistically significant rules. However, a high dimensional feature vector in the training data impacts the performance of the model. In this study we propose Dynamic Ensemble Associative Learning (DEAL) where we use associative classifiers as base learners on feature sub-spaces and a dynamic feature sampling procedure which automatically defines the number of base learners and ensures diversity and completeness among the subset of feature vectors. This method eliminates the limitation of high memory requirement and runtime of recent associative classifiers for training datasets having large feature vectors without jeopardising the accuracy of the model.

  In addition, to better understand the decision process of our model, we propose an ensemble model, Classification by Frequent Association Rules (CFAR) using associative classifiers as base learners. In our approach, instead of using classical ensemble and a voting method, we rank the generated rules based on frequency and select a subset of the rules for predicting class labels. This ensemble approach CFAR also eliminates the limitation of high memory requirement and runtime of recent associative classifiers. This approach removes the noisy rules for the classification process which further enhances the performance of the model in terms of accuracy.

  Further, inspired by the tremendous performance of deep neural networks, we propose Deep Associative Classifier (DAC), an ensemble of associative classifiers that transforms features in a deep model representation. This model has deep neural network like architecture with associative classifiers as a base learner and overcomes some of the limitations of deep neural network architecture as well as associative classifiers.

  We use 10 datasets from the UCI repository to evaluate the performance of the models. We compare our approaches with different machine learning models. All three of our proposed models address the limitation of recent associative classifier of requiring high memory and long runtime along with showing competitive performance in accuracy in contrast to various state-of-the-art classifiers.

• Subjects / Keywords

  • Ensemble Models
  • Associative Classification
  • Explainable Model

• Graduation date

  Spring 2023

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-8m6r-tc92

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.