An Empirical Study to Repair Deep Object Detectors

• Author / Creator

  Yue, Yuxin

• Visual object detection predicts the categories of objects in an image and estimates bounding boxes that can wrap those objects accurately, playing a crucial role in many vision-based AI systems like autonomous cars, robotics, and smart monitoring. Although achieving significant progress, even the state-of-the-art (SOTA) detectors could inevitably raise errors when we deploy them in the real-world application scenario due to the potential distribution shift between the training dataset and testing data. For example, the detectors trained on a clean dataset usually yield a significant precision reduction under the corrupted images (e.g., noisy or rainy images). Such an error type is often inevitable in the operational environment where the system is deployed, and cannot be completely solved by modifying the network architecture or optimization algorithm either since the deployed detectors could unavoidably encounter unseen corruptions.
  A promising and practical solution is to use the examples captured in the operational environment to guide the updating of the object detector and avoid similar errors. So far, there are few works studying object detection from the repairing perspective, which hinders the deployment of SOTA object detectors and a better understanding of their limitations. To bridge this gap, we conduct an in-depth and extensive empirical study to benchmark the object detection repairing techniques for DetRepair tasks. Our goal is to understand the current status, limitations, and challenges of DetRepair, and to identify future opportunities for the research community.
  Specifically, we build a detector repairing benchmark by considering different failure patterns of objects and different error types and by collecting state-of-the-art detectors and various corruptions. The analysis enables us to gain a deep understanding and insight into the failure patterns and the underlying potential causes. Then based on clues from the analysis, we systematically and comprehensively study a series of repairing schemes and conduct extensive experiments on different schemes based on the constructed benchmark. We notice many inspirational facts on repairing schemes for object detectors. With all the above efforts and the evaluation results, we can understand the significance and challenges of this task and the strengths and weaknesses of different solutions. Furthermore, we propose several potential future directions based on our benchmark, opening new doors for developing robust object detectors.

• Subjects / Keywords

  • DNN Repairing
  • Object Detection
  • Software Engineering

• Graduation date

  Spring 2024

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-dyzf-5x59

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.