Visual Salient Object Detection: Interactive, Unsupervised and Supervised Methods

• Author / Creator

  Qin, Xuebin

• The human vision system has an effective mechanism for retrieving and localizing the most important information from visual scenes. In computer vision,Salient Object Detection (SOD) algorithms aim at modeling this mechanism by extracting or segmenting these salient targets from given images or video frames. Such algorithms can be used in a wide range of applications such as image segmentation, image editing, visual tracking, robot navigation, etc. Three categories of salient object detection methods are studied in this thesis. 1) Interactive Annotation with ByLabel: A Boundary based Semi-Automatic Image Annotation Tool. We develop a novel boundary based semi-automatic tool, ByLabel, for accurate image annotation. Given an image, ByLabel first detects its edge features and computes high-quality boundary fragments. Current labeling tools require the operators to accurately click on numerous boundary points. ByLabel simplifies this process to selecting of the automatically detected boundary fragments. To evaluate the performance of ByLabel, 10 volunteers, with no experience of image annotation, were asked to label both synthetic and real images. Compared to the commonly used tool LabelMe, ByLabel reduces image-clicks and time by 73% and 56% respectively, while improving the accuracy by 73%. The results show that our ByLabel outperforms the popular annotation tool, LabelMe, in terms of efficiency, accuracy and user experience. 2) Unsupervised Salient Closed Boundary Extraction by Perceptual Grouping. Salient closed boundary extraction aims to automatically identify and connect a subset of detected fragments to form a closed boundary based on the principles of Gestalt laws. Particularly, we propose a novel method for perceptual grouping of the salient closed boundary, in which the salient closed boundary extraction problem is formulated as a problem of searching for a special cycle from an undirected graph. We propose a novel graph-based optimization algorithm “Bi-Directional Shortest Path (BDSP)” for searching the special graph cycle. In addition, we adapt our new method to different applications including building outline extraction and salient closed boundary tracking. Experimental results show that our methods outperform the state-of-the-art (SOTA) methods of different applications in terms of both robustness and accuracy. 3) Supervised Salient Object Detection by Deep Convolutional Neural Networks. Deep Convolutional Neural Networks (DCNN) have been adopted for salient object detection and achieved state-of-the-art performance. Most of the previous works, however, focus on region accuracy but not on the boundary quality. We propose a predict-refine architecture, BASNet (348.5 MB, 25 frames per second (FPS)), and a new hybrid loss for Boundary-Aware Salient object detection. Experimental results show that our method outperforms the SOTA methods both in terms of regional and boundary evaluation measures. To achieve lighter models with faster speed, we further design a simple yet powerful deep network architecture, U^2-Net, with a two-level nested U-structure for salient object detection. A full size U^2-Net (176.3 MB, 30 FPS) and a small size U^2-Nety (4.7 MB, 40 FPS) are instantiated based on the nested architecture. Both of them achieve very competitive results against the SOTA models.

• Subjects / Keywords

  • computer vision
  • salient object detection
  • image segmentation

• Graduation date

  Spring 2020

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/r3-txzq-1591

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.