Extraction of a Projected Pattern in a Single Image using Deep Learning

• Author / Creator

  Ata-E-Rabbi, H M

• The appearance of an image projected by a projector onto an arbitrary surface has the potential to appear differently depending on several factors, e.g., the properties of the surface being projected on, the colors of the light projected by the projector and the colors and the optical properties of the surface. Recovering the original colors of the projected image and hence, the scene structure using a Structured Light (SL) system are challenged by the aforementioned factors. One of the key aspects of doing 3D reconstruction using SL systems is to detect the colors of the elements of the projected pattern in the image captured by the camera which contains important information needed for decoding the pattern to reconstruct the 3D shape of the object. However, this can be a challenging task when the colors of the projected lights interact with the colors of objects in the scene making the detection of the original colors dicult or when the surface absorbs most of the projected light making even the detection of the shapes of the elements of projected pattern difficult, if not impossible. Many of-the-shelf SL devices like the Microsoft Kinect use infrared (IR) cameras and projectors to get around the issue of color blending. But IR devices are not as readily available as normal RGB cameras and projectors which are usually the devices used in cost-efficient do- it-yourself custom SL systems. In this thesis, we explore the problem of pattern extraction for SL systems. We first focus on extracting the dots of the Kinect pattern, without any regard for the colors of the dots as color information is not used for decoding this pattern, by posing the problem of pattern extraction as a pixel classification problem and propose a simple encoder-decoder based model that can surpass the classification accuracy of a straightforward image processing approach. Because obtaining ground truth to train such models is expensive and also because many RGB SL systems developed so far use many different kinds of patterns, we next propose a method, inspired by the ideas from recent advances in scene decomposition, to extract the elements of any pattern with original colors in an unsupervised manner at the cost of longer processing time. Our experiments show that this approach works best with dense patterns, but has trouble extracting fine details accurately required for sparse patterns like that of the Kinect's.

• Subjects / Keywords

  • strucured light
  • pattern
  • extraction
  • deep learning

• Graduation date

  Fall 2021

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-z85f-pr64

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.