Vision-Based Methods for Joint State Estimation of Robotic Manipulators

• Author / Creator

  Han, Mingjie

• This thesis applied a combination of machine learning and computer vision
  to an engineering research project, using a two-armed Baxter robot hardware
  platform. The challenge was estimating the robot arm’s joint angles from
  monocular camera images. After evaluating several methods from traditional
  computer vision, we settled on the method of convolutional neural networks,
  which provided better accuracy and outlier rejection performance. A simulation
  environment toolchain was developed to generate automatically labelled
  training images for the neural network in order to eliminate the tedious manual
  labelling usually required for these methods. This brought the challenge of the
  domain gap between simulation and real-world images, which was solved using
  a generative adversarial network for transferring image textures. A hardware
  evaluation was performed for both joint keypoint detection and joint angle
  estimation performance, whose ground-truth values were accurately captured
  in the laboratory environment.

• Subjects / Keywords

  • computer vision
  • joint state estimation
  • domain gap
  • keypoint detection
  • construction equipment
  • sensor less equipment
  • Convolutional Neural Network

• Graduation date

  Fall 2021

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-rhr5-eq76

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.