Vision-Based Methods for Joint State Estimation of Robotic Manipulators

  • Author / Creator
    Han, Mingjie
  • This thesis applied a combination of machine learning and computer vision
    to an engineering research project, using a two-armed Baxter robot hardware
    platform. The challenge was estimating the robot arm’s joint angles from
    monocular camera images. After evaluating several methods from traditional
    computer vision, we settled on the method of convolutional neural networks,
    which provided better accuracy and outlier rejection performance. A simulation
    environment toolchain was developed to generate automatically labelled
    training images for the neural network in order to eliminate the tedious manual
    labelling usually required for these methods. This brought the challenge of the
    domain gap between simulation and real-world images, which was solved using
    a generative adversarial network for transferring image textures. A hardware
    evaluation was performed for both joint keypoint detection and joint angle
    estimation performance, whose ground-truth values were accurately captured
    in the laboratory environment.

  • Subjects / Keywords
  • Graduation date
    Fall 2021
  • Type of Item
  • Degree
    Master of Science
  • DOI
  • License
    This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.