Teaching a Powered Prosthetic Arm with an Intact Arm Using Reinforcement Learning

• Author / Creator

  Vasan, Gautham

• The idea of an amputee playing the piano with all the flair and grace of an able-handed person may seem like a futuristic fantasy. While many prosthetic limbs look lifelike, finding one that also moves naturally has proved more of a challenge for both researchers and amputees. Even though sophisticated upper extremity prostheses like the Modular Prosthetic Limb (MPL) are capable of effecting almost all of the movements as a human arm and hand, they can be useful only if robust systems of control are available. The fundamental issue is that there is a significant mismatch between the number of controllable functions available in modern prosthetic arms and the number of control signals that can be provided by an amputee at any given moment. In order to bridge the gap in control, we require a neural interface that can translate the physiological signals of the user into a large number of joint commands for simultaneous, coordinated control of the artificial limb. In this thesis, we focus on a collaborative approach towards the control of powered prostheses. In our approach, the user shifts greater autonomy to the prosthetic device, thereby sharing the burden of control between the human and the machine. In essence, the prosthesis or rehabilitative device learns to fill in the gaps'' for the user. With this view in our mind, we developed a method that could allow someone with an amputation to use their non-amputated arm to teach their prosthetic arm how to move in a natural and coordinated way by simply showing the prosthetic arm the right way to move in response to inputs from the user. Such a paradigm could well exploit the muscle synergies already learned by the user. Consider cases where an amputee has a desired movement goal, e.g.,add sugar to my coffee," button up my shirt," orshake hands with an acquaintance". In these more complicated examples, it may be difficult for a user to frame their objectives in terms of device control parameters or existing device gestures, but they may be able to execute these motions skillfully with their remaining biological limb. As a first contribution of this thesis, we present results from our work on learning from demonstration using Actor-Critic Reinforcement Learning (ACRL), and show that able-bodied subjects (n = 3) are able to train a prosthetic arm to perform synergistic movements in three degrees of freedom(DOF) (wrist flexion, wrist rotation and hand open/close). The learning system uses only the joint position and velocity information from the prosthesis and above-elbow myoelectric signals from the user. We also assessed the performance of the system with an amputee participant and demonstrate that the learning-from-demonstration paradigm can be used to teach a prosthetic arm natural, coordinated movements with the intact arm. For our second contribution, we describe a sensor fusion and artificial vision based control approach that could potentially give rise to context-aware control of a multi-DOF prosthesis. Our results indicate that the learning system can make use of the addition sensory and motor information to determine and context and differentiate between different movement synergies. Our results suggest that this learning-from-demonstration paradigm may be well suited to use by both patients and clinicians with minimal technical knowledge, as it allows a user to personalize the control of his or her prosthesis without having to know the underlying mechanics of the prosthetic limb. This approach may extend in a straightforward way to next-generation prostheses with precise finger and wrist control, such that these devices may someday allow users to perform fluid and intuitive movements like playing the piano, catching a ball, and comfortably shaking hands.

• Subjects / Keywords

  • Myoelectric prosthesis
  • Actor critic
  • Learning from demonstration
  • Imitation learning
  • Learn synergies
  • Context aware control
  • Prosthetic arm
  • Robot arm
  • Teaching with intact arm
  • Reinforcement learning

• Graduation date

  Fall 2017

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/R3GF0N94H

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.