Download the full-sized PDF of Teaching a Powered Prosthetic Arm with an Intact Arm Using Reinforcement LearningDownload the full-sized PDF



Permanent link (DOI):


Export to: EndNote  |  Zotero  |  Mendeley


This file is in the following communities:

Graduate Studies and Research, Faculty of


This file is in the following collections:

Theses and Dissertations

Teaching a Powered Prosthetic Arm with an Intact Arm Using Reinforcement Learning Open Access


Other title
myoelectric prosthesis
context aware control
prosthetic arm
learn synergies
imitation learning
teaching with intact arm
learning from demonstration
robot arm
actor critic
reinforcement learning
Type of item
Degree grantor
University of Alberta
Author or creator
Vasan, Gautham
Supervisor and department
Pilarski, Patrick M (Computing Science, Div. of Physical Medicine and Rehabilitation, Dept. of Medicine)
Examining committee member and department
White, Martha (Computing Science)
Chan, Ming (Division of Physical Medicine & Rehabilitation, Dept. of Medicine)
Department of Computing Science

Date accepted
Graduation date
2017-11:Fall 2017
Master of Science
Degree level
The idea of an amputee playing the piano with all the flair and grace of an able-handed person may seem like a futuristic fantasy. While many prosthetic limbs look lifelike, finding one that also moves naturally has proved more of a challenge for both researchers and amputees. Even though sophisticated upper extremity prostheses like the Modular Prosthetic Limb (MPL) are capable of effecting almost all of the movements as a human arm and hand, they can be useful only if robust systems of control are available. The fundamental issue is that there is a significant mismatch between the number of controllable functions available in modern prosthetic arms and the number of control signals that can be provided by an amputee at any given moment. In order to bridge the gap in control, we require a neural interface that can translate the physiological signals of the user into a large number of joint commands for simultaneous, coordinated control of the artificial limb. In this thesis, we focus on a collaborative approach towards the control of powered prostheses. In our approach, the user shifts greater autonomy to the prosthetic device, thereby sharing the burden of control between the human and the machine. In essence, the prosthesis or rehabilitative device learns to ``fill in the gaps'' for the user. With this view in our mind, we developed a method that could allow someone with an amputation to use their non-amputated arm to teach their prosthetic arm how to move in a natural and coordinated way by simply showing the prosthetic arm the right way to move in response to inputs from the user. Such a paradigm could well exploit the muscle synergies already learned by the user. Consider cases where an amputee has a desired movement goal, e.g., ``add sugar to my coffee," ``button up my shirt," or ``shake hands with an acquaintance". In these more complicated examples, it may be difficult for a user to frame their objectives in terms of device control parameters or existing device gestures, but they may be able to execute these motions skillfully with their remaining biological limb. As a first contribution of this thesis, we present results from our work on learning from demonstration using Actor-Critic Reinforcement Learning (ACRL), and show that able-bodied subjects (n = 3) are able to train a prosthetic arm to perform synergistic movements in three degrees of freedom(DOF) (wrist flexion, wrist rotation and hand open/close). The learning system uses only the joint position and velocity information from the prosthesis and above-elbow myoelectric signals from the user. We also assessed the performance of the system with an amputee participant and demonstrate that the learning-from-demonstration paradigm can be used to teach a prosthetic arm natural, coordinated movements with the intact arm. For our second contribution, we describe a sensor fusion and artificial vision based control approach that could potentially give rise to context-aware control of a multi-DOF prosthesis. Our results indicate that the learning system can make use of the addition sensory and motor information to determine and context and differentiate between different movement synergies. Our results suggest that this learning-from-demonstration paradigm may be well suited to use by both patients and clinicians with minimal technical knowledge, as it allows a user to personalize the control of his or her prosthesis without having to know the underlying mechanics of the prosthetic limb. This approach may extend in a straightforward way to next-generation prostheses with precise finger and wrist control, such that these devices may someday allow users to perform fluid and intuitive movements like playing the piano, catching a ball, and comfortably shaking hands.
This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for the purpose of private, scholarly or scientific research. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.
Citation for previous publication
G. Vasan and P. M. Pilarski, "Learning from demonstration: Teaching a myoelectric prosthesis with an intact limb via reinforcement learning," 2017 International Conference on Rehabilitation Robotics (ICORR), London, United Kingdom, 2017, pp. 1457-1464. doi: 10.1109/ICORR.2017.8009453 keywords: {Elbow;Manipulators;Muscles;Prosthetics;Training;Wrist}, URL:

File Details

Date Uploaded
Date Modified
Audit Status
Audits have not yet been run on this file.
File format: pdf (Portable Document Format)
Mime type: application/pdf
File size: 21449938
Last modified: 2017:11:08 16:46:43-07:00
Filename: Vasan_Gautham_201709_MSc.pdf
Original checksum: 3ac130762b12c096382cf990a1337cb0
Well formed: false
Valid: false
Status message: Invalid page tree node offset=21191942
Status message: Unexpected error in findFonts java.lang.ClassCastException: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject cannot be cast to edu.harvard.hul.ois.jhove.module.pdf.PdfDictionary offset=5358
Status message: Invalid name tree offset=21430204
Status message: Invalid name tree offset=21430204
Status message: Invalid name tree offset=21430204
Activity of users you follow
User Activity Date