Download the full-sized PDF of Adaptive and Autonomous Switching: Shared Control of Powered Prosthetic Arms Using Reinforcement LearningDownload the full-sized PDF



Permanent link (DOI):


Export to: EndNote  |  Zotero  |  Mendeley


This file is in the following communities:

Graduate Studies and Research, Faculty of


This file is in the following collections:

Theses and Dissertations

Adaptive and Autonomous Switching: Shared Control of Powered Prosthetic Arms Using Reinforcement Learning Open Access


Other title
reinforcement learning
shared control
myoelectric prosthetic arms
switched control
adaptive switching
real-time machine learning
autonomous switching
unlearning predictions
Type of item
Degree grantor
University of Alberta
Author or creator
Edwards, Ann L
Supervisor and department
Pilarski, Patrick (Medicine)
Hebert, Jacqueline (Medicine)
Examining committee member and department
Bowling, Michael (Computing Science)
Faculty of Rehabilitation Medicine
Rehabilitation Science
Date accepted
Graduation date
Master of Science
Degree level
Powered prosthetic arms with numerous controllable functions (i.e., grip patterns or movable joints) can be challenging to operate. Gated control---a common control method for myoelectric arms and other human-machine interfaces---allows users to select a function by switching through a static list of possible functions. However, switching between many controllable functions often entails significant time and cognitive effort on the part of the user when performing tasks. One way to decrease the number of switching interactions required of a user is to shift greater autonomy to the prosthetic device, thereby sharing the burden of control between the human and the machine. Previous work has demonstrated that reinforcement learning (RL), and specifically general value functions (GVFs), has the potential to reduce the time and switching cost of gated control methods. In the current work, we extend previous studies by advancing an RL method termed adaptive switching for use during real time control of a prosthetic arm. Adaptive switching uses contextual factors to build up predictions about the use of functions during a task. Based on these predictions, adaptive switching will continually optimize and change the order in which functions are presented to the user during switching. We also combine adaptive switching with another machine learning control method, termed autonomous switching, to further decrease the number of manual switching interactions required of a user. Autonomous switching uses predictions, learned in real time through the use of GVFs, to switch automatically between functions for the user. Over the course of several studies, we collected results from subjects with and without amputations, performing simple and more challenging tasks with a myoelectric robot arm. As a first contribution of this thesis, we present results from work on adaptive switching, and show that it leads to a reduction in the total switching cost (both in terms of time and the total number of switches), an effect that is especially pronounced for experienced myoelectric arm users. For our second and third contributions, we describe our autonomous switching approach and demonstrate that it is able to both learn and subsequently unlearn to switch autonomously during ongoing use, a key requirement for maintaining human-centered shared control. We found that although autonomous switching decreases the number of user-initiated switches compared to conventional control, some work remains to be done to improve accuracy of predictions leading to a switch. We also show that the addition of feedback to the user can significantly improve the performance of autonomous switching. This work promises to help improve the gated control method for prosthetic arms, as well as other domains involving human-machine interaction---in particular, assistive or rehabilitative devices that require switching between different modes of operation such as exoskeletons and powered orthotics.
This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for the purpose of private, scholarly or scientific research. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.
Citation for previous publication
P.M. Pilarski, A.L. Edwards, and K.M. Chan, "Novel Control Strategies for Arm Prostheses: A Partnership Between Man and Machine," The Japanese Journal of Rehabilitation Medicine, vol. 52, no. 2, pp. 91-95, 2015.A.L. Edwards, M.R. Dawson, J.S. Hebert, C. Sherstan, R.S. Sutton, K.M. Chan, and P.M. Pilarski, "Application of Real-time Machine Learning to Myoelectric Prosthesis Control: A Case Series in Adaptive Switching," Prosthetics and Orthotics International. Published online before print September 30, 2015, doi: 10.1177/0309364615605373.A.L. Edwards, M.R. Dawson, J.S. Hebert, R.S. Sutton, K.M. Chan, P.M. Pilarski, "Adaptive Switching in Practice: Improving Myoelectric Prosthesis Performance through Reinforcement Learning," Proc. of MEC'14: Myoelectric Controls Symposium, Fredericton, New Brunswick, August 18-22, 2014, pp. 69-73.A.L. Edwards, A. Kearney, M.R. Dawson, R.S. Sutton, and P.M. Pilarski, "Temporal-Difference Learning to Assist Human Decision Making during the Control of an Artificial Limb," 1st Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM), Oct. 25–-27, Princeton, New Jersey, USA, 2013.A.L. Edwards, J.S. Hebert, and P.M. Pilarski, "Machine Learning and Unlearning to Autonomously Switch Between the Functions of a Myoelectric Arm," 6th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Singapore, June 26-29, 2016. To appear.

File Details

Date Uploaded
Date Modified
Audit Status
Audits have not yet been run on this file.
File format: pdf (PDF/A)
Mime type: application/pdf
File size: 3978577
Last modified: 2016:11:16 13:50:26-07:00
Filename: Edwards_Ann_L_201605_MSc.pdf
Original checksum: 446a8b9db8f42222c726e206845088f6
Well formed: true
Valid: true
Page count: 100
Activity of users you follow
User Activity Date