Low-Level control of small scale helicopter using Soft Actor-Critic method

  • Author / Creator
    Kamyab, Majid
  • Unmanned Aerial Vehicles (UAVs), or drones, have been employed in a variety of
    applications, ranging from surveillance to emergency operations. These systems comprise an ”inner loop” that provides stability and control and an ”outer loop” in charge
    of mission-level tasks, such as way-point navigation. Despite their inherent instability, different techniques for controlling these robots have been devised under stable
    environmental conditions. However, these algorithms must know a robot’s dynamics
    to be effective; furthermore, more complex control is necessary for UAVs to perform
    in unstable environmental conditions. In this research, a simulated drone has been
    successfully controlled using model-free reinforcement learning with no prior knowledge of the robot’s model. Soft Actor-Critic (SAC) method is trained to perform
    low-level control of a small-scaled helicopter in a set-point control system. First,
    a simulation environment is created in which all tests were carried out and then it
    is shown that SAC can not only develop a strong policy, but it can also deal with
    unknown circumstances. The result obtained by the SAC agent is also compared to
    a sliding mode controller to compare the capability of this method to a traditional
    nonlinear control method. The SMC method proved to be superior by a steady state
    error of 0, compared to a steady state error of 0.05% for the SAC agent. However,
    the SAC agent is a model free technique which does not have access to the model of
    the helicopter, on the other hand the SMC is a model based technique whuch needs
    the system identification of the helicopter system.

  • Subjects / Keywords
  • Graduation date
    Fall 2021
  • Type of Item
  • Degree
    Master of Science
  • DOI
  • License
    This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.