Improving Water Treatment Using Reinforcement Learning

• Author / Creator

  Liu, Puer

• We have witnessed the rising popularity of real-world applications of reinforcement learning (RL). However, most successful real-world applications of RL rely on high-fidelity simulators that enable rapid iteration of prototypes, hyperparameter selection and policy training. On the other hand, RL is not wildly used in industrial control problems where simulators are too expensive or complicated to build. In addition to the lack of simulators, industrial control problems often face other challenges including high dimensionality of the observation space, missing or noisy sensor values, and control happening on multiple time scales. These challenges combined make industrial control a unique area of research that has not been fully explored yet.
  Water treatment plants (WTPs) are one of the most important infrastructures in today's world. They not only deliver clean and accessible water to households but are also responsible for wastewater treatment before returning to the water cycle. Nowadays, accurate sensors and remove control functionalities make it possible to have fully automated WTPs with minimal human intervention. Such ``smart" WTPs will be valuable for providing life-giving water to areas where full-time WTP operators are inaccessible. Since the water treatment process shares many characteristics with other industrial control tasks, it is an excellent platform for developing novel algorithms that handle challenges common in industrial control tasks.
  The objective of this thesis is to formulate the water treatment process as a collection of RL tasks. To achieve this, we develop the software stack for real-time monitoring, data collection, and control on a small-scale plant that shares similarities to the main plant. We conduct a detailed survey on the characteristic of installed sensors and their behaviour in different operating modes. We determine sub-tasks that are suitable for RL. We then identify the challenges we found while experimenting with the WTP. Based on these findings, we present a case study on a sub-task: chemical dosing rate control in water pretreatment, which utilizes the offline logs and does not use a simulator.

• Subjects / Keywords

  • Reinforcement Learning
  • Water Treatment

• Graduation date

  Fall 2022

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-7e11-kv09

• License

  This thesis is made available by the University of Alberta Library with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.