- 39 views
- 42 downloads
Reinforcement Learning-Driven Local Transactive Energy Market for Distributed Energy Resources
-
- Author / Creator
- Zhang, Shida
-
Technological breakthroughs in renewable power generation, battery storage, electric mobility, and advanced data logistics are changing the electric grid. The huge influx of distributed energy resources (DERs), while important to curb carbon emissions, is not without consequences. The highly intermittent nature of renewable energy resources (RES), combined with the decreased visibility of DERs (from the system operator’s perspective), makes it increasingly difficult for the grid utility companies to balance generation with loads over time. If this trend continues, then phenomena such as voltage fluctuations, reverse power flow, and degraded power quality are expected to increase in frequency, bringing higher costs to the customers while decreasing quality of service they receive.
To overcome these challenges, research have been underway to find alternative demand management strategies that can better integrate DERs. Transactive Energy (TE) is a framework that promises to achieve flexible, robust, and adaptive energy management systems that properly integrate DERs. Local energy markets are emerging as a tool for coordinating generation, storage, and consumption of energy from distributed resources. In combination with automation, they promise to provide an effective energy management framework that is fair and brings system-level savings. This thesis presents work towards advancing the practical implementation of economy focused TE systems. Specifically, on multi-agent systems with learning energy trading agents that exploits the cooperative-competitive nature of auction markets to dynamically balance of supply and demand in a local community.
A TE simulator is first developed to complete the subsequent tasks. Next, the relationship between double auction market properties and reinforcement learning is examined. This is a critical step, as a poorly designed market may yield unintended behavior of market participants. Results show that the market must be truthful and weakly budget balanced in order for the agents to develop behaviours that reflect price theory, which is a necessary condition to generate strong and relevant reward signals.
Since the price generation process in the local energy market is fundamentally different from contemporary pricing schemes (such as time-of-use), a mathematical model that aggregates and converts individual agent policies to a global pricing scheme is created so that some properties of pricing schemes can be compared directly. In this study, it can be shown that the TE price model is far more responsive and relevant compared to time-of-use, which is suggests that agent behaviours can be more accurate and efficient when used for load shaping. Furthermore, significant bill reductions are achieved when compared to net billing. The community as a whole experienced a bill reduction of 35.9%, and the mean and median of individual bill reductions are 74.51% and 38.8%, respectively.
Finally, the dynamic power balancing aspect of the proposed TE system is studied with the integration of battery energy storage. A test circuit was created so that voltage violations exist, but cannot be eliminated or reduced via self sufficiency alone. By combining local energy trading with a battery storage, all of the voltage violations are completely eliminated. -
- Subjects / Keywords
-
- Graduation date
- Fall 2023
-
- Type of Item
- Thesis
-
- Degree
- Doctor of Philosophy
-
- License
- This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.