- 88 views
- 112 downloads
An Investigation into Reinforcement Learning in FPGA Placement Optimization
-
- Author / Creator
- Chen, Ruichen
-
With the increasing complexity and capacity of modern Field-Programmable Gate Arrays (FPGAs), there is a growing demand for efficient FPGA computer-aided design (CAD) tools, particularly at the placement stage. While some previous works, such as RLPlace, have explored the efficacy of single-state Reinforcement Learning (RL) to optimize FPGA placement by framing it as a multi-armed bandit (MAB) problem, numerous AI techniques remain unexplored due to the outstanding engineering challenges of integrating them into the FPGA CAD flow, which is implemented typically using C++. In this thesis, we propose VPR-Gym, a Python environment built on OpenAI Gym that allows seamless integration with various machine learning libraries, including PyTorch, TensorFlow, and Nevergrad, while enabling the comparison between different AI techniques for FPGA placement optimization. To determine the optimal RL algorithm for FPGA placement, we perform regret analysis and non-stationary analysis of MAB algorithms used in FPGA placement optimization. Moreover, we introduce a learning objective that reformulates the MAB problem as an optimization problem, thereby expanding the range of AI techniques that can be investigated beyond those for MAB problems. To investigate the effectiveness of different algorithms in FPGA placement and thus showcase the capabilities of our VPR-Gym platform, we conduct experiments that compare the performance of various MAB algorithms and evolution strategy (ES) algorithms. Our findings demonstrate that the ES approaches exhibit superior performance over the existing MAB approaches, highlighting the effectiveness of VPR-Gym in facilitating AI research to enhance FPGA placement.
-
- Subjects / Keywords
-
- Graduation date
- Fall 2023
-
- Type of Item
- Thesis
-
- Degree
- Master of Science
-
- License
- This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.