Machine Learning Strategies for Steam Injection Optimization in SAGD

• Author / Creator

  Guevara Urdaneta, Jose L

• Finding optimal steam injection policies in the context of Steam Assisted Gravity Drainage (SAGD) represents a major challenge due to the complex dynamics of the process. This complexity is reflected by: i) several concurrent sub-processes, e.g., heat transfer, counter-current flow, imbibition, ii) potential reservoir heterogeneity, and iii) the lagged nature of the process. As a result, conventional steam injection strategies or policies in SAGD are typically not a result of a formal optimization process but rather empirically found. Furthermore, available optimization methods exhibit important drawbacks such as, requiring the full mathematical description of the process (adjoint-optimization) or may not be suitable for long-term optimization (Model Predictive Control).

  In this work, we propose two (2) alternatives to solve the cited challenge. The first alternative is the use of reinforcement learning (RL), in which no information of the physical SAGD phenomena is needed and can potentially optimize for long-term cumulative performance. In particular, we present the implementation of the two (2) main RL approaches: action-value function and policy gradient, for one and multiwell applications, respectively. In both implementations, obtained optimal steam injection policies exhibit a significant improvement both with respect to the initial (random) policies and constant steam injection strategies. Furthermore, these optimal policies exhibit two distinctive regions: initially, an increase or slight increase of steam injection rates (Region 1), and afterwards, a sharp decrease until reaching the minimum value (Region 2). This shape suggests that for optimal SAGD operations: i) steam chamber expansion is key until the overburden is reached (Region 1), afterwards, reservoir temperature should be kept high and ii) pressure plays a vital role until the steam chamber reaches the overburden, afterwards temperature is the driving mechanism of oil production.

  Reinforcement learning although represents a promising solution to the SAGD optimization challenge, it may require the continuous execution of a potentially computationally expensive numerical reservoir simulation model. As a result, the second alternative presented in this work is the use of dynamic surrogate modeling and optimization (DSMO) framework. In particular, we propose a methodology to build surrogate models that can provide fast approximations of time-varying outputs (e.g., daily oil production rates) of the SAGD process which could be used to solve the cited optimization problem.

  The proposed method represents an improvement of the conventional recursive based prediction approach in which a one-step prediction model(s) is identified and then used recursively to predict n-steps in the future. We propose the use of a second model that can capture the variability of the residual exhibited by the first model, and then act as a correction term. The underlying assumption, which is empirically discussed, is that the residuals of the recursive approach are correlated with time and thus can be generalized over the input space. The methodology consists of an extension of the traditional surrogate model and optimization (SMO) framework that considers non time-varying variables and consists of identifying surrogate models of any physical process using samples from high fidelity models.

  We test the proposed approach on a multi well reservoir simulation model using recurrent neural networks for both the one-step prediction model and the residual or correction model. Results show that the proposed approach offers significantly better prediction capabilities as compared to the conventional recursive approach. In particular, the corrected model is able to capture output variability (R2) and reduce error (Mean Absolute Percentage Error) consistently over several statistical realizations of the selected samples. Furthermore, we can show these results are similar when considering different limited and extended samples sizes, suggesting the efficiency of the method.

• Subjects / Keywords

  • reinforcement learning
  • surrogate modeling
  • optimization
  • SAGD

• Graduation date

  Spring 2023

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/r3-jx22-9544

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.