Focus of Attention in Reinforcement Learning

• Author(s) / Creator(s)

  • Li, Lihong

• Technical report TR07-12. One key topic in reinforcement learning is function approximation which is critical for the success of reinforcement learning in domains with large state spaces. Unfortunately, function approximation can lead to several problems including the suboptimality of the produced policies and even divergence of learning. Thus, reinforcement learning with function approximation has remained an area of active research. We demonstrate that in reinforcement learning, it is helpful for the agent to focus on more important states thereby producing better policies using less computing resources. The problem of focused learning is investigated formally, and two classes of reinforcement learning methods are considered: the classification-based approach and the value-function-based approach. For each of these two approaches, we will (i) define a formal metric of state importance, and (ii) utilize it in reinforcement learning with function approximation. The advantages of focusing attention on important states are supported both theoretically and empirically. | TRID-ID TR07-12

• Date created

  2007

• Subjects / Keywords

  • Reinforcement learning
  • State importance
  • Reinforcement Learning
  • Focused learning

• Type of Item

  Report

• DOI

  https://doi.org/10.7939/R31G0HX9N

• License

  Attribution 3.0 International