Using counterfactual regret minimization to create a competitive multiplayer poker agent

• Author / Creator

  Abou Risk, Nicholas

• Games have been used to evaluate and advance techniques in the field of Artificial Intelligence since
  before computers were invented. Many of these games have been deterministic perfect information
  games (e.g. Chess and Checkers). A deterministic game has no chance element and in a perfect
  information game, all information is visible to all players. However, many real-world scenarios
  involving competing agents can be more accurately modeled as stochastic (non-deterministic), im-
  perfect information games, and this dissertation investigates such games. Poker is one such game
  played by millions of people around the world; it will be used as the testbed of the research presented
  in this dissertation. For a specific set of games, two-player zero-sum perfect recall games, a recent
  technique called Counterfactual Regret Minimization (CFR) computes strategies that are provably
  convergent to an ϵ-Nash equilibrium. A Nash equilibrium strategy is very useful in two-player games
  as it maximizes its utility against a worst-case opponent. However, once we move to multiplayer
  games, we lose all theoretical guarantees for CFR. Furthermore, we have no theoretical guarantees
  about the performance of a strategy from a multiplayer Nash equilibrium against two arbitrary op-
  ponents. Despite the lack of theoretical guarantees, my thesis is that CFR-generated agents may
  perform well in multiplayer games. I created several 3-player limit Texas Hold’em Poker agents
  and the results of the 2009 Computer Poker Competition demonstrate that these are the strongest
  3-player computer Poker agents in the world. I also contend that a good strategy can be obtained by
  grafting a set of two-player subgame strategies to a 3-player base strategy when one of the players
  is eliminated.

• Subjects / Keywords

  • Poki
  • Poker
  • 3-player
  • Heads-up experts
  • Multiplayer
  • CFR
  • Counterfactual regret minimization

• Graduation date

  Fall 2009

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/R3R360

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.