Inference-Based Deterministic Messaging for Multi-Agent Communication

• Author / Creator

  Bhatt, Varun S.

• Communication is essential for coordination among humans and animals. Therefore, with the introduction of intelligent agents into the world, agent-to-agent and agent-to-human communication become necessary. Ideally, these agents should be trained in an incremental and decentralized manner. In this thesis, we first study learning in matrix-based signaling games to empirically show that, with certain payoff matrices, decentralized reinforcement learning methods can converge to a suboptimal policy. We then propose a modification to the messaging policy, in which the sender deterministically chooses the best message that helps the receiver to infer the sender’s observation. Using this modification, we see, empirically, that the agents converge to the optimal policy in nearly all the runs. We then extend this method to function approximation settings, first applying it to larger matrix-based signaling games and then to a partially observable gridworld environment that requires cooperation between two agents. We show that, with appropriate approximation methods, the proposed sender modification can enhance existing decentralized training methods for more complex domains as well.

• Subjects / Keywords

  • Reinforcement Learning
  • Game Theory
  • Multi-Agent Systems

• Graduation date

  Fall 2020

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-2z4k-8m33

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.