Accelerating Empowerment Computation with UCT Tree Search

Christoph Salge, Christian Guckelsberger, Rodrigo Canaan, Tobias Mahlmann

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)


Models of intrinsic motivation present an important means to produce sensible behaviour in the absence of extrinsic rewards. Applications in video games are varied, and range from intrinsically motivated general game-playing agents to non-player characters such as companions and enemies. The information-theoretic quantity of Empowerment is a particularly promising candidate motivation to produce believable, generic and robust behaviour. However, while it can be used in the absence of external reward functions that would need to be crafted and learned, empowerment is computationally expensive. In this paper, we propose a modified UCT tree search method to mitigate empowerment's computational complexity in discrete and deterministic scenarios. We demonstrate how to modify a Monte-Carlo Search Tree with UCT to realise empowerment maximisation, and discuss three additional modifications that facilitate better sampling. We evaluate the approach both quantitatively, by analysing how close our approach gets to the baseline of exhaustive empowerment computation with varying amounts of computational resources, and qualitatively, by analysing the resulting behaviour in a Minecraft-like scenario.
Original languageEnglish
Title of host publicationProceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
ISBN (Electronic)9781538643594
Publication statusPublished - 15 Oct 2018
Event14th IEEE Conference on Computational Intelligence and Games, CIG 2018 - Maastricht, Netherlands
Duration: 14 Aug 201817 Aug 2018


Conference14th IEEE Conference on Computational Intelligence and Games, CIG 2018


  • Empowerment
  • Intrinsic motivation
  • MCTS
  • Minecraft
  • Tree search
  • UCT


Dive into the research topics of 'Accelerating Empowerment Computation with UCT Tree Search'. Together they form a unique fingerprint.

Cite this