University of Hertfordshire

By the same authors

Accelerating Empowerment Computation with UCT Tree Search

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Standard

Accelerating Empowerment Computation with UCT Tree Search. / Salge, Christoph; Guckelsberger, Christian; Canaan, Rodrigo; Mahlmann, Tobias.

Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. Vol. 2018-August IEEE Computer Society, 2018. 8490447.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Harvard

Salge, C, Guckelsberger, C, Canaan, R & Mahlmann, T 2018, Accelerating Empowerment Computation with UCT Tree Search. in Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. vol. 2018-August, 8490447, IEEE Computer Society, 14th IEEE Conference on Computational Intelligence and Games, CIG 2018, Maastricht, Netherlands, 14/08/18. https://doi.org/10.1109/CIG.2018.8490447

APA

Salge, C., Guckelsberger, C., Canaan, R., & Mahlmann, T. (2018). Accelerating Empowerment Computation with UCT Tree Search. In Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018 (Vol. 2018-August). [8490447] IEEE Computer Society. https://doi.org/10.1109/CIG.2018.8490447

Vancouver

Salge C, Guckelsberger C, Canaan R, Mahlmann T. Accelerating Empowerment Computation with UCT Tree Search. In Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. Vol. 2018-August. IEEE Computer Society. 2018. 8490447 https://doi.org/10.1109/CIG.2018.8490447

Author

Salge, Christoph ; Guckelsberger, Christian ; Canaan, Rodrigo ; Mahlmann, Tobias. / Accelerating Empowerment Computation with UCT Tree Search. Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. Vol. 2018-August IEEE Computer Society, 2018.

Bibtex

@inproceedings{10790c0866f041f087894da624fec60b,
title = "Accelerating Empowerment Computation with UCT Tree Search",
abstract = "Models of intrinsic motivation present an important means to produce sensible behaviour in the absence of extrinsic rewards. Applications in video games are varied, and range from intrinsically motivated general game-playing agents to non-player characters such as companions and enemies. The information-theoretic quantity of Empowerment is a particularly promising candidate motivation to produce believable, generic and robust behaviour. However, while it can be used in the absence of external reward functions that would need to be crafted and learned, empowerment is computationally expensive. In this paper, we propose a modified UCT tree search method to mitigate empowerment's computational complexity in discrete and deterministic scenarios. We demonstrate how to modify a Monte-Carlo Search Tree with UCT to realise empowerment maximisation, and discuss three additional modifications that facilitate better sampling. We evaluate the approach both quantitatively, by analysing how close our approach gets to the baseline of exhaustive empowerment computation with varying amounts of computational resources, and qualitatively, by analysing the resulting behaviour in a Minecraft-like scenario.",
keywords = "Empowerment, Intrinsic motivation, MCTS, Minecraft, Tree search, UCT",
author = "Christoph Salge and Christian Guckelsberger and Rodrigo Canaan and Tobias Mahlmann",
year = "2018",
month = "10",
day = "11",
doi = "10.1109/CIG.2018.8490447",
language = "English",
volume = "2018-August",
booktitle = "Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018",
publisher = "IEEE Computer Society",
address = "United States",

}

RIS

TY - GEN

T1 - Accelerating Empowerment Computation with UCT Tree Search

AU - Salge, Christoph

AU - Guckelsberger, Christian

AU - Canaan, Rodrigo

AU - Mahlmann, Tobias

PY - 2018/10/11

Y1 - 2018/10/11

N2 - Models of intrinsic motivation present an important means to produce sensible behaviour in the absence of extrinsic rewards. Applications in video games are varied, and range from intrinsically motivated general game-playing agents to non-player characters such as companions and enemies. The information-theoretic quantity of Empowerment is a particularly promising candidate motivation to produce believable, generic and robust behaviour. However, while it can be used in the absence of external reward functions that would need to be crafted and learned, empowerment is computationally expensive. In this paper, we propose a modified UCT tree search method to mitigate empowerment's computational complexity in discrete and deterministic scenarios. We demonstrate how to modify a Monte-Carlo Search Tree with UCT to realise empowerment maximisation, and discuss three additional modifications that facilitate better sampling. We evaluate the approach both quantitatively, by analysing how close our approach gets to the baseline of exhaustive empowerment computation with varying amounts of computational resources, and qualitatively, by analysing the resulting behaviour in a Minecraft-like scenario.

AB - Models of intrinsic motivation present an important means to produce sensible behaviour in the absence of extrinsic rewards. Applications in video games are varied, and range from intrinsically motivated general game-playing agents to non-player characters such as companions and enemies. The information-theoretic quantity of Empowerment is a particularly promising candidate motivation to produce believable, generic and robust behaviour. However, while it can be used in the absence of external reward functions that would need to be crafted and learned, empowerment is computationally expensive. In this paper, we propose a modified UCT tree search method to mitigate empowerment's computational complexity in discrete and deterministic scenarios. We demonstrate how to modify a Monte-Carlo Search Tree with UCT to realise empowerment maximisation, and discuss three additional modifications that facilitate better sampling. We evaluate the approach both quantitatively, by analysing how close our approach gets to the baseline of exhaustive empowerment computation with varying amounts of computational resources, and qualitatively, by analysing the resulting behaviour in a Minecraft-like scenario.

KW - Empowerment

KW - Intrinsic motivation

KW - MCTS

KW - Minecraft

KW - Tree search

KW - UCT

UR - http://www.scopus.com/inward/record.url?scp=85053164908&partnerID=8YFLogxK

U2 - 10.1109/CIG.2018.8490447

DO - 10.1109/CIG.2018.8490447

M3 - Conference contribution

VL - 2018-August

BT - Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018

PB - IEEE Computer Society

ER -