Leveraging Empowerment to Model Tool Use in Reinforcement Learning

Research output: Chapter in Book/Report/Conference proceedingConference contribution

16 Downloads (Pure)

Abstract

Intrinsic motivation plays a key role in learning how to use tools, a fundamental aspect of human cultural evolution and child development that remains largely unexplored within the context of Reinforcement Learning (RL). This paper introduces “object empowerment” as a novel concept within this realm, showing its role as information-theoretic intrinsic motivation that underpins tool discovery and usage. Using empowerment, we propose a new general framework to model the utilization of tools within RL. We explore how maximizing empowerment can expedite the RL of tasks involving tools, highlighting its capacity to solve the challenge posed by sparse reward signals. By employing object empowerment as an intrinsically motivated regulariser, we guide the RL agent in simple grid-worlds towards states beneficial for learning how to master tools for efficient task completion. We will show how object empowerment can be used to measure and compare the effectiveness of different tools in handling an object. Our findings indicate efficient strategies to learn tool use and insights into the integration and modeling of tool control in the context of RL.
Original languageEnglish
Title of host publication2023 IEEE International Conference on Development and Learning (ICDL)
Place of PublicationMacau, China
Pages28-36
Number of pages9
ISBN (Electronic)978-1-6654-7075-9
DOIs
Publication statusPublished - 25 Dec 2023
Event2023 IEEE International Conference on Development and Learning (ICDL) - University of Macau, Macau, China
Duration: 9 Nov 202311 Nov 2023
https://www.icdl-2023.org/

Conference

Conference2023 IEEE International Conference on Development and Learning (ICDL)
Abbreviated titleIEEE ICDL 2023
Country/TerritoryChina
CityMacau
Period9/11/2311/11/23
Internet address

Fingerprint

Dive into the research topics of 'Leveraging Empowerment to Model Tool Use in Reinforcement Learning'. Together they form a unique fingerprint.

Cite this