Object Empowerment-Driven Tool Selection for Exploration in Reinforcement Learning

Tool use enables humans to solve complex physical tasks beyond their immediate capabilities. However, discovering tool use remains a major challenge for reinforcement learning (RL) agents, as it requires mastering long-horizon behaviors with sparse, delayed feedback—resulting in poor exploration and sample efficiency. While classic intrinsic motivation (IM) improves exploration, its lack of bias toward object-tool interactions leads agents to explore irrelevant states, resulting in many costly real-world interactions. In this paper, we investigate how RL agents can efficiently learn to use tools by optimizing object-centric intrinsic motivations — specifically, object empowerment, which quantifies the agent’s potential influence over specific objects in the environment. We extend this intrinsic motivation to multi-tool, multi-object environments that better reflect real-world lifelong learning challenges. Our method enables agents to identify meaningful tool-object relationships, learn when and how to use tools, and understand their lasting effects. Experiments in grid-based Minihack environments demonstrate that agents guided by object empowerment explore more effectively, generalize to new object configurations, and outperform PPO under sparse reward conditions.