Towards Dynamic Energy/Carbon Trading and Resource Allocation for Mobile Edge Computing: A Two-Timescale Deep Reinforcement Learning Approach

Integrating smart grid and carbon market into mobile edge computing (MEC) systems presents significant potential for reducing both operational energy costs and carbon footprints. This paper proposes a joint optimization framework for energy/carbon trading and resource allocation in MEC systems participating in grid-energy and carbon markets, aiming to minimize the long-term time-averaged cost of the energy/carbon tradings and the energy consumption of the system. Built on a two-timescale multi-agent deep reinforcement learning (TTMADRL) optimization framework, the Deep Deterministic Policy Gradient (DDPG) is generalized to make decisions on energy and carbon transactions at the large timescale; while at the small timescale, the task offloading schedules and CPU frequencies are distributively determined at each device by using the Multi-Agent DDPG (MADDPG) algorithm with enhanced scalability. Simulations demonstrate that the proposed TTMADRL achieves a 75.44% reduction in system costs compared to baseline approaches.