Optimal Vehicle-to-Grid Control for Supplementary Frequency Regulation Using Deep Reinforcement Learning

The expanding Electric Vehicle (EV) market presents a new opportunity for electric vehicles to deliver a wide range of valuable grid services. Indeed, the emerging Vehicle-to-Grid (V2G) technology with bi-directional flow of power provides the grid with access to mobile energy storage for demand response, frequency regulation and balancing of the local distribution system. This reduces electricity costs at peak hours and can be profitable for customers, network operators and energy retailers. In this paper, an optimal V2G control strategy using Deep Reinforcement Learning (DRL) is proposed to simultaneously maximise the benefits of EV owners and aggregators while fulfilling the driving needs of EV owners. In the proposed DRL-based V2G control strategy, a Deep Deterministic Policy Gradient (DDPG) agent is used to dynamically adjust the V2G power scheduling to satisfy the driving demand of EV users and simultaneously perform frequency regulation tasks. The proposed V2G control scheme is tested on a two-area power system undergoing frequency deviations. The results showed that the proposed V2G control leads to a better frequency deviation reduction and improved Area Control Error (ACE), while satisfying the charging demands of EVs as compared to other strategies.