Iranian Electric Industry Journal of Quality and Productivity

fa مدیریت بهینه شارژ خودروهای الکتریکی در شبکه توزیع با استفاده از یادگیری تقویتی عمیق Optimal Charging Management of Electric Vehicles in Distribution Networks Using Deep Reinforcement Learning برق و کامپیوتر پژوهشي Research افزایش نفوذ خودروهای الکتریکی در سال‌های اخیر، چالش‌های فنی قابل‌توجهی را در بهره‌برداری بهینه از شبکه‌های توزیع ایجاد کرده است؛ به‌گونه‌ای که شارژ کنترل‌نشده این وسایل نقلیه می‌تواند موجب افزایش پیک بار، تشدید تلفات توان و انحراف ولتاژ در باس‌ها شود. در این مقاله، یک چارچوب هوشمند مدیریت شارژ و دشارژ خودروهای الکتریکی مبتنی بر یادگیری تقویتی عمیق ارائه می‌شود. در رویکرد پیشنهادی، عامل یادگیری با درنظرگرفتن وضعیت لحظه‌ای شبکه و قیود دینامیکی باتری‌ها، سیاست کنترلی بهینه‌ای را برای زمان‌بندی توان تبادلی خودروها استخراج می‌کند. تابع پاداش به‌گونه‌ای طراحی شده است که کاهش تلفات شبکه و بهبود پروفیل ولتاژ را به‌صورت هم‌زمان مدنظر قرار دهد. ارزیابی عملکرد روش پیشنهادی بر روی شبکه استاندارد توزیع IEEE 33-bus  و از طریق تحلیل پخش بار در هر گام زمانی انجام شده است. نتایج شبیه‌سازی بیانگر آن است که چارچوب ارائه‌شده می‌تواند ضمن مدیریت مؤثر بار ناشی از خودروهای الکتریکی، شاخص‌های بهره‌برداری شبکه را به‌طور معناداری بهبود بخشد.   The rapid growth of electric vehicles (EVs) has created new opportunities for reducing greenhouse gas emissions and improving energy sustainability. However, the large‑scale integration of EVs into power systems can introduce operational challenges for distribution networks if charging processes are not properly coordinated. Uncontrolled charging may lead to increased peak demand, higher network losses, voltage deviations, and potential overloading of distribution feeders. Therefore, developing intelligent charging management strategies for EVs has become an important research topic in modern power systems. In this paper, a deep reinforcement learning (DRL)–based framework is proposed for optimal charging management of electric vehicles in distribution networks. The main objective of the proposed method is to determine appropriate charging and discharging actions for EVs in order to improve the operational performance of the network while maintaining acceptable battery energy levels. In the proposed approach, the EV charging problem is formulated as a sequential decision‑making process in which a learning agent interacts with the network environment. The agent observes the system state, including the operating condition of the distribution network and the state of charge (SOC) of EV batteries, and then selects suitable actions such ::::as char::::ging, discharging, or remaining idle. A reward function is designed to guide the learning process by considering important network performance indices, particularly power losses and voltage profile. The increasing penetration of EVs has motivated many researchers to investigate different approaches for EV charging scheduling. Conventional methods based on mathematical optimization or metaheuristic algorithms have been widely used in previous studies. Although these methods can provide effective solutions under certain assumptions, they often rely on deterministic information regarding load demand, EV arrival times, and charging requirements. In real-world environments, however, these parameters are uncertain and time‑varying. Consequently, traditional optimization approaches may face limitations when dealing with dynamic and uncertain operating conditions. Reinforcement learning offers a promising alternative because it allows the control strategy to be learned directly through interaction with the environment without requiring precise mathematical modeling of uncertainties. In particular, deep reinforcement learning combines reinforcement learning with deep neural networks and enables the handling of complex and high‑dimensional decision‑making problems in modern power systems. To evaluate the effectiveness of the proposed approach, the IEEE 33‑bus distribution network is used as the test system. Power flow analysis is performed at each simulation step to accurately capture the impact of EV charging and discharging decisions on network operation. The learning agent continuously updates its control policy based on the received reward and gradually learns a charging strategy that improves network performance while maintaining a reasonable SOC level for EV users. Simulation results demonstrate the effectiveness of the proposed DRL‑based charging management strategy. The numerical results indicate that the proposed method significantly improves several key operational indices of the distribution network. Specifically, the total network energy losses decrease from 4.150 MWh to 3.420 MWh, corresponding to a reduction of approximately 17.59%. In addition, the average minimum voltage of the network increases from 0.9410 p.u. to 0.9635 p.u., representing an improvement of about 2.39% in the voltage profile. The proposed strategy also effectively reduces the network peak load from 4.850 MW to 4.120 MW, which indicates a peak reduction of approximately 15.05%. Furthermore, the results show that the proposed method maintains an appropriate balance between improving grid performance and preserving EV battery energy. The average state of charge of vehicles changes from 0.820 to 0.765, indicating that the algorithm utilizes EV flexibility to support network operation while still maintaining a sufficient level of battery energy for vehicle usage. This demonstrates the capability of the reinforcement learning agent to learn an adaptive and balanced charging policy. Overall, the obtained results confirm that deep reinforcement learning provides an effective and flexible approach for EV charging management in distribution networks. Unlike conventional optimization techniques that depend on fixed models and predefined scheduling rules, the proposed method can learn an adaptive control policy directly from interaction with the system environment. Therefore, the proposed framework can serve as a promising solution for enhancing the operational efficiency and reliability of distribution networks in the presence of high penetration levels of electric vehicles.   مدیریت شارژ خودروهای الکتریکی, یادگیری تقویتی عمیق, شبکه توزیع هوشمند, بهینه‌سازی بهره‌برداری, پروفیل ولتاژ Electric Vehicle Charging Management, Deep Reinforcement Learning, Smart Distribution Network, Operational Optimization, Voltage Profile 23 35 http://ieijqp.ir/browse.php?a_code=A-10-1651-1&slc_lang=fa&sid=1 Hamed MirFathi حامد میرفتحی کمارعلیا hamed.mirfathi@aut.ac.ir 1362722431 10031947532846007713 No Department of Electrical Engineering, AmirKabir University of Technology, Tehran, Iran گروه مهندسی برق، دانشگاه صنعتی امیرکبیر، تهران، ایران Hossein Askarian Abyane حسین عسکریان ابیانه askarian@aut.ac.ir 10031947532846007714 10031947532846007714 Yes Department of Electrical Engineering, AmirKabir University of Technology, Tehran, Iran گروه مهندسی برق، دانشگاه صنعتی امیرکبیر، تهران، ایران