|
Electricity Consumption Forecasting Using a Hybrid Approach Based on Transformer Model and LSTM Neural Network
|
Mohammadreza Ahmadipour1   , Esmat Rashedi1 , Maryam Amoozegar * 1 |
| 1- KGUT |
|
|
Abstract: (245 Views) |
| In recent years, electricity consumption forecasting has gained significant importance by relying on new technologies and utilizing big data processing methods. Various methods including classical statistical time series analysis techniques, support vector machines, and recurrent neural networks with long short-term memory have been studied in this field. Statistical methods may not be suitable for forecasting and modeling some complex phenomena due to their inability to account for sudden changes. On the other hand, the support vector machine method operates based on increasing the data dimensions. Therefore, in cases where the data has high dimensions, this leads to increasing the complexity of the problem space. Various types of neural networks also face limitations such as vanishing gradients and the inability to account for temporal relationships. To achieve more accurate electricity consumption forecasting, this paper proposes a hybrid approach using the Transformer model and long short-term memory neural network. By addressing the gradient problem and learning complex patterns, this approach offers higher accuracy compared to other methods. Additionally, the Transformer model, with its attention mechanism, has the ability to focus on important components of the data, creating a more interpretable model with high resistance to noise. The proposed method has been evaluated on a standard dataset and compared with existing methods. The results show that this method achieves higher accuracy and lower error in metrics such as mean squared error and mean absolute percentage error. |
|
| Keywords: Predicting time series, Long Short-Term Memory Neural Networks, Transformer Model, Attention Mechanism. |
|
|
|
Type of Study: Research |
Received: 2023/12/31 | Accepted: 2024/10/6
|
|
|
|
|
|
|
| References |
1. A. Xu, M.-W. T., B. Firouzi, K. A. Alattas, A. Mohammadzadeh, and E. Ghaderpour, "A new deep learning Restricted Boltzmann Machine for energy consumption forecasting," Sustainability, vol. 14, no. 16, p. 10081, 2022. [ DOI:10.3390/su141610081] 2. Al Mamun, A., Sohel, M., Mohammad, N., Sunny, M. S. H., Dipta, D. R., & Hossain, E. (2020). A comprehensive review of the load forecasting techniques using single and hybrid predictive models. IEEE Access, 8, 134911-134939. [ DOI:10.1109/ACCESS.2020.3010702] 3. Amalou, I., Mouhni, N., & Abdali, A. (2022). Multivariate time series prediction by RNN architectures for energy consumption forecasting. Energy Reports, 8, 1084-1091. [ DOI:10.1016/j.egyr.2022.07.139] 4. Cui, C., He, M., Di, F., Lu, Y., Dai, Y., & Lv, F. (2020). Research on power load forecasting method based on LSTM model. 2020 IEEE 5th information technology and mechatronics engineering conference (ITOEC), [ DOI:10.1109/ITOEC49072.2020.9141684] 5. Farsi, B., Amayri, M., Bouguila, N., & Eicker, U. (2021). On short-term load forecasting using machine learning techniques and a novel parallel deep LSTM-CNN approach. IEEE Access, 9, 31191-31212. [ DOI:10.1109/ACCESS.2021.3060290] 6. L'Heureux, A., Grolinger, K., & Capretz, M. A. (2022). Transformer-based model for electrical load forecasting. Energies, 15(14), 4993. [ DOI:10.3390/en15144993] 7. Lu, W., Li, J., Wang, J., & Qin, L. (2021). A CNN-BiLSTM-AM method for stock price prediction. Neural Computing and Applications, 33, 4741-4753. [ DOI:10.1007/s00521-020-05532-z] 8. Mahjoub, S., Chrifi-Alaoui, L., Marhic, B., & Delahoche, L. (2022). Predicting Energy Consumption Using LSTM, Multi-Layer GRU and Drop-GRU Neural Networks. Sensors, 22(11), 4062. [ DOI:10.3390/s22114062] 9. Ozcan, A., Catal, C., & Kasif, A. (2021). Energy load forecasting using a dual-stage attention-based recurrent neural network. Sensors, 21(21), 7115. [ DOI:10.3390/s21217115] 10. Shah, I., Iftikhar, H., & Ali, S. (2022). Modeling and forecasting electricity demand and prices: A comparison of alternative approaches. Journal of Mathematics, 2022. [ DOI:10.1155/2022/3581037] 11. Song, X., Liu, Y., Xue, L., Wang, J., Zhang, J., Wang, J., Jiang, L., & Cheng, Z. (2020). Time-series well performance prediction based on Long Short-Term Memory (LSTM) neural network model. Journal of Petroleum Science and Engineering, 186, 106682. [ DOI:10.1016/j.petrol.2019.106682] 12. Tarmanini, C., Sarma, N., Gezegin, C., & Ozgonenel, O. (2023). Short term load forecasting based on ARIMA and ANN approaches. Energy Reports, 9, 550-557. [ DOI:10.1016/j.egyr.2023.01.060] 13. Wang, C., Wang, Y., Ding, Z., Zheng, T., Hu, J., & Zhang, K. (2022). A transformer-based method of multienergy load forecasting in integrated energy system. IEEE Transactions on Smart Grid, 13(4), 2703-2714. [ DOI:10.1109/TSG.2022.3166600] 14. Wang, D., Gan, J., Mao, J., Chen, F., & Yu, L. (2023). Forecasting power demand in China with a CNN-LSTM model including multimodal information. Energy, 263, 126012. [ DOI:10.1016/j.energy.2022.126012] 15. Wang, H., Zhang, Y., Liang, J., & Liu, L. (2023). DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for time series prediction. Neural Networks, 157, 240-256. [ DOI:10.1016/j.neunet.2022.10.009] 16. توان, م., حاجیانی, پ., & پارسا, ح. (1396). ارزیابی الگوهای سری زمانی و فازی برای پیش بینی مصرف برق بخش های مختلف ایران تا افق 1410 دومین همایش بین المللی انسجام مدیریت و اقتصاد در توسعه, https://civilica.com/doc/715782 17. A. Xu, M.-W. T., B. Firouzi, K. A. Alattas, A. Mohammadzadeh, and E. Ghaderpour, "A new deep learning Restricted Boltzmann Machine for energy consumption forecasting," Sustainability, vol. 14, no. 16, p. 10081, 2022. [ DOI:10.3390/su141610081] 18. Al Mamun, A., Sohel, M., Mohammad, N., Sunny, M. S. H., Dipta, D. R., & Hossain, E. (2020). A comprehensive review of the load forecasting techniques using single and hybrid predictive models. IEEE Access, 8, 134911-134939. [ DOI:10.1109/ACCESS.2020.3010702] 19. Amalou, I., Mouhni, N., & Abdali, A. (2022). Multivariate time series prediction by RNN architectures for energy consumption forecasting. Energy Reports, 8, 1084-1091. [ DOI:10.1016/j.egyr.2022.07.139] 20. Cui, C., He, M., Di, F., Lu, Y., Dai, Y., & Lv, F. (2020). Research on power load forecasting method based on LSTM model. 2020 IEEE 5th information technology and mechatronics engineering conference (ITOEC), [ DOI:10.1109/ITOEC49072.2020.9141684] 21. Farsi, B., Amayri, M., Bouguila, N., & Eicker, U. (2021). On short-term load forecasting using machine learning techniques and a novel parallel deep LSTM-CNN approach. IEEE Access, 9, 31191-31212. [ DOI:10.1109/ACCESS.2021.3060290] 22. L'Heureux, A., Grolinger, K., & Capretz, M. A. (2022). Transformer-based model for electrical load forecasting. Energies, 15(14), 4993. [ DOI:10.3390/en15144993] 23. Lu, W., Li, J., Wang, J., & Qin, L. (2021). A CNN-BiLSTM-AM method for stock price prediction. Neural Computing and Applications, 33, 4741-4753. [ DOI:10.1007/s00521-020-05532-z] 24. Mahjoub, S., Chrifi-Alaoui, L., Marhic, B., & Delahoche, L. (2022). Predicting Energy Consumption Using LSTM, Multi-Layer GRU and Drop-GRU Neural Networks. Sensors, 22(11), 4062. [ DOI:10.3390/s22114062] 25. Ozcan, A., Catal, C., & Kasif, A. (2021). Energy load forecasting using a dual-stage attention-based recurrent neural network. Sensors, 21(21), 7115. [ DOI:10.3390/s21217115] 26. Shah, I., Iftikhar, H., & Ali, S. (2022). Modeling and forecasting electricity demand and prices: A comparison of alternative approaches. Journal of Mathematics, 2022. [ DOI:10.1155/2022/3581037] 27. Song, X., Liu, Y., Xue, L., Wang, J., Zhang, J., Wang, J., Jiang, L., & Cheng, Z. (2020). Time-series well performance prediction based on Long Short-Term Memory (LSTM) neural network model. Journal of Petroleum Science and Engineering, 186, 106682. [ DOI:10.1016/j.petrol.2019.106682] 28. Tarmanini, C., Sarma, N., Gezegin, C., & Ozgonenel, O. (2023). Short term load forecasting based on ARIMA and ANN approaches. Energy Reports, 9, 550-557. [ DOI:10.1016/j.egyr.2023.01.060] 29. Wang, C., Wang, Y., Ding, Z., Zheng, T., Hu, J., & Zhang, K. (2022). A transformer-based method of multienergy load forecasting in integrated energy system. IEEE Transactions on Smart Grid, 13(4), 2703-2714. [ DOI:10.1109/TSG.2022.3166600] 30. Wang, D., Gan, J., Mao, J., Chen, F., & Yu, L. (2023). Forecasting power demand in China with a CNN-LSTM model including multimodal information. Energy, 263, 126012. [ DOI:10.1016/j.energy.2022.126012] 31. Wang, H., Zhang, Y., Liang, J., & Liu, L. (2023). DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for time series prediction. Neural Networks, 157, 240-256. [ DOI:10.1016/j.neunet.2022.10.009] 32. توان, م., حاجیانی, پ., & پارسا, ح. (1396). ارزیابی الگوهای سری زمانی و فازی برای پیش بینی مصرف برق بخش های مختلف ایران تا افق 1410 دومین همایش بین المللی انسجام مدیریت و اقتصاد در توسعه, https://civilica.com/doc/715782
|
|
|
|
