Weighted Deep Learning Approach for Better Forecasting

Seng  Hansun; Farica Perdana  Putri

doi:10.6180/jase.202404_27(4).0013

Weighted Deep Learning Approach for Better Forecasting

Computer Science and Information Engineering Mathematics / Statistics

Seng HansunThis email address is being protected from spambots. You need JavaScript enabled to view it., Farica Perdana Putri

Informatics Department, Faculty of Engineering and Informatics, Universitas Multimedia Nusantara, Tangerang, Indonesia

Received: January 29, 2023
Accepted: June 9, 2023
Publication Date: September 27, 2023

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202404_27(4).0013

There are different kinds of time series analysis and forecasting techniques can be found in the literature. The prediction of unknown future values based on known historical data is one of the goals to be achieved. Here, another approach by combining well-known Deep Learning methods with Weighted Moving Average method is introduced. The Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) from the Recurrent Neural Networks family are utilized in this study. We also compare the prediction results of the proposed approach, namely weighted LSTM (w-LSTM) and weighted GRU (w-GRU), with the original implementation of LSTM and GRU. Different scenarios using real-world import values dataset are developed in the experimentation phase. It was found that the proposed approach could get lower Root Mean Square Error, Mean Absolute Error, and Mean Absolute Percentage Error at 1143.242, 999.028, and 0.155 respectively than the original Deep Learning methods.

Keywords: Gated Recurrent Unit; Long Short-Term Memory; Weighted Moving Average; Time series forecasting

[1] OECD. Data and Metadata Reporting and Presentation Handbook. Paris, France: OECD Publishing, 2007, 61.
[2] S. Hansun, V. Charles, C. R. Indrati, and Subanar, (2019) “Revisiting the Holt-Winters’ Additive Method for Better Forecasting" International Journal of Enterprise Information Systems 15(2): 43–57. DOI: 10.4018/IJEIS.2019040103.
[3] Y. Uygun, M. O. Erboy, M. S. Aktas, O. Kalipsiz, and I. Aykurt. “Technical Analysis on Financial Time Series Data Based on Map-Reduce Programming Model: A Case Study”. In: 2018 International Congress on Big Data, Deep Learning and Fighting Cyber Terrorism (IBIGDELFT). Ankara, Turkey: IEEE, 2018, 92–97. DOI: 10.1109/IBIGDELFT.2018.8625357.
[4] V. M. Kureychick and T. G. Kaplunov, (2019) “Time series forecasting method based on genetic algorithm for predicting the conditions of technical systems" Journal of Physics: Conference Series 1333(3): 032046. DOI: 10.1088/1742-6596/1333/3/032046.
[5] S. Makridakis, E. Spiliotis, and V. Assimakopoulos, (2020) “The M4 Competition: 100,000 time series and 61 forecasting methods" International Journal of Forecasting 36(1): 54–74. DOI: 10.1016/j.ijforecast.2019.04.014.
[6] S. Hansun and S. Subanar, (2016) “H-WEMA: A New Approach of Double Exponential Smoothing Method" TELKOMNIKA (Telecommunication Computing Electronics and Control) 14(2): 772. DOI: 10.12928/telkomnika.v14i2.3096.
[7] S. Hansun. “A new approach of moving average method in time series analysis”. In: 2013 Conference on New Media Studies (CoNMedia). IEEE, 2013, 1–4. DOI: 10.1109/CoNMedia.2013.6708545.
[8] G. Zhang, (2003) “Time series forecasting using a hybrid ARIMA and neural network model" Neurocomputing 50: 159–175. DOI: 10.1016/S0925-2312(01)00702-0.
[9] R. H. Shumway and D. S. Stoffer. “ARIMA Models”. In: Time Series Analysis and Its Applications. Springer, Cham, 2017, 75–163. DOI: 10.1007/978-3-319-52452- 8_3.
[10] B. Lim and S. Zohren, (2021) “Time-series forecasting with deep learning: a survey" Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 379(2194): 20200209. DOI: 10.1098/rsta.2020.0209.
[11] S. Hansun and J. C. Young, (2021) “Predicting LQ45 Financial Sector Indices using RNN-LSTM" Journal of Big Data 8(1): 104. DOI: 10.1186/s40537-021-00495-x.
[12] D. Yi, S. Bu, and I. Kim, (2019) “An Enhanced Algorithm of RNN Using Trend in Time-Series" Symmetry 11(7): 912. DOI: 10.3390/sym11070912.
[13] F. Shahid, A. Zameer, and M. Muneeb, (2020) “Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM" Chaos, Solitons & Fractals 140: 110212. DOI: 10.1016/j.chaos.2020.110212.
[14] S. Hansun and A. Suryadibrata, (2021) “Gold Price Prediction in COVID-19 Era" International Journal of Computational Intelligence in Control 13(2):
[15] M. Biswas, A. Shome, M. A. Islam, A. J. Nova, and S. Ahmed. “Predicting Stock Market Price: A Logical Strategy using Deep Learning”. In: 2021 IEEE 11th IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE). Penang, Malaysia: IEEE, 2021, 218–223. DOI: 10.1109/ISCAIE51753.2021.9431817.
[16] J. Zheng, X. Chen, K. Yu, L. Gan, Y. Wang, and K. Wang. “Short-term Power Load Forecasting of Residential Community Based on GRU Neural Network”. In: 2018 International Conference on Power System Technology (POWERCON). Guangzhou, China: IEEE, 2018, 4862–4868. DOI: 10.1109/POWERCON.2018.8601718.
[17] J. Becerra-Rico, M. A. Aceves-Fernández, K. EsquivelEscalante, and J. C. Pedraza-Ortega, (2020) “Airborne particle pollution predictive model using Gated Recurrent Unit (GRU) deep neural networks" Earth Science Informatics 13(3): 821–834. DOI: 10.1007/s12145-020- 00462-9.
[18] M. Xia, H. Shao, X. Ma, and C. W. de Silva, (2021) “A Stacked GRU-RNN-Based Approach for Predicting Renewable Energy and Electricity Load for Smart Grid Operation" IEEE Transactions on Industrial Informatics 17(10): 7050–7059. DOI: 10.1109/TII.2021.3056867.
[19] G. Memarzadeh and F. Keynia, (2021) “Short-term electricity load and price forecasting by a new optimal LSTM-NN based prediction algorithm" Electric Power Systems Research 192: 106995. DOI: 10.1016/j.epsr.2020.106995.
[20] Z. Chang, Y. Zhang, and W. Chen, (2019) “Electricity price prediction based on hybrid model of adam optimized LSTM neural network and wavelet transform" Energy 187: 115804. DOI: 10.1016/j.energy.2019.07.134.
[21] N. Buslim, I. L. Rahmatullah, B. A. Setyawan, and A. Alamsyah. “Comparing Bitcoin’s Prediction Model Using GRU, RNN, and LSTM by Hyperparameter Optimization Grid Search and Random Search”. In: 2021 9th International Conference on Cyber and IT Service Management (CITSM). Bengkulu, Indonesia: IEEE, 2021, 1–6. DOI: 10.1109/CITSM52892.2021.9588947.
[22] F. Ferdiansyah, S. H. Othman, R. Zahilah Raja Md Radzi, D. Stiawan, Y. Sazaki, and U. Ependi. “A LSTM-Method for Bitcoin Price Prediction: A Case Study Yahoo Finance Stock Market”. In: 2019 International Conference on Electrical Engineering and Computer Science (ICECOS). Batam, Indonesia: IEEE, 2019, 206–210. DOI: 10.1109/ICECOS47637.2019.8984499.
[23] M. Pavithra, K. Saruladha, and K. Sathyabama. “GRU Based Deep Learning Model for Prognosis Prediction of Disease Progression”. In: 2019 3rd International Conference on Computing Methodologies and Communication (ICCMC). Erode, India: IEEE, 2019, 840–844. DOI: 10.1109/ICCMC.2019.8819830.
[24] M. Djerioui, Y. Brik, M. Ladjal, and B. Attallah. “Heart Disease prediction using MLP and LSTM models”. In: 2020 International Conference on Electrical Engineering (ICEE). Istanbul, Turkey: IEEE, 2020, 1–5. DOI: 10.1109/ICEE49691.2020.9249935.
[25] S. Hochreiter and J. Schmidhuber, (1997) “Long ShortTerm Memory" Neural Computation 9(8): 1735–1780. DOI: 10.1162/neco.1997.9.8.1735.
[26] A. Sherstinsky, (2020) “Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network" Physica D: Nonlinear Phenomena 404: 132306. DOI: 10.1016/j.physd.2019.132306.
[27] H. M. Lynn, S. B. Pan, and P. Kim, (2019) “A Deep Bidirectional GRU Network Model for Biometric Electrocardiogram Classification Based on Recurrent Neural Networks" IEEE Access 7: 145395–145405. DOI: 10.1109/ACCESS.2019.2939947.
[28] S. Li and X. Zhao, (2019) “Image-Based Concrete Crack Detection Using Convolutional Neural Network and Exhaustive Search Technique" Advances in Civil Engineering 2019: 1–12. DOI: 10.1155/2019/6520620.
[29] K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. “Learning phrase representations using RNN encoder–decoder for statistical machine translation”. In: Conference on Empirical Methods in Natural Language Processing (EMNLP 2014). 2014, 1–15.
[30] T. A. Tang, L. Mhamdi, D. McLernon, S. A. R. Zaidi, and M. Ghogho. “Deep Recurrent Neural Network for Intrusion Detection in SDN-based Networks”. In: 2018 4th IEEE Conference on Network Softwarization and Workshops (NetSoft). Montreal, QC, Canada: IEEE, 2018, 202–206. DOI: 10.1109/NETSOFT.2018.8460090.
[31] Q. Li and Y. Xu, (2019) “VS-GRU: A Variable Sensitive Gated Recurrent Neural Network for Multivariate Time Series with Massive Missing Values" Applied Sciences 9(15): 3041. DOI: 10.3390/app9153041.
[32] S. Hansun, V. Charles, T. Gherman, and V. Varadarajan, (2022) “Hull-WEMA: a novel zero-lag approach in the moving average family, with an application to COVID19" International Journal of Management and Decision Making 21(1): 92–112. DOI: 10.1504/IJMDM.2022.119582.
[33] N. F. Omran, S. F. Abd-el Ghany, H. Saleh, A. A. Ali, A. Gumaei, and M. Al-Rakhami, (2021) “Applying Deep Learning Methods on Time-Series Data for Forecasting COVID-19 in Egypt, Kuwait, and Saudi Arabia" Complexity 2021: 1–13. DOI: 10.1155/2021/6686745.
[34] BPS - Statistics Indonesia. Value of Import Oil&Gas - Non Oil&Gas (Million US). 2022.
[35] A. Raudys, V. Leniauskas, and E. Malius. “Moving Averages for Financial Data Smoothing”. In: Communications in Computer and Information Science, vol 403. Ed. by T. Skersys, R. Butleris, and R. Butkiene. Berlin, Heidelberg: Springer, 2013. Chap. Informatio, 34–45. DOI: 10.1007/978-3-642-41947-8_4.
[36] S. Hansun, A. Wicaksana, and A. Q. M. Khaliq, (2022) “Multivariate cryptocurrency prediction: comparative analysis of three recurrent neural networks approaches" Journal of Big Data 9(50): 1–15. DOI: 10.1186/s40537-022-00601-7.
[37] S. Smyl, (2020) “A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting" International Journal of Forecasting 36(1): 75–85. DOI: 10.1016/j.ijforecast.2019.03.017.
[38] Y. Su, C. Cui, and H. Qu, (2022) “Self-Attentive Moving Average for Time Series Prediction" Applied Sciences 12(7): 3602. DOI: 10.3390/app12073602.

Journal of Applied Science and Engineering

Weighted Deep Learning Approach for Better Forecasting

Latest Articles

You may also like these articles below...

Most Popular Articles

Research Categories