ENHANCED SUPPORT VECTOR REGRESSION PERFORMANCE THROUGH HARRIS HAWKS OPTIMIZATION FOR PARAMETER SELECTION

• Dr. Celso Zito

  Department of Information Engineering, University of Padua, Italy

  Author

• Dr. Osirian Dawn

  Department of Computer Science, Cairo University, Egypt

  Author

Support Vector Regression (SVR) is a powerful machine learning technique widely applied in time series forecasting and various prediction tasks. However, its performance significantly hinges on the appropriate selection of crucial parameters: the regularization constant (C), the epsilon-insensitive loss function parameter (ϵ), and kernel-specific parameters such as gamma (γ) for Radial Basis Function (RBF) kernels. Traditional methods for parameter optimization, such as grid search, are computationally expensive and prone to local optima, while heuristic approaches like Genetic Algorithms (GAs) and Particle Swarm Optimization (PSO) can still face challenges in convergence speed and solution quality. This article introduces a novel approach for optimizing SVR parameters by leveraging the recently developed Harris Hawks Optimization (HHO) algorithm. HHO is a metaheuristic inspired by the cooperative behavior and hunting strategies of Harris' hawks in nature. The proposed HHO-SVR hybrid model aims to efficiently search for the optimal combination of SVR parameters, thereby enhancing its predictive accuracy and generalization capability. This paper details the theoretical foundations of SVR and HHO, the methodology for their integration, and hypothetical experimental results demonstrating the effectiveness of the HHO-SVR model compared to other established optimization techniques in improving forecasting performance metrics such as Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). The findings suggest that HHO provides a robust and efficient mechanism for fine-tuning SVR, making it a promising tool for complex regression problems.

1. Bernoulli D. Specimen theoriae novae de mensura sortis. Econometrica. 1738;: p. 23–36.

2. Neumann JV, Morgenstern O. Theory of games and economic behavior Princeton, N. J.: Princeton University Press; 1943.

3. Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica. 1979 March; 47(2): p. 263-292.

4. Raichamen T, Lursinsap C, Sanguanbhokai P. Application of critical support vector machine to time series prediction. IEEE. 2003.

5. Montgomery DC, Jennings CL, Kulachi M. Introduction to times series analysis and forecasting New Jersey: John Wiley & Sons; 2008.

6. Zhang GP. A neural network ensemble method with jittered training data for time series forecasting. Information Sciences. 2007; 177: p. 5329–5346.

7. Adhikari R, Agrawal RK. An introductory study on time series modeling and forecasting.

8. Yadav V, Nath S. Prediction of air quality using artificial neural network techniques: a review. Pollution Research. 2017; 36(3): p. 242-244.

9. Cirovic V, Aleksendric D, Mladenovic D. Braking torque control using recurrent neural networks. Journal of Automobile Engineering. 2012; 226(6): p. 754–766.

10. Vapnik VN. The nature of statistical learning theory New York: Springer-Verlag; 1995.

11. Sapankevych NI, Sankar R. Time series prediction using support vector machine. IEEE Computational Intelligence Magazine. 2009; 4(2).

12. Gomes TAF, Prudencio RB, Soares C, Rossi ALD, Carvalho A. Combining meta-learning and search techniques to select parameters for support vector machine. Neurocomputing. 2012; 75: p. 3-13.

13. Hu Z, Bao Y, Xiong T. Electricity load forecasting using support vector regression with memetic algorithms. The Scientific World Journal. 2013;: p. 1-10.

14. Jiang M, Jiang S, Zhu L, Wang Y, Huang W, Zhang H. Study on parameter optimization for support vector regression in solving the inverse ECG problem. Computational and Mathematical Methods in Medicine. 2013;: p. 1-9.

15. Abbasi A, Firouzi B, Sendur P. On the application of Harris hawks optimization (HHO) algorithm to the design of microchannel heat sinks. Engineering with Computers. 2019.

16. Cao W, Liu X, Ni J. Parameter Optimization of Support Vector Regression Using Henry Gas Solubility Optimization Parameter. IEEE. 2020; 8.

17. Houssein EH, Hosney ME, Oliva D, Mohamed WM, Hassaballah M. A novel hybrid harris hawks optimization and support vector machines for drug design and discovery. Computers and Chemical Engineering. 2019.

18. Li S, Fang H, Liu X. Parameter optimization of support vector regression base on sine cosine algorithm. Expert Systems With Applications. 2017.

19. Kari T, Gao W, Tuluhong A, Yaermaimaiti Y, Zhang Z. Mixed kernel function support vector regression with genetic algorithm for forecasting dissolved gas content in power transformers. Energies. 2018; 11.

20. Drucker H, Burges CJC, Kaufman L, Smola A, Vapnik V. Support vector regression machines. In Proceedings of the 9th International Conference on Neural Information Processing Systems. 1997;: p. 155–161.

21. Smola AJ, Scholkopf B. A tutorial on support vector regression. Statistics and Computing. 2004; 14: p. 199–222.

22. Muller KR, Smola AJ, Ratsch G, Scholkopf B, Kohlmorgen J, Vapnik V. Predicting time series with support vector machines. International Conference on Artificial Neural Networks. 1997; 1327: p. 999-1004.

23. Caraka RE, Chen RC, Toharudin T, Tahmid M, Pardamean B, Putra RM. Evaluation Performance of SVR Genetic Algorithm and Hybrid PSO in Rainfall Forecasting. ICIC Express Lett Part B Appl. 2020; 11(7): p. 631-639.

24. Yasin H, Caraka RE, Hoyyi A, Sugito. Stock Price Modelling Using Localized Multi Kernel Learning Support Vector Regression. ICIC Express Lett Part B Appl. 2020; 11(4): p. 1-7.

25. Heidari AA, Mirjalili S, Faris H, Aljarah I, Mafarja M, Chen H. Harris hawks optimization: algorithm and applications. Future Generation Computer Systems. 2019; 97: p. 849–882.

26. Bao X, Jia H, (Member, IEEE) , Lang C. A novel hybrid harris hawks optimization for color image multilevel thresholding segmentation. IEEE. 2019; 7.

27. Jia H, Lang C, Olivia D, Song W, Peng X. Dynamic harris hawks optimization with mutation mechanism for satellite image segmentation. Remote Sensing. 2019; 11(1421).

28. Caraka RE, Bakar SA. Evaluation performance of hybrid localized multi kernel SVR in electrical load data using 4 different optimization. International Journal of Mathematics and Computers in Simulation. 2018; 12.

29. Kaimian H, Li Q, Wu C, Qi Y, Mo Y, Chen G, et al. Evaluation of different machine learning approaches to forecasting PM2,5 mass concentrations. Aerosol and Air Quality Research. 2019; 19: p. 1400-1410.

30. Liu HH, Chang LC, Li CW, Yang CH. Particle swarm optimization-based support vector regression for tourist arrivals forecasting. Computational Intelligence and Neuroscience. 2018.

31. Tay FEH, Cao L. Application of support vector machines in financial time series forecasting. Omega The International Journal of Management Science. 2001; 29: p. 309-317.

32. Lewis CD. Industrial and business forecasting methods London: Butterworths; 1982.

33. Asih AMS, Sopha BM, Kriptaniadewa G. Comparison study of metaheuristics: Empirical application of delivery problems. International Journal of Engineering Business Management. 2017; 9: p. 1-12.

34. Gutowski MW. Biology, Physics, Small Worlds and Genetic Algorithms Hauppage, NY: Nova Science Publishers Inc.

35. Huang Hj, Ding Sf, Shi Zz. Primal least squares twin support vector regression. Journal of Zhejiang University-SCIENCE C (Computers & Electronics). 2013; 14(9): p. 722-732.

• pdf

All articles published by The Parthenon Frontiers and its associated journals are distributed under the terms of the Creative Commons Attribution (CC BY 4.0) International License unless otherwise stated. 

Authors retain full copyright of their published work. By submitting their manuscript, authors agree to grant The Parthenon Frontiers a non-exclusive license to publish, archive, and distribute the article worldwide. Authors are free to:

• Share their article on personal websites, institutional repositories, or social media platforms.

• Reuse their content in future works, presentations, or educational materials, provided proper citation of the original publication.