ارزیابی تطبیقی و بهینه‌سازی مدل‌های یادگیری ماشین در تخمین گرانروی نفت مرده

نوع مقاله : مقاله پژوهشی

نویسندگان

1 مرکز ازدیاد برداشت از مخازن نفت و گاز، دانشکده مهندسی شیمی، نفت و گاز، دانشگاه شیراز، شیراز، ایران/ بخش مهندسی نفت، دانشکده مهندسی شیمی، نفت و گاز، دانشگاه شیراز، شیراز، ایران

2 مرکز ازدیاد برداشت از مخازن نفت و گاز، دانشکده مهندسی شیمی، نفت و گاز، دانشگاه شیراز، شیراز، ایران

3 دانشکده معدن و علوم زمین، دانشگاه نظربایف، آستانه، قزاقستان

4 بخش مهندسی نفت، دانشکده مهندسی شیمی، نفت و گاز، دانشگاه شیراز، شیراز، ایران

5 دانشکده فنی فومن، دانشگاه تهران، گیلان، ایران

6 بخش مهندسی شیمی، دانشکده مهندسی شیمی، نفت و گاز، دانشگاه شیراز، شیراز، ایران

10.22078/pr.2025.5728.3547

چکیده

درک گرانروی نفت نقشی اساسی در حل چالش‌های متنوع مهندسی مخزن دارد. اندازه‌گیری تجربی این ویژگی عموماً پرهزینه و زمان‌بر است و ازاین‌رو، توسعه مدل‌های پیش‌بینی دقیق و کارآمد اهمیت ویژه‌ای دارد. در این پژوهش، افزون بر توسعه یک شبکه عصبی پرسپترون چندلایه برای برآورد گرانروی نفت مرده و ارزیابی ده تابع بهینه‌سازی، عملکرد مدل‌های یادگیری ماشین دیگر شامل جنگل تصادفی، گرادیان بوستینگ و برازش بردار پشتیبان نیز مقایسه گردید. برای شبکه عصبی پرسپترون چندلایه، تأثیر ابرپارامترهای کلیدی نظیر نرخ یادگیری، تعداد لایه‌های پنهان و پیکربندی نورون‌ها به‌صورت نظام‌مند بررسی شد تا ساختار بهینه تعیین گردد. نوآوری اصلی پژوهش، به‌کارگیری هم‌زمان ویژگی‌های فیزیکی-شیمیایی و ویژگی‌های ترکیبی برش‌های مختلف نفت‌خام (از سبک تا سنگین) در قالب مجموعه‌داده‌ای جامع شامل ۲۲۹۹ نمونه است که به افزایش دقت و تعمیم‌پذیری مدل‌ها منجر شد. نتایج نشان داد الگوریتم گرادیان بوستینگ با دست‌یابی به تعادل میان دقت و توان تعمیم‌دهی، بهترین عملکرد کلی را ارائه می‌دهد (8266/0=R² ، 0012/2 =RMSE و 5004/3 =AAPRE در داده‌های آزمایش). مدل جنگل تصادفی نیز با ثبت مقدار 8191/0 =R² در داده‌های آزمایش، عملکردی رقابتی داشت، هرچند در مرحله آموزش (9749/0=R²) نشانه‌هایی از بیش‌‌برازش مشاهده شد. مدل MLP-AdamW با 8541/0=R² و 8064/1 =RMSE در داده‌های آزمایش دقت قابل‌توجهی ارائه داد و نشان داد شبکه‌های عصبی در شناسایی روابط غیرخطی میان متغیرها مؤثر هستند، هرچند حساسیت بیشتری نسبت به ناهمگنی داده‌ها دارند. در مقابل، مدل SVR به‌دلیل ماهیت نسبتاً خطی خود، پایین‌ترین دقت را در داده‌های غیرخطی و حجیم نشان داد (7699/=0 R²). بر این اساس، می‌توان نتیجه گرفت که الگوریتم گرادیان بوستینگ با بهره‌گیری از یادگیری مرحله‌ای و جلوگیری از بیش‌برازش، گزینه بهینه برای پیش‌بینی گرانروی نفت مرده در داده‌های وسیع محسوب می‌شود.

کلیدواژه‌ها

موضوعات


عنوان مقاله [English]

Comparative Evaluation and Optimization of Machine Learning Models for Dead Oil Viscosity Prediction

نویسندگان [English]

  • Mohammad Haji-Savameri 1
  • Rafat Parsaei 2
  • Masoud Riazi 3
  • Jafar Qajar 4
  • Suleiman Hassan 3
  • Ali Safaei 5
  • Payam Sotoudeh 6
1 Enhanced Oil Recovery (EOR) Research Centre, IOR/EOR Research Institute, Shiraz University, Shiraz, Iran\ Department of Petroleum Engineering, School of Chemical and Petroleum Engineering, Shiraz University, Shiraz, Iran
2 Enhanced Oil Recovery (EOR) Research Centre, IOR/EOR Research Institute, Shiraz University, Shiraz, Iran
3 School of Mining and Geosciences, Nazarbayev University, Astana, Kazakhstan
4 Fouman Faculty of Engineering, College of Engineering, University of Tehran, Tehran, Iran
5 Fouman Faculty of Engineering, College of Engineering, University of Tehran, Tehran, Iran
6 Department of Chemical Engineering, School of Chemical and Petroleum Engineering, Shiraz University, Shiraz, Iran
چکیده [English]

Understanding oil viscosity plays a vital role in addressing various reservoir engineering challenges. Since the experimental measurement of viscosity is both costly and time-consuming, it is essential to develop accurate and efficient predictive models. In this study, in addition to developing a multilayer perceptron (MLP) neural network for estimating dead oil viscosity and evaluating ten optimization functions, three additional machine learning algorithms—random forest, gradient boosting, and support vector regression (SVR)—were comparatively analyzed. For the MLP model, the effects of key hyperparameters, such as learning rate, number of hidden layers, and neuron configuration, were systematically examined to determine the optimal architecture. The main innovation of this work lies in integrating both physicochemical and compositional features of crude oil fractions (from light to heavy cuts) within a comprehensive dataset of 2,299 samples, which significantly enhances model accuracy and generalization. Statistical analyses revealed that the Gradient Boosting algorithm achieved the best balance between accuracy and generalization (R² = 0.8266, RMSE = 2.0012, AAPRE = 3.5004 for the test data). The Random Forest model exhibited competitive results (R² = 0.8191 for testing, 0.9749 for training), though slight overfitting was observed. The MLP-AdamW model attained strong accuracy (R² = 0.8541, RMSE = 1.8064 on the test set), confirming the effectiveness of neural networks in capturing nonlinear relationships despite their sensitivity to data heterogeneity. Conversely, the SVR model, due to its relatively linear nature, yielded the lowest accuracy on large and nonlinear datasets (R² = 0.7699). Overall, the gradient boosting demonstrated the most robust and balanced performance, making it the optimal approach for large-scale dead oil viscosity prediction.

کلیدواژه‌ها [English]

  • Viscosity
  • Machine Learning
  • Multilayer Perceptron Neural Network
  • Random Forest
  • Gradient Boosting
  • Support Vector Regression
[1]. Clark, N. J. (1962). Fundamentals of Reservoir Fluids. Journal of Petroleum Technology, 14(01), 11-16. doi.org/10.2118/91-PA.##
[2]. Xu, D. H., & Khurana, A. K. (1996, October). A simple and efficient approach for improving the prediction of reservoir fluid viscosity. In SPE Asia Pacific Oil and Gas Conference and Exhibition (pp. SPE-37011). SPE. doi.org/10.2118/37011-MS.##
[3]. Raupov, I., Rogachev, M., & Sytnik, J. (2025). Overview of modern methods and technologies for the well production of high-and extra-high-viscous oil. Energies, 18(6), 1498.##
[4]. El-Hoshoudy, A. N., Farag, A. B., Ali, O. I. M., El-Batanoney, M. H., Desouky, S. E. M., & Ramzi, M. (2013). New correlations for prediction of viscosity and density of Egyptian oil reservoirs. Fuel, 112, 277-282. doi.org/10.1016/j.fuel.2013.05.045.##
[5]. Shao, M., Shao, Y., Wang, Y., Fu, L., Liao, K., Zhu, T., & Yang, Z. (2026). A review and perspective on viscosity reduction development technologies for heavy oil. Petroleum Science and Technology, 44(12), 1835-1851. doi.org/10.1080/10916466.2025.2465835.##
[6]. McCain, W.D., J.P. Spivey, and C.P. Lenn, Petroleum reservoir fluid property correlations. 2011: PennWell Books. ##
[7]. Zhang, Y., Yang, S., Luo, Y., Han, Y., Tian, L., Liu, Q., Gao, X., Shen, B., Yang, R. and Li, J., 2025. Visual characterization, exergy and mechanism analysis of in-situ nonequilibrium dynamic phase-behavior of CO2-oil. Energy, 328, p.136560. doi.org/10.1016/j.energy.2025.136560. ##
[8]. Makinde, F. A., Ako, C. T., Orodu, O. D., & Asuquo, I. U. (2012). Prediction of crude oil viscosity using feed-forward back-propagation neural network (FFBPNN). Petroleum & Coal, 54(2). Available online at www.vurup.sk/petroleum-coal. ##
[9]. McCain Jr, W. D. (1991). Reservoir-fluid property correlations—state of the art. SPE Reservoir Engineering, 6(02), 266-272. ##
[10]. Hemmati-Sarapardeh, A., et al., Reservoir oil viscosity determination using a rigorous approach. Fuel, 2014. 116: pp. 39-48.
[11]. Hemmati-Sarapardeh, A., Khishvand, M., Naseri, A., & Mohammadi, A. H. (2013). Toward reservoir oil viscosity correlation. Chemical Engineering Science, 90, 53-68. doi.org/10.1016/j.ces.2012.12.009. ##
[12]. Dolatyari, A., Ahmady, M., & Kazemi, A. (2024). A novel mathematical model for modeling viscosity and temperature relationship for dead oils. Scientific Reports, 14(1), 22836. ##
[13]. Ghorbani, B., Ziabasharhagh, M., & Amidpour, M. (2014). A hybrid artificial neural network and genetic algorithm for predicting viscosity of Iranian crude oils. Journal of Natural Gas Science and Engineering, 18, 312-323. ##
[14]. Al-Gathe, A. A., Abd-El Fattah, K. A., El-Banbi, A. H., & El-Metwally, K. A. (2015). A hybrid neuro-fuzzy approach for black oil viscosity prediction. International Journal of Innovation and Applied Studies, 13(4), 946. ##
[15]. Soltanbekova, K., Ramazanova, G., & Zhapbasbayev, U. (2025). Potential of Gas-Enhanced Oil Recovery (EOR) Methods for High-Viscosity Oil: A Core Study from a Kazakhstani Reservoir. Energies, 18(15), 4182. doi.org/10.3390/en18154182. ##
[16]. Al-Marhoun, M. A., Nizamuddin, S., Raheem, A. A., Ali, S. S., & Muhammadain, A. A. (2012). Prediction of crude oil viscosity curve using artificial intelligence techniques. Journal of Petroleum Science and Engineering, 86, 111-117. doi: 10.1016/j.petrol.2012.03.029. ##
[17]. Glaso, O. (1980). Generalized pressure-volume-temperature correlations. Journal of Petroleum Technology, 32(05), 785-795. doi.org/10.2118/8016-PA. ##
[18]. Beggs, H. D., & Robinson, J. R. (1975). Estimating the viscosity of crude oil systems. Journal of Petroleum technology, 27(09), 1140-1141. doi.org/10.2118/5434-PA. ##
[19]. Al-Khafaji, A. H., Abdul-Majeed, G. H., & Hassoon, S. F. (1987). Viscosity correlation for dead, live and undersaturated crude oils. J. Pet. Res, 6(2), 1-16. ##
[20]. Beal, C. (1946). The viscosity of air, water, natural gas, crude oil and its associated gases at oil field temperatures and pressures. Transactions of the AIME, 165(01), 94-115. doi.org/10.2118/946094-G. ##
[21]. Kaye, S. E. (1985). Offshore California viscosity correlations. COFRC, TS85000940. ##
[22]. Alomair, O., Elsharkawy, A., & Alkandari, H. (2011, December). Viscosity predictions of Kuwaiti heavy crudes at elevated temperatures. In SPE International Heavy Oil Conference and Exhibition (pp. SPE-150503). SPE. doi.org/10.2118/150503-MS. ##
[23]. Naseri, A., Nikazar, M., & Dehghani, S. M. (2005). A correlation approach for prediction of crude oil viscosities. Journal of petroleum science and engineering, 47(3-4), 163-174. doi: 10.1016/j.petrol.2005.03.008. ##
[24]. Hossain, M. S., Sarica, C., Zhang, H. Q., Rhyne, L., & Greenhill, K. L. (2005, November). Assessment and development of heavy-oil viscosity correlations. In SPE International Thermal Operations and Heavy Oil Symposium (pp. SPE-97907). SPE. doi.org/10.2118/97907-MS. ##
[25]. Elsharkawy, A. M., & Alikhan, A. A. (1999). Models for predicting the viscosity of Middle East crude oils. Fuel, 78(8), 891-903. doi.org/10.1016/S0016-2361(99)00019-8. ##
[26]. Bennison, T. (1998, December). Prediction of heavy oil viscosity. In IBC Heavy oil field development conference (Vol. 2, p. 4). Citeseer. ##
[27]. De Ghetto, G., Paone, F., & Villa, M. (1994, October). Reliability analysis on PVT correlations. In SPE Europec featured at EAGE Conference and Exhibition? (pp. SPE-28904). SPE. doi.org/10.2118/28904-MS. ##
[28]. Kartoatmodjo, T., & Schmidt, Z. (1994). Large data bank improves crude physical property correlations. Oil and Gas Journal;(United States), 92(27). ##
[29]. Labedi, R. (1992). Improved correlations for predicting the viscosity of light crudes. Journal of Petroleum Science and Engineering, 8(3), 221-234. doi: 10.1016/0920-4105(92)90035-Y. ##
[30]. Egbogah, E. O., & Ng, J. T. (1990). An improved temperature-viscosity correlation for crude oil systems. Journal of Petroleum Science and Engineering, 4(3), 197-200. doi: 10.1016/0920-4105(90)90009-R. ##
[31]. Petrosky, G. E. (1990). PVT correlations for gulf of mexico crude oils (Doctoral dissertation, University of Southwestern Louisiana). ##
[32]. Teja, A. S., & Rice, P. (1981). Generalized corresponding states method for the viscosities of liquid mixtures. Industrial & engineering chemistry fundamentals, 20(1), 77-81. doi.org/10.1021/i100001a015. doi.org/10.1021/i100001a015. ##
[33]. Mehrotra, A. K. (1991). Generalized one-parameter viscosity equation for light and medium liquid hydrocarbons. Industrial & engineering chemistry research, 30(6), 1367-1372. doi.org/10.1021/ie00054a044.
[34]. Svrcek, W. Y., & Mehrotra, A. K. (1988). One parameter correlation for bitumen viscosity. Chem. Eng. Res. Des.;(United Kingdom), 66(4). ##
[35]. Johnson, S. E., Svrcek, W. Y., & Mehrotra, A. K. (1987). Viscosity prediction of Athabasca bitumen using the extended principle of corresponding states. Industrial & engineering chemistry research, 26(11), 2290-2298. doi.org/10.1021/ie00071a020. ##
[36]. Al-Marhoun, M. A. (2004). Evaluation of empirically derived PVT properties for Middle East crude oils. Journal of Petroleum Science and Engineering, 42(2-4), 209-221. doi; 10.1016/j.petrol.2003.12.012. ##
[37]. Alboudwarej, H., Svrcek, W. Y., & Yarranton, H. W. (2002, June). PVT investigation of asphaltene precipitation and redissolution from bitumens. In PETSOC Canadian International Petroleum Conference (pp. PETSOC-2002). PETSOC. doi.org/10.2118/2002-015-EA. ##
[38]. Hadavimoghaddam, F., Ostadhassan, M., Heidaryan, E., Sadri, M.A., Chapanova, I., Popov, E., Cheremisin, A. and Rafieepour, S., (2021). Prediction of dead oil viscosity: Machine learning vs. classical correlations. Energies, 14(4), p.930. doi.org/10.3390/en14040930. ##
[39]. Oloso, M. A., Hassan, M. G., Bader-El-Den, M. B., & Buick, J. M. (2018). Ensemble SVM for characterisation of crude oil viscosity. Journal of Petroleum Exploration and Production Technology, 8(2), 531-546. ##
[40]. Ghorbani, B., Hamedi, M., Shirmohammadi, R., Mehrpooya, M., & Hamedi, M. H. (2016). A novel multi-hybrid model for estimating optimal viscosity correlations of Iranian crude oil. Journal of Petroleum Science and Engineering, 142, 68-76. doi: 10.1016/j.petrol.2016.01.041. ##
[41]. Hemmati-Sarapardeh, A., Aminshahidy, B., Pajouhandeh, A., Yousefi, S. H., & Hosseini-Kaldozakh, S. A. (2016). A soft computing approach for the determination of crude oil viscosity: Light and intermediate crude oil systems. Journal of the Taiwan Institute of Chemical Engineers, 59, 1-10. doi.org/10.1016/j.jtice.2015.07.017. ##
[42]. Naseri, A., Yousefi, S. H., Sanaei, A., & Gharesheikhlou, A. A. (2012). A neural network model and an updated correlation for estimation of dead crude oil viscosity. Brazilian journal of petroleum and gas, 6(1). ##
[43]. Oloso, M. A., Khoukhi, A., Abdulraheem, A., & Elshafei, M. (2009, October). Prediction of crude oil viscosity and gas/oil ratio curves using recent advances to neural networks. In SPE/EAGE Reservoir Characterization & Simulation Conference (pp. cp-170). European Association of Geoscientists & Engineers. doi.org/10.3997/2214-4609-pdb.170.spe125360. ##
[44]. Torabi, F., Abedini, A., & Abedini, R. (2011). The development of an artificial neural network model for prediction of crude oil viscosities. Petroleum Science and Technology, 29(8), 804-816. doi.org/10.1080/10916460903485876. ##
[45]. Gharbi, R. B., & Elsharkawy, A. M. (1997, March). Neural network model for estimating the PVT properties of Middle East crude oils. In SPE Middle East Oil and Gas Show and Conference (pp. SPE-37695). SPE. doi.org/10.2118/37695-MS. ##
[46]. Mohd Fadzil, M. A., Razak, S. A., Razali, A. A., Salleh, M. N. M., & Zabiri, H. (2025). Calculation of the Viscosity of a Mixture of Heavy Hydrocarbon Oil with Temperature Relation Consideration. ACS omega, 10(29), 31274-31297. doi.org/10.1021/acsomega.4c06517. ##
[47]. Mohammadi A., Kheirollahi S., Ayatollahi H., Pishvai S. Sh., Seyed Mahmoud Reza. (2025). Using machine learning models to predict acid and oil emulsion formation in static acidizing tests using a hybrid database. Petroleum Research, February-March. 34, 2025-6 (139): 73-91. 10.22078/pr.2024.5452.3426. ##
[48]. Kheirollahi, H., Zaedi, M., Sobhani, S., Chehardoli, M., Simju, M. (2023). Screening of oil reservoir EOR methods using integration of artificial intelligence methods. Petroleum Research, 33, 2023-5 (132): 51-62. doi:10.22078/pr.2023.5151.3284. ##
[49]. Li, D., Zhang, X., & Kang, Q. (2023). Machine learning estimation of crude oil viscosity as function of API, temperature, and oil composition: Model optimization and design space. Plos one, 18(2), e0282084. doi.org/10.1371/journal.pone.0282084. ##
[50]. Langeroudy, K. P. A., Esfahani, P. K., & Movaghar, M. R. K. (2023). Enhanced intelligent approach for determination of crude oil viscosity at reservoir conditions. Scientific Reports, 13, 1666. doi: 10.1038/s41598-023-28770-2. ##
[51]. Fotias, S. P., & Gaganis, V. (2023). Workflow for predicting undersaturated oil viscosity using machine
learning. Results in Engineering, 20, 101502. doi.org/10.1016/j.rineng.2023.101502. ##
[52]. Hadavimoghaddam, F., Ostadhassan, M., Heidaryan, E., Sadri, M.A., Chapanova, I., Popov, E., Cheremisin, A. and Rafieepour, S., 2021. Prediction of dead oil viscosity: Machine learning vs. classical correlations. Energies, 14(4), p.930. doi.org/10.3390/en14040930. ##
[53]. Bahonar, E., Chahardowli, M., Ghalenoei, Y., & Simjoo, M. (2022). New correlations to predict oil viscosity using data mining techniques. Journal of Petroleum Science and Engineering, 208, 109736. doi: 10.1016/j.petrol.2021.109736. ##
[54]. Dolatyari, A., Ahmady, M., & Kazemi, A. (2024). A novel mathematical model for modeling viscosity and temperature relationship for dead oils. Scientific Reports, 14(1), 22836. ##
[55]. Gao, X., Dong, P., Cui, J., & Gao, Q. (2022). Prediction model for the viscosity of heavy oil diluted with light oil using machine learning techniques. Energies, 15(6), 2297. doi.org/10.3390/en15062297. ##
[56]. Stratiev, D., Nenov, S., Shishkova, I., Sotirov, S., Sotirova, E., Dinkov, R., Yordanov, D., Pilev, D., Atanassov, K., Vasilev, S. and Stratiev, D.D. (2023). Prediction of viscosity of blends of heavy oils with diluents by empirical correlations and artificial neural network. Industrial & Engineering Chemistry Research, 62(49), pp.21449-21463. doi.org/10.1021/acs.iecr.3c02472. ##
[57]. Almudhhi, S., Lababidi, H. M., & Garrouch, A. A. (2025). Application of machine learning for modeling heavy oil viscosity. Journal of Engineering Research. doi.org/10.1016/j.jer.2025.05.009. ##
[58]. Lei, H., Hussein, M.A., Vora, T., Yadav, A., Rajiv, A., Shankhyan, A., Jaidka, S., Manu, M., Kadhim, I.M., Hamoodah, Z.J. and Faez, F., 2025. Accurate machine-learning based modeling of apparent viscosity of waxy oils mixed with polymeric wax inhibitors. Flow Measurement and Instrumentation, p.102968. doi.org/10.1016/j.flowmeasinst.2025.102968. ##
[59]. Lillis, P., & DOE, N.  , Crude Oil Analysis (COA) database. [Data set]. Open Energy Data Initiative (OEDI). National Renewable Energy Laboratory., 2019. https://data.openei.org/submissions/178. ##
[60]. Bebis, G., & Georgiopoulos, M. (2002). Feed-forward neural networks. Ieee Potentials, 13(4), 27-31.
[61]. Echamsi, S., Belouafi, E., Kassimi, A. E. B., Alali, A., Bouroumi, A., & Guennoun, A. (2025, May). ##Optimizing Training Hyperparameters for Multilayer Perceptrons in Deep Learning. In 2025 5th International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET) (pp. 1-8). IEEE.
[62]. Irmak, B., Karakoyun, M., & Gülcü, Ş. (2023). An improved butterfly optimization algorithm for training the feed-forward artificial neural networks. Soft Computing, 27(7), 3887-3905. ##
[63]. Bansal, P., Gupta, S., Kumar, S., Gorelick, S., & Sharma, S. (2019). MLP-LOA: a metaheuristic approach to design an optimal multilayer perceptron: P. Bansal et al. Soft Computing, 23(23), 12331-12345. ##
[64]. Mirjalili, S. Z., Saremi, S., & Mirjalili, S. M. (2015). Designing evolutionary feedforward neural networks using social spider optimization algorithm. Neural Computing and Applications, 26(8), 1919-1928. ##
[65]. Lv, L., Zhang, C., Fan, J., & Shen, W. (2025). Deep reinforcement learning for job shop scheduling problems: A comprehensive literature review. Knowledge-Based Systems, 321, 113633. doi: 10.1016/j.knosys.2025.113633. ##
[66]. Pamadi, V. N., & Rastogi, D. (2025). Optimizing Neural Network Performance Through Adaptive Learning Algorithms. International Journal of Research in all Subjects in Multi Languages [Subject: Computer Science], 13, 2321-2853. ##
[67]. Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12(7). ##
[68]. Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://doi.org/10.48550/arXiv.1412.6980. ##
[69]. Almudhhi, S., Lababidi, H. M., & Garrouch, A. A. (2025). Application of machine learning for modeling heavy oil viscosity. Journal of Engineering Research. doi.org/10.1016/j.jer.2025.05.009. ##
[70]. Liu, Y., Pan, R., & Zhang, T. (2025, May). Adagrad under anisotropic smoothness. In International Conference on Learning Representations (Vol. 2025, pp. 19574-19608). ##
[71]. Jiang, R., Maladkar, D., & Mokhtari, A. (2024). Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization. arXiv preprint arXiv:2406.04592. ##
[72]. Liu, Q., Yang, S., Yu, D., & Gai, T. (2024). BP neural network construction cost prediction model based on different optimization algorithms. doi.org/10.21203/rs.3.rs-3602749/v1. ##
[73]. Poudel, A., Burns, D., Adhikari, R., Duron, D., Hendrix, J., Gentimis, T., Tubana, B. and Setiyono, T., (2025). Cover Crop Biomass Predictions with Unmanned Aerial Vehicle Remote Sensing and TensorFlow Machine Learning. Drones, 9(2), 131. doi.org/10.3390/drones9020131. ##
[74]. Uppal, M., Gupta, D., Juneja, S., Gadekallu, T. R., El Bayoumy, I., Hussain, J., & Lee, S. W. (2023). Enhancing accuracy in brain stroke detection: Multi-layer perceptron with Adadelta, RMSProp and AdaMax optimizers. Frontiers in Bioengineering and Biotechnology, 11, 1257591. doi.org/10.3389/fbioe.2023.1257591. ##
[75]. Sakketou, F., & Ampazis, N. (2024, September). A Computationally Efficient Second Order Deep Learning Training Algorithm with Adaptive Momentum. In Proceedings of the 13th Hellenic Conference on Artificial Intelligence (pp. 1-6). doi.org/10.1145/3688671.3688754. ##
[76]. Upadhyay, R., Pipersenia, A. K., & Nidhya, M. S. (2024, January). Analyzing Multilayer Perception Architectures for Reinforcement Learning. In 2024 International Conference on Optimization Computing and Wireless Communication (ICOCWC) (pp. 1-7). IEEE. doi: 10.1109/ICOCWC60930.2024.10470491. ##
[77]. Elshamy, R., Abu-Elnasr, O., Elhoseny, M., & Elmougy, S. (2023). Improving the efficiency of RMSProp optimizer by utilizing Nestrove in deep learning. Scientific Reports, 13(1), 8814. ##
[78]. Kurbiel, T., & Khaleghian, S. (2017). Training of deep neural networks based on distance measures using RMSProp. arXiv preprint arXiv:1708.01911. doi.org/10.48550/arXiv.1708.01911. ##
[79]. Riedmiller, M., & Braun, H. (1993, March). A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In IEEE international conference on neural networks (pp. 586-591). IEEE. ##
[80]. Norouzian, M., Orosnjak, M., Kedziora, S. A., Useldinger, R., & Aslan, F. Acoustic Emission Signatures as Predictors of Laser Weld Integrity: A Machine Learning Framework for Real-Time Laser Welding Quality Assessment. Available at SSRN 5556630. ##
[81]. Saini, R. (2025). A review on artificial neural networks for structural analysis. Journal of Vibration Engineering & Technologies, 13(2), 142. ##
[82]. Babu, H., Ganesan, L. K., Kanniappan, J., & Marimuthu, H. R. (2025, March). Prediction of oral cancer with CT scanned images and Dysplasia slides using machine learning algorithm. In AIP Conference Proceedings (Vol. 3175, No. 1, p. 020036). AIP Publishing LLC. ##
[83]. Bottou, L., & Bousquet, O. (2007). The tradeoffs of large scale learning. Advances in neural information processing systems, 20. ##
[84]. Patnala, S. K., BG, S., Sethi, A., Gupta, S., & Asthana, S. (2024, January). SGD-MLP: Structure Generation and Distillation using a graph free MLP. In Proceedings of the 7th Joint International Conference on Data Science & Management of Data (11th ACM IKDD CODS and 29th COMAD) (pp. 64-72). doi.org/10.1145/3632410.3632431. ##
[85]. Bottou, L. (1998). Online algorithms and stochastic approximations. Online learning in neural networks.
[86]. Talebi, H., Samadianfard, S., & Valizadeh Kamran, K. (2025). Estimation of daily reference evapotranspiration implementing satellite image data and strategy of ensemble optimization algorithm of stochastic gradient descent with multilayer perceptron. Environment, Development and Sustainability, 27(2), 3707-3729. ##
[87]. Shi, H., Yang, N., Tang, H., & Yang, X. (2022). aSGD: Stochastic gradient descent with adaptive batch size for every parameter. Mathematics, 10(6), 863. doi.org/10.3390/math10060863. ##
[88]. Kimanzi, R., Kimanga, P., Cherori, D., & Gikunda, P. K. (2024). Deep Learning algorithms used in intrusion detection systems--a review. arXiv preprint arXiv:2402.17020. doi.org/10.48550/arXiv.2402.17020
Focus to learn more. ##
[89]. Bank, D., Koenigstein, N., & Giryes, R. (2023). Autoencoders. Machine learning for data science handbook: data mining and knowledge discovery handbook, 353-374.
[90]. Brownlee, J. (2017). Deep learning for natural language processing: develop deep learning models for your natural language problems. Machine Learning Mastery. ##
[91]. Hospodarskyy, O., Martsenyuk, V., Kukharska, N., Hospodarskyy, A., & Sverstiuk, S. (2024). Understanding the Adam Optimization Algorithm in Machine Learning. CITI, 2024, 2nd.
[92]. Tanbhir, G., Shahriyar, M. F., Shahed, K., Chy, A. M. R., & Al Adnan, M. (2024, December). Hybrid machine learning model for detecting bangla smishing text using bert and character-level cnn. In 2024 13th International Conference on Electrical and Computer Engineering (ICECE) (pp. 57-62). IEEE. ##
[93]. Madni, S. H. H., A/L Pathmanatan, L., Faheem, M., Shahzad, H. M. F., & Shah, S. (2025). Exploring optimizer efficiency for facial expression recognition with convolutional neural networks. The Journal of Engineering, 2025(1), e70060. doi.org/10.1049/tje2.70060Digital Object Identifier (DOI). ##
[94]. Rikhari, S., & Lakshmi, K. M. (2025). Tumor extraction system using U-Net and Adamax optimizer. In Recent Trends in VLSI and Semiconductor Packaging (pp. 1-6). CRC Press. ##
[95.] Sriwahyuni, L., Nurdiati, S., Nugrahani, E. H., & Najib, M. K. (2025). Performance Of Machine Learning for Imputing Missing Daily Rainfall Data In East Java Under Multiple Satellite Data Models. Geographia Technica, 20(1). ##
[96]. Jirakitpuwapat, W., Dubey, P., Prasertsuk, N., Phanthong, C., Tritham, C., Tritham, C., Chandharakool, S., Tharathep, C. and Soontornpipit, P., (2024). Convergence property of Nesterov‐accelerated adaptive moment estimation with safety helmet detection and classification in smart industry application. Mathematical Methods in the Applied Sciences, 47(16), pp.12650-12667. doi.org/10.1002/mma.10174. ##
[97]. Saber, A., Emara, T., Elbedwehy, S., & Hassan, E. (2025). A novel approach for breast cancer detectionusing a Nesterov accelerated adam optimizer with an attention mechanism. Scientific Reports, 15(1), 27065. ##
[98]. Khosravi, M., & Zeraati, M. (2019). Predictive modeling of the length of prepared CNT by CVD through ANN-MPSO and GEP. Journal of Particle Science and Technology, 5(4), 145-159. doi: 10.22104/jpst.2020.3835.1156. ##
[99]. Iranzad, R., & Liu, X. (2025). A review of random forest-based feature selection methods for data science education and applications. International Journal of Data Science and Analytics, 20(2), 197-211.
[100]. Salman, H. A., Kalakech, A., & Steiti, A. (2024). Random forest algorithm overview. Babylonian Journal of Machine Learning, 2024, 69-79. doi.org/10.58496/BJML/2024/007. ##
[101]. Choudhury, A., Mondal, A., & Sarkar, S. (2024). Searches for the BSM scenarios at the LHC using decision tree-based machine learning algorithms: a comparative study and review of random forest, AdaBoost, XGBoost and LightGBM frameworks. The European Physical Journal Special Topics, 233(15), 2425-2463. ##
[102]. Haddouchi, M., & Berrado, A. (2024). A survey and taxonomy of methods interpreting random forest models. arXiv preprint arXiv:2407.12759. doi.org/10.48550/arXiv.2407.12759. ##
[103]. Malashin, I., Tynchenko, V., Gantimurov, A., Nelyub, V., & Borodulin, A. (2025). Boosting-based machine learning applications in polymer science: A review. Polymers, 17(4), 499. doi.org/10.3390/polym17040499. ##
[104]. Khatir, A., Capozucca, R., Khatir, S., Magagnini, E., Benaissa, B., & Cuong-Le, T. (2024). An efficient improved gradient boosting for strain prediction in near-surface mounted fiber-reinforced polymer strengthened reinforced concrete beam. Frontiers of structural and civil engineering, 18(8), 1148-1168. ##
[105]. Tepakhan, W., Srisintorn, W., Penglong, T., & Saelue, P. (2025). Machine learning approach for differentiating iron deficiency anemia and thalassemia using random forest and gradient boosting algorithms. Scientific Reports, 15(1), 16917. ##
[106]. Malashin, I., Tynchenko, V., Gantimurov, A., Nelyub, V., & Borodulin, A. (2025). Support vector machines in polymer science: A review. Polymers, 17(4), 491. doi.org/10.3390/polym17040491. ##
[107]. Nasim, F., Khatoon, S., Nasiruddin, I., Shahid, M., Urooj, S., & Bilal, B. (2025). Support-Vector-Regression-Based Intelligent Control Strategy for DFIG Wind Turbine Systems. Machines, 13(8), 687. doi.org/10.3390/machines13080687. ##
[108]. Tanveer, M., Rajani, T., Rastogi, R., Shao, Y. H., & Ganaie, M. A. (2024). Comprehensive review on twin support vector machines. Annals of Operations Research, 339(3), 1223-1268. ##
[109]. Haque, M.A., Nirob, J.H., Nahin, K.H., Jizat, N.M., Zakariya, M.A., Ananta, R.A., Abdulkawi, W.M., Aljaloud, K. and Al-Bawri, S.S., (2025). Machine learning-based technique for gain prediction of mm-wave miniaturized 5G MIMO slotted antenna array with high isolation characteristics. scientific reports, 15(1), 276. ##
[110]. Habibi, M., Bernard, S., Wang, J., & Fuge, M. (2025). Mean squared error may lead you astray when optimizing your inverse design methods. Journal of Mechanical Design, 147(2), 021701. doi.org/10.1115/1.4066102. ##
[111]. Wang, H., Ma, J., Wang, C., Sun, H., Du, S., Wen, H., & Li, Y. (2025). Machine learning-based performance prediction for energy storage medium-deep borehole ground source heat pump systems. Journal of Building Engineering, 99, 111471. doi: 10.1016/j.jobe.2024.111471. ##
[112]. Hodson, T. O. (2022). Root means square error (RMSE) or mean absolute error (MAE): When to use them or not. Geoscientific Model Development Discussions, 2022, 1-10.
[113]. Meyer, G. P. (2021). An alternative probabilistic interpretation of the huber loss. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition (pp. 5261-5269). ##
[114]. Routray, A., Mistry, K. D., Arya, S. R., & Chittibabu, B. (2025). Applied machine learning in wind speed prediction and loss minimization in unbalanced radial distribution system. Energy sources, Part A: recovery, utilization, and environmental effects, 47(1), 3790-3810. doi.org/10.1080/15567036.2020.1859010. ##
[115]. Chicco, D., Warrens, M. J., & Jurman, G. (2021). The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. Peerj computer science, 7, e623. ##