The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem

Alsaç Ali; Yenisey Mehmet; Ganiz Murat; Dağtekin Mustafa; Ulusinan Taner

doi:https://dx.doi.org/10.26650/acin.1394019

Research Article

DOI :10.26650/acin.1394019 IUP :10.26650/acin.1394019 Full Text (PDF)

The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem

Ali Alsaç, Mehmet Mutlu Yenisey, Murat Can Ganiz, Mustafa Dağtekin, Taner Ulusinan

Designing a prediction method with machine learning algorithms and increasing the prediction success is one of the most important research areas and aims of today. Models designed using classification algorithms are frequently used especially in problem types that require prediction. In this study, real life data is used to answer the question of which problem type should be included in the Information Technology Service Management (ITSM) system. An important step in the search for a solution is to examine the dataset with regularization methods. Experimental results have been obtained to establish the overfitting or underfitting balance of the dataset with L1 and L2 regularization methods. While the Root-Mean-Square Error (RMSE) value was approximately 0.13 in the regression model without regularization, this value was found to be approximately 0.083 after L1 regularization.With the regularized dataset, new results were obtained using Artificial Neural Network (ANN), Logistic Regression (LR), Support Vector Machine (SVM) classifier algorithms. SVM algorithm was the most successful model with a performance of approximately 0.73. It is followed by LR and ANN respectively. Accuracy, Precision, Recall and F1Score were used as evaluation metrics. It is seen that the use of regularization methods, especially in the preparation of real-life data for use in machine learning or other artificial intelligence research, will contribute to increasing the success level of the model.

Keywords: IT service management, regularization, prediction, classification

DOI :10.26650/acin.1394019 IUP :10.26650/acin.1394019 Full Text (PDF)

Sorun Türü Tahmini Probleminde Düzenlileştirme Yönteminin Model Başarısı Üzerindeki Etkisi

Ali Alsaç, Mehmet Mutlu Yenisey, Murat Can Ganiz, Mustafa Dağtekin, Taner Ulusinan

Matematik düzleminde bir tahmin yöntemi tasarlamak ve başarılı sonuçlarından faydalanmak günümüzün önemli araştırma alanlarından ve amaçlarından biri olarak öne çıkmaktadır. Sınıflandırma algoritmaları kullanılarak tasarlanan modeller özellikle tahmin gerektiren problem türlerinde sıklıkla kullanılmaktadır. Çalışmada gerçek hayat verileri kullanılarak bir gerçek hayat problemi olan müşteriden gelen çözüm talebinin Bilgi Teknolojisi Hizmet Yönetimi (BTHY) sistemi içinde hangi sorun tipine dahil edilmesi gerektiği sorusuna cevap aranmaktadır. Çözüm arayışının önemli bir aşamasında veri kümesinin Regülarizasyon yöntemleri ile incelenmesi yer almaktadır. L1 ve L2 regülarizasyon yöntemleri ile veri kümesinin overfitting ya daunderfitting dengesinin kurulması için deneysel sonuçlar alınmıştır. Regülarizasyon uygulanmamış regresyon modelinde Kök Ortalama Kare Hatası (RMSE) değeri yaklaşık olarak 0,13 iken L1 regülarizasyonu sonucunda bu değer yaklaşık 0,083 olarak bulunmuştur. Düzenlileştirilmiş veri kümesi ile Yapay Sinir Ağları (YSA), Lojistik Regresyon (LR), Destek Vektör Makinaları (DVM) sınıflandırıcı algoritmaları kullanılarak yeni sonuçlar elde edilmiştir. DVM algoritması yaklaşık 0,73 başarım sonucu ile en başarılı model olmuştur. Sırasıyla LR ve YSA takip etmektedir. Değerlendirme metrikleri olarak Accuracy, Precision, Recall ve F1Score kullanılmıştır. Özellikle gerçek hayat verilerinin makina öğrenmesi ya da diğer yapay zeka araştırmalarında kullanımı için hazırlanması aşamasında Regülarizasyon yöntemlerinden faydalanmanın modelin başarı düzeyinin artmasında katkısı olacağı görülmektedir.

Keywords: Bilgi işlem servis yönetimi, regülarizasyon, tahmin, sınıflandırma

References

ALAN, A., & KARABATAK, M. (2020). Veri Seti - Sınıflandırma İlişkisinde Performansa Etki Eden Faktörlerin Değerlendirilmesi. Fırat Üniversitesi Mühendislik Bilimleri Dergisi, 32(2). https://doi.org/10.35234/fumbd.738007 google scholar
Anderson D, M. G. (1992). Artificial Neural Networks Technology. Kaman Sciences Corporation, 258(6). google scholar
Aran, O., Yildiz, O. T., & Alpaydin, E. (2009). An incremental framework based on cross-validation for estimating the architecture of a multilayer perceptron. International Journal of Pattern Recognition and Artificial Intelligence, 23(2). https://doi.org/10.1142/S0218001409007132 google scholar
ARSLAN, H., ÜNEŞ, F., DEMİRCİ, M., TAŞAR, B., & YILMAZ, A. (2020). Keban Baraj Gölü Seviye Değişiminin ANFIS ve Destek Vektör Makineleri ile Tahmini. Osmaniye Korkut Ata Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 3(2). https://doi.org/10.47495/okufbed.748018 google scholar
Bharambe, Prof. P., Bagul, B., Dandekar, S., & Ingle, P. (2022). Used Car Price Prediction using Different Machine Learning Algorithms. International Journal for Research in Applied Science and Engineering Technology, 10(4). https://doi.org/10.22214/yraset.2022.41300 google scholar
Bhattacharya, P., Neamtiu, I., & Shelton, C. R. (2012). Automated, highly-accurate, bug assignment using machine learning and tossing graphs. Journal of Systems and Software, 85(10). https://doi.org/10.1016/j.jss.2012.04.053 google scholar
ÇELİK, E., DAL, D., & AYDİN, T. (2021). Duygu Analizi İçin Veri Madenciliği Sınıflandırma Algoritmalarının Karşılaştırılması. European Journal of Science and Technology. https://doi.org/10.31590/ejosat.905259 google scholar
Cook, D., Dixon, P., Duckworth, W. M., Kaiser, M. S., Koehler, K., Meeker, W. Q., & Stephenson, W. R. (2001). Binary Response and Logistic Regression Analysis. Project Beyond Traditional Statistical Methods, Ml. google scholar
Cortes, C., & Vapnik, V. (1995). Support-Vector Networks. Machine Learning, 20(3). https://doi.org/10.1023/A:1022627411411 google scholar
Dangeti, P. (2017). Statistics for Machine Learning: Techniques for exploring supervised, unsupervised, and reinforcement learning models with Python and R. In Packt Publishing. google scholar
Deloitte, & TUBISAD. (2022). Bilgi ve İletişim Teknolojileri Sektörü 2021 Pazar Verileri. google scholar
Doğan, C. (2021). İstatistiksel ve Makine Öğrenme ile Derin Sinir Ağlarında Hiper-Parametre Seçimi İçin Melez Yaklaşım [Yüksek Lisans]. Hacettepe Üniversitesi. google scholar
Domingos, P. (2000). A Unified Bias-Variance Decomposition. Aaai/Iaai. google scholar
Emmert-Streib, F., & Dehmer, M. (2019). High-Dimensional LASSO-Based Computational Regression Models: Regularization, Shrinkage, and Selection. In Machine Learning and Knowledge Extraction (Vol. 1, Issue 1). https://doi.org/10.3390/make1010021 google scholar
Friedrich, S., Groll, A., Ickstadt, K., Kneib, T., Pauly, M., Rahnenführer, J., & Friede, T. (2023). Regularization approaches in clin-ical biostatistics: A review of methods and their applications. In Statistical Methods in Medical Research (Vol. 32, Issue 2). https://doi.org/10.1177/09622802221133557 google scholar
Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural Networks and the Bias/Variance Dilemma. Neural Computation, 4(1). https://doi.org/10.1162/neco.1992.4.1.1 google scholar
Golam Kibria, B. M., & Banik, S. (2016). Some ridge regression estimators and their performances. Journal of Modern Applied Statistical Methods, 15(1). https://doi.org/10.22237/jmasm/1462075860 google scholar
Goldberg, N., & Eckstein, J. (2012). Sparse weighted voting classifier selection and its linear programming relaxations. Information Processing Letters, 112(12). https://doi.org/10.1016/j.ipl.2012.03.004 google scholar
Ha, J., Kambe, M., & Pe, J. (2011). Data Mining: Concepts and Techniques. In Data Mining: Concepts and Techniques. https://doi.org/10.1016/C2009-0-61819-5 google scholar
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate Data Analysis. In Vectors. https://doi.org/10.1016/j.ypharm.2011.02.019 google scholar
Hautamaki, V., Kinnunen, T., Sedlak, F., Lee, K. A., Ma, B., & Li, H. (2013). Sparse classifier fusion for speaker verification. IEEE Transactions on Audio, Speech and Language Processing, 21(8). https://doi.org/10.1109/TASL.2013.2256895 google scholar
Helming, J., Arndt, H., Hodaie, Z., Koegel, M., & Narayan, N. (2011). Automatic Assignment of Work Items. Communications in Computer and Information Science, 230. https://doi.org/10.1007/978-3-642-23391-3_17 google scholar
Jonsson, L., Borg, M., Broman, D., Sandahl, K., Eldh, S., & Runeson, P. (2016). Automated bug assignment: Ensemble-based machine learning in large scale industrial contexts. Empirical Software Engineering, 21(4). https://doi.org/10.1007/s10664-015-9401-9 google scholar
Koçoğlu, F. Ö., & Esnaf, Ş. (2022). Machine Learning Approach and Model Performance Evaluation for Tele-Marketing Success Classification. International Journal of Business Analytics, 9(5). https://doi.org/10.4018/yban.298014 google scholar
Koçoğlu, F. Ö., & Özcan, T. (2022). A grid search optimized extreme learning machine approach for customer churn prediction. Journal of Engineering Research. google scholar
Kotsilieris, T., Anagnostopoulos, I., & Livieris, I. E. (2022). Special Issue: Regularization Techniques for Machine Learning and Their Appli-cations. In Electronics (Switzerland) (Vol. 11, Issue 4). https://doi.org/10.3390/electronics11040521 google scholar
Li, N., & Zhou, Z. H. (2009). Selective ensemble under regularization framework. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 5519 LNCS. https://doi.org/10.1007/978-3-642-02326-2_30 google scholar
Mantovani, R. G., Horvath, T., Cerri, R., Vanschoren, J., & De Carvalho, A. C. P. L. F. (2017). Hyper-Parameter Tuning of a Decision Tree Induction Algorithm. Proceedings - 2016 5th Brazilian Conference on Intelligent Systems, BRACIS 2016. https://doi.org/10.1109/BRACIS.2016.018 google scholar
Mao, S., Xiong, L., Jiao, L. C., Zhang, S., & Chen, B. (2013). Weighted ensemble based on 0-1 matrix decomposition. Electronics Letters, 49(2). https://doi.org/10.1049/el.2012.3528 google scholar
Muller, A. C., & Guido, S. (2017). Introduction to Machine Learning with Python: a guide for data scientist. In O’Reilly Media, Inc. google scholar
Orynbassar, A., Sapazhanov, Y., Kadyrov, S., & Lyublinskaya, I. (2022). Application of ROC Curve Analysis for Predicting Students’ Passing Grade in a Course Based on Prerequisite Grades. Mathematics, 10(12). https://doi.org/10.3390/math10122084 google scholar
ÖZBİLGİN, F., & KURNAZ, Ç. (2023). Koroner Arter Hastalığının İris Görüntülerinden Yerel İkili Örüntüler ve Yapay Sinir Ağı Kullanılarak Tahmini. Karadeniz Fen Bilimleri Dergisi, 13(2). https://doi.org/10.31466/kfbd.1266996 google scholar
Özgür, A., Nar, F., & Erdem, H. (2018). Sparsity-driven weighted ensemble classifier. International Journal of Computational Intelligence Systems, 11 (1). https://doi.Org/10.2991/ijcis.11.1.73 google scholar
Paper, D. (2019). Hands-on Scikit-Learn for Machine Learning Applications: Data Science Fundamentals with Python. In Hands-on Scikit-Learn for Machine Learning Applications: Data Science Fundamentals with Python. https://doi.org/10.1007/978-1-4842-5373-1 google scholar
Sahoo, K., Samal, A. K., Pramanik, J., & Pani, S. K. (2019). Exploratory data analysis using python. International Journal of Innovative Technology and Exploring Engineering, 8(12), 4727-4735. https://doi.org/10.35940/jitee.L3591.1081219 google scholar
Şen, M. U., & Erdogan, H. (2013). Linear classifier combination and selection using group sparse regularization and hinge loss. Pattern Recognition Letters, 34(3). https://doi.org/10.1016/j.patrec.2012.10.008 google scholar
ŞENEL, S., & ALATLI, B. (2014). Lojistik Regresyon Analizinin Kullanıldığı Makaleler Üzerine Bir İnceleme. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 5(1). https://doi.org/10.21031/epod.67169 google scholar
Sinha, K., Uddin, Z., Kawsar, H. I., Islam, S., Deen, M. J., & Howlader, M.M.R. (2023). Analyzing chronic disease biomarkers using electrochem-ical sensors and artificial neural networks. In TrAC - Trends in Analytical Chemistry (Vol. 158). https://doi.org/10.1016/j.trac.2022.116861 google scholar
Şipal, B., Ormancı, B. B., & Altınel, A. B. (2022). KELİME ANLAM BULANIKLIĞINI GİDERMEK İÇİN DİFÜZYON REGÜLARİZASYON VE NORMALİZASYON TEKNİKLERİNİN KULLANILMASI. In MÜHENDİSLİK ALANINDA ULUSLARARASI ARAŞTIRMALAR VI (pp. 75-85). google scholar
Tanyildizi, E., & Demirtas, F. (2019). Hiper Parametre Optimizasyonu Hyper Parameter Optimization. 1st International Infor-matics and Software Engineering Conference: Innovative Technologies for Digital Transformation, IISEC 2019 - Proceedings. https://doi.org/10.1109/UBMYK48245.2019.8965609 google scholar
TAZEGÜL, A., YAZARKAN, H., & YERDELEN, C. (2016). İşletmelerin Finansal Başarılı ve Başarısız Olma Durumlarının Veri Madenciliği ve Lojistik Regresyon Analizi İle Tahmin Edilebilirliği. Ege Akademik Bakis (Ege Academic Review), 16(1). https://doi.org/10.21121/eab.2016119960 google scholar
Tian, Y., & Zhang, Y. (2022). A comprehensive survey on regularization strategies in machine learning. In Information Fusion (Vol. 80). https://doi.org/10.1016/j.inffus.2021.11.005 google scholar
Tinoco, S. L. J. L., Santos, H. G., Menotti, D., Santos, A. B., & Dos Santos, J. A. (2013). Ensemble of classifiers for remote sensed hyperspectral land cover analysis: An approach based on Linear Programming and Weighted Linear Combination. International Geoscience and Remote Sensing Symposium (IGARSS). https://doi.org/10.1109/IGARSS.2013.6723730 google scholar
Witten, I. H., Frank, E., Hall, M. A., & Pal, C. J. (2016). Data Mining: Practical Machine Learning Tools and Techniques. In Data Mining: Practical Machine Learning Tools and Techniques. google scholar
YENİSU, E. (2021). Ekonomiyi Harekete Geçiren Kilit Sektörler Nelerdir? Türkiye Üzerine Bir Girdi-Çıktı Analizi. İzmir İktisat Dergisi, 36(4). https://doi.org/10.24988/je.721302 google scholar
YETGINLER, B., & ATACAK, İ. (2020). Sentiment Analyses on Movie Reviews using Machine Learning-Based Methods. Artificial Intelligence Studies, 3(2). https://doi.org/10.30855/ais.2020.03.02.01 google scholar
Yildiz, M., Alsac, A., Ulusinan, T., Ganiz, M. C., & Yenisey, M. M. (2022). IT Support Ticket Completion Time Prediction. Proceedings - 7th International Conference on Computer Science and Engineering, UBMK 2022. https://doi.org/10.1109/UBMK55850.2022.9919591 google scholar
Yin, X. C., Huang, K., Hao, H. W., Iqbal, K., & Wang, Z. Bin. (2012). Classifier ensemble using a heuristic learning with sparsity and diversity. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7664 LNCS(PART 2). https://doi.org/10.1007/978-3-642-34481-7_13 google scholar
Yoon, B.L. (1989). Artificial neural network technology. ACM SIGSMALL/PC Notes, 15(3), 3-16. https://doi.org/10.1145/74657.74658 google scholar
Zhang, L., & Zhou, W. Da. (2011). Sparse ensembles using weighted combination methods based on linear programming. Pattern Recognition, 44(1). https://doi.org/10.1016/j.patcog.2010.07.021 google scholar
Zibran, M.F. (2016). On the effectiveness of labeled latent dirichlet allocation in automatic bug-report categorization. Proceedings - International Conference on Software Engineering. https://doi.org/10.1145/2889160.2892646 google scholar

Citations

Copy and paste a formatted citation or use one of the options to export in your chosen format

EXPORT

APA

Alsaç, A., Yenisey, M.M., Ganiz, M.C., Dağtekin, M., & Ulusinan, T. (2023). The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem. Acta Infologica, 7(2), 360-383. https://doi.org/10.26650/acin.1394019

AMA

Alsaç A, Yenisey M M, Ganiz M C, Dağtekin M, Ulusinan T. The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem. Acta Infologica. 2023;7(2):360-383. https://doi.org/10.26650/acin.1394019

ABNT

Alsaç, A.; Yenisey, M.M.; Ganiz, M.C.; Dağtekin, M.; Ulusinan, T. The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem. Acta Infologica, [Publisher Location], v. 7, n. 2, p. 360-383, 2023.

Chicago: Author-Date Style

Alsaç, Ali, and Mehmet Mutlu Yenisey and Murat Can Ganiz and Mustafa Dağtekin and Taner Ulusinan. 2023. “The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem.” Acta Infologica 7, no. 2: 360-383. https://doi.org/10.26650/acin.1394019

Chicago: Humanities Style

Alsaç, Ali, and Mehmet Mutlu Yenisey and Murat Can Ganiz and Mustafa Dağtekin and Taner Ulusinan. “The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem.” Acta Infologica 7, no. 2 (May. 2024): 360-383. https://doi.org/10.26650/acin.1394019

Harvard: Australian Style

Alsaç, A & Yenisey, MM & Ganiz, MC & Dağtekin, M & Ulusinan, T 2023, 'The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem', Acta Infologica, vol. 7, no. 2, pp. 360-383, viewed 2 May. 2024, https://doi.org/10.26650/acin.1394019

Harvard: Author-Date Style

Alsaç, A. and Yenisey, M.M. and Ganiz, M.C. and Dağtekin, M. and Ulusinan, T. (2023) ‘The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem’, Acta Infologica, 7(2), pp. 360-383. https://doi.org/10.26650/acin.1394019 (2 May. 2024).

MLA

Alsaç, Ali, and Mehmet Mutlu Yenisey and Murat Can Ganiz and Mustafa Dağtekin and Taner Ulusinan. “The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem.” Acta Infologica, vol. 7, no. 2, 2023, pp. 360-383. [Database Container], https://doi.org/10.26650/acin.1394019

Vancouver

Alsaç A, Yenisey MM, Ganiz MC, Dağtekin M, Ulusinan T. The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem. Acta Infologica [Internet]. 2 May. 2024 [cited 2 May. 2024];7(2):360-383. Available from: https://doi.org/10.26650/acin.1394019 doi: 10.26650/acin.1394019

ISNAD

Alsaç, Ali - Yenisey, MehmetMutlu - Ganiz, MuratCan - Dağtekin, Mustafa - Ulusinan, Taner. “The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem”. Acta Infologica 7/2 (May. 2024): 360-383. https://doi.org/10.26650/acin.1394019

Volume 7, Issue 22023, P. 360-383

TIMELINE

Submitted	22.11.2023
Accepted	01.12.2023
Published Online	14.12.2023

LICENCE

Attribution-NonCommercial (CC BY-NC)

This license lets others remix, tweak, and build upon your work non-commercially, and although their new works must also acknowledge you and be non-commercial, they don’t have to license their derivative works on the same terms.

Acta Infologica

Research Article

The Efficiency of Regularization Method on Model Success in Issue Type Prediction Problem

Sorun Türü Tahmini Probleminde Düzenlileştirme Yönteminin Model Başarısı Üzerindeki Etkisi

PDF View

References

Citations

EXPORT

APA

AMA

ABNT

Chicago: Author-Date Style

Chicago: Humanities Style

Harvard: Australian Style

Harvard: Author-Date Style

MLA

Vancouver

ISNAD

TIMELINE

LICENCE

SHARE