PERFORMANCE EVALUATION OF NEURAL NETWORKS FOR ANOMALY DETECTION IN BUSINESS PROCESSES
Keywords:
BPM, Anomaly Detection, DNN, GNN, CNN, RNN, LSTM, Autoencoder, TransformersAbstract
The of this study is to evaluate the effectiveness of various deep neural network (DNN) architectures for anomaly detection in business processes, which is a critical task in the context of digital transformation and automation of managerial decisions. The study examines and compares different architectures: graph neural networks (GNN) for detecting structural anomalies, recurrent neural networks (LSTM, RNN) and transformers (Transformers) for time series analysis and forecasting, as well as autoencoders (Autoencoders) for processing attribute-based data. An adaptive approach is proposed, integrating the advantages of different models depending on the type and characteristics of a business process. The research is based on a large dataset extracted from a BPMS system, including enriched process graphs with business parameters. Four main types of anomalies are considered: Missing Steps, Duplicate Steps, Wrong Route, and Abnormal Duration.The experimental part of the study evaluates model performance using key metrics, including Precision, Recall, F1-score, AUC-ROC, AUPRC, ADR (Anomaly Detection Rate), FAR (False Alarm Rate), and FNR (False Negative Rate), as well as training time and a confusion matrix for a detailed analysis of predicted class distributions. The results indicate that Transformers achieve the highest accuracy in detecting complex event sequences, GNNs excel in identifying structural anomalies, and Autoencoders perform effectively when working with attribute-based datasets.Recommendations are provided for selecting an appropriate architecture based on the specific characteristics of a business process and computational resource constraints. The findings can be applied to optimize process monitoring, automate anomaly detection, and enhance the efficiency of business process management in a rapidly evolving digital environment.
References
Aalst W. Van der Process Mining. Berlin, Heidelberg : Springer Berlin Heidelberg, 2016. 1-467 p. DOI:10.1007/978-3-662-49851-4.
Al-Jebrni A., Cai H., Jiang L. Predicting the Next Process Event Using Convolutional Neural Networks. 2018 IEEE International Conference on Progress in Informatics and Computing (PIC)(12.2018). IEEE, 2018. DOI:10.1109/PIC.2018.8706282. P. 332–338.
Alizadeh M., Leoni M. de, Zannone N. History-Based Construction of Alignments for Conformance Checking: Formalization and Implementation. 2015. P. 58–78. DOI:10.1007/978-3-319-27243-6_3.
Ariyaluran Habeeb R. A., Nasaruddin F., Gani A. et al. Real-time big data processing for anomaly detection: A Survey. International Journal of Information Management. Vol. 45, 01.04.2019. P. 289–307. DOI:10.1016/j.ijinfomgt.2018.08.006.
Bezerra F., Wainer J. Anomaly Detection Algorithms In Business Process Logs. Proceedings of the Tenth International Conference on Enterprise Information Systems(2008). SciTePress – Science and and Technology Publications, 2008. DOI:10.5220/0001674700110018. P. 11–18.
Cai J., Zhang Y., Fan J. Self-Discriminative Modeling for Anomalous Graph Detection. 09.10.2023. URL: http://arxiv.org/abs/2310.06261
Chen J., Sathe S., Aggarwal C. et al. Outlier Detection with Autoencoder Ensembles. Proceedings of the 2017 SIAM International Conference on Data Mining. Philadelphia, PA : Society for Industrial and Applied Mathematics, 2017. P. 90–98. DOI :10.1137/1.9781611974973.11.
Darmawan H., Sarno R., Ahmadiyah A. S. et al. Anomaly detection based on control-flow pattern of Parallel Business Processes. Telkomnika (Telecommunication Computing Electronics and Control). Vol. 16, Issue 6. P. 2809–2816. DOI:10.12928/ TELKOMNIKA.v16i6.10568.
Eberle W., Holder L. Discovering Structural Anomalies in Graph-Based Data. Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007) (10.2007). IEEE, 2007. DOI:10.1109/ICDMW.2007.91. P. 393–398.
Galanti R., Coma-Puig B., Leoni M. de et al. Explainable Predictive Process Monitoring. 2020 2nd International Conference on Process Mining (ICPM)(10.2020). IEEE, 2020. DOI:10.1109/ICPM49681.2020.00012. P. 1–8.
Gong Z., Fang X., Wu P. Anomaly Correction of Business Processes Using Transformer Autoencoder. 2024. P. 1–16. Also available online, URL: http://arxiv.org/ abs/2404.10211
Guan W., Cao J., Yao Y. et al. COMB: Interconnected Transformers-Based Autoencoder for Multi-Perspective Business Process Anomaly Detection. 2024 IEEE International Conference on Web Services (ICWS)(07.07.2024). IEEE, 2024. DOI:10.1109/ICWS62655.2024.00134. P. 1115–1124.
Ha J., Kambe M., Pe J. Data Mining: Concepts and Techniques. 2011. 1-703 p. DOI:10.1016/C2009-0-61819-5.
Ioannidis V. N., Berberidis D., Giannakis G. B. GraphSAC: Detecting anomalies in large-scale graphs. 21.10.2019. P. 1–13. Also available online, URL: http://arxiv.org/ abs/1910.09589
Ji Y., Wang J., Li S. et al. An Anomaly Event Detection Method Based on GNN Algorithm for Multi-data Sources. Proceedings of the 3rd ACM International Symposium on Blockchain and Secure Critical Infrastructure(24.05.2021). New York, NY, USA : ACM, 2021. DOI:10.1145/3457337.3457846. P. 91–96.
Landauer M., Onder S., Skopik F. et al. Deep learning for anomaly detection in log data: A survey. Machine Learning with Applications. Vol. 12, Issue March. P. 100470. DOI:10.1016/j.mlwa.2023.100470.
Ni W., Zhao G., Liu T. et al. Predictive Business Process Monitoring Approach Based on Hierarchical Transformer. Electronics. Vol. 12, Issue 6. P. 1273. DOI:10.3390/ electronics12061273.
Nolle T., Luettgen S., Seeliger A. et al. Analyzing business process anomalies using autoencoders. Machine Learning. Vol. 107, Issue 11. P. 1875–1893. DOI:10.1007/ s10994-018-5702-8.
Nolle T., Luettgen S., Seeliger A. et al. BINet: Multi-perspective business process anomaly classification. Information Systems. Vol. 103, 01.2022. P. 101458. DOI:10.1016/j.is.2019.101458.
Park G., Song M. Predicting performances in business processes using deep neural networks. Decision Support Systems. Vol. 129, Issue November 2019. P. 113191. DOI:10.1016/j.dss.2019.113191.
Vidacic T., Strahonja V. Taxonomy of Anomalies in Business Process Models. Information System Development. Cham : Springer International Publishing, 2014. P. 283–294. DOI:10.1007/978-3-319-07215-9_23.
Wang Z., Chen Z., Ni J. et al. Multi-Scale One-Class Recurrent Neural Networks for Discrete Event Sequence Anomaly Detection. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining(14.08.2021). New York, NY, USA : ACM, 2021. DOI:10.1145/3447548.3467125. P. 3726–3734.
Weinzierl S., Zilker S., Dunzer S. et al. Machine learning in business process management: A systematic literature review. Expert Systems with Applications. Vol. 253, 01.11.2024. P. 124181. DOI:10.1016/j.eswa.2024.124181.
Ye S., Wang J., Ali S. et al. Recovering Latent Data Flow from Business Process Model Automatically. Wireless Communications and Mobile Computing. ed. Y. Wang. Vol. 2022, 20.06.2022. P. 1–11. DOI:10.1155/2022/7579515.
Zhao D., Zhou Z. Anomaly and Interestingness Detection in Timed Hierarchical Business Processes. IEEE Transactions on Engineering Management. Vol. 71, 2024. P. 12619–12634. DOI:10.1109/TEM.2022.3182413.
