PRINCIPLES OF MACHINE ANALYSIS PROCEDURE ORGANIZATION BASED ON CONVOLUTIONAL NEURAL NETWORK ARCHITECTURE
DOI:
https://doi.org/10.32851/tnv-tech.2022.3.8Keywords:
convolutional neural networks, activation function, loss function, parameters initialization, weights regularization, iterative optimizer algorithms, target functionAbstract
The areas of application of machine analysis algorithms based on the convolutional neural network model are reviewed. The basic architecture of the convolutional neural network is determined: the organization of neural network layers, the principles of activation function selection and the scheme of loss function calculation. The formalization of the learning process of the convolutional neural network based on preprocessing of data, parameters initialization, weights regularization and iterative optimizer algorithms selection is carried out. A complex methodology is proposed, which provides an opportunity to organize, configure and optimize machine analysis algorithms based on the model of convolutional neural network in accordance with the target performance efficiency of neural network analysis and the load on the computing resource of the general complex hardware and software platform. The effectiveness of the solution of the assigned characteristics of the accuracy and adaptivity of the machine analysis system, as well as the load on the computing resource and the data processing time depends on the features of the neural network architecture organization and the approaches used in the process of CNN training. The author has defined the principles of development of integral and universal methodology of neural network algorithms based on CNN architecture, which are characterized by high accuracy of machine analysis in conditions of data processing time minimization under the existing restrictions on the computing resource of hardware and software platform. At the same time in this research it was carried out: the definition of the principles of construction of the convolutional neural network structure of deep learning; formalization of the mathematical apparatus of the convolutional procedure; formalization of the mathematical apparatus of the pooling procedure; the model of the organization of the adjustment and optimization of machine analysis algorithms, based on the convolutional neural network architecture, on the quantitative indicators level.
References
Zhao, Y., Ge, F., & Liu, T. (2018). Automatic recognition of holistic functional brain networks using iteratively optimized convolutional neural networks (IO-CNN) with weak label Initialization. Medical Image Analysis, 47, 111–126. https://doi.org/10.1016/j.media. 2018.04.002.
Li, B., Keikhosravi, A., Loeffler, A. G., & Eliceiri, K. W. (2021). Single image super-resolution for whole slide image using convolutional neural networks and self-supervised color normalization. Medical Image Analysis, 68, 101938. https://doi.org/10.1016/j.media. 2020.101938.
Maggiori, E., Tarabalka, Y., Charpiat, G., & Alliez, P. (2017). Convolutional neural networks for large-scale remote-sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(2), 645–657. https://doi.org/10.1109/tgrs.2016.2612821.
Ji, C., & Tang, H. (2020). Number of building stories estimation from monocular satellite image using a modified mask R-CNN. Remote Sensing, 12 (22), 3833. https://doi.org/10.3390/rs12223833.
Rasti, P., Uiboupin, T., Escalera, S., & Anbarjafari, G. (2016). Convolutional Neural Network Super resolution for face recognition in surveillance monitoring. Articulated Motion and Deformable Objects, 175–184. https://doi.org/10.1007/978-3-319-41778-3_18.
Kumar, S., & Singh, S. K. (2020). Occluded thermal face recognition using bag of CNN. IEEE Signal Processing Letters, 27, 975–979. https://doi.org/10.1109/lsp.2020.2996429.
Opitz, M., Diem, M., Fiel, S., Kleber, F., & Sablatnig, R. (2014). End-to-end text recognition using local ternary patterns, mser and deep convolutional nets. 2014 11th IAPR International Workshop on Document Analysis Systems. https://doi.org/10.1109/das.2014.29.
Wang, Z.-R., Du, J., & Wang, J.-M. (2020). Writer-aware CNN for parsimonious HMM‑based offline handwritten Chinese text recognition. Pattern Recognition, 100, 107102. https://doi.org/10.1016/j.patcog.2019.107102.
Verdhan, V. (2021). Image classification using LeNet. Computer Vision Using Deep Learning, 67–101. https://doi.org/10.1007/978-1-4842-6616-8_3.
Zhang, X., Pan, W., & Xiao, P. (2018). In-vivo skin capacitive image classification using AlexNet Convolution Neural Network. 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC). https://doi.org/10.1109/icivc.2018.8492860.
Kaddoun, S. S., Aberni, Y., Boubchir, L., Raddadi, M., & Daachi, B. (2021). Convolutional Neural Algorithm for palm vein recognition using ZFNet architecture. 2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART). https://doi.org/10.1109/biosmart54244.2021.9677799.
Chaudhari, S., Sardar, V., Rahul, D. S., Chandan, M., Shivakale, M. S., & Harini, K. R. (2021). Performance analysis of CNN, Alexnet and vggnet models for drought prediction using satellite images. 2021 Asian Conference on Innovation in Technology (ASIANCON). https://doi.org/10.1109/asiancon51346.2021.9545068.
Teymournezhad, K., Azgomi, H., & Asghari, A. (2022). Detection of counterfeit banknotes by security components based on image processing and GoogLeNet Deep Learning Network. Signal, Image and Video Processing. https://doi.org/10.1007/s11760-021-02104-z.
M., N. K. (2020). Breast cancer classification of image using modified ResNet. Journal of Advanced Research in Dynamical and Control Systems, 12(3), 134–140. https://doi.org/10.5373/jardcs/v12i3/20201175.
Li, G., Zhang, C., Lei, R., Zhang, X., Ye, Z., & Li, X. (2019). Hyperspectral remote sensing image classification using three-dimensional-squeeze-and-excitation-densenet (3D-Se-DenseNet). Remote Sensing Letters, 11(2), 195–203. https://doi.org/10.1080/2150704x. 2019.1697001
Wang, X., Ma, H., & Chen, X. (2016). Salient object detection via Fast R-CNN and low-level cues. 2016 IEEE International Conference on Image Processing (ICIP).
Adam, B., Zaman, F., Yassin, I., Abidin, H., & Rizman, Z. (2018). Performance evaluation of faster R-CNN on GPU for object detection. Journal of Fundamental and Applied Sciences, 9 (3S), 909.
Kızıloluk, S., & Sert, E. (2022). Hurricane-faster R-CNN-JS: Hurricane Detection with faster R-CNN using artificial Jellyfish Search (JS) optimizer. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-022-13156-9.
Wei, X., Xie, C., Wu, J., & Shen, C. (2018). Mask-CNN: Localizing parts and selecting descriptors for fine-grained bird species categorization. Pattern Recognition, 76, 704-714.
Yan, W., Liu, T., & Fu, Y. (2021). Yolo-tight: An efficient dynamic compression method for Yolo Object Detection Networks. 2021 13th International Conference on Machine Learning and Computing. https://doi.org/10.1145/3457682.3457740.
Du, J. (2018). Understanding of Object Detection Based on CNN Family and YOLO. Journal of Physics: Conference Series, 1004, 012029.
Qin, P. (2019). Fully convolutional-based dense network for lung nodule image retrieval algorithm. International Journal of Performability Engineering. https://doi.org/10.23940/ ijpe.19.01.p33.326336.
Cai, Y., & Li, Q. (2021). DeepLab network for Meteorological Trough Line Recognition. 2021 4th International Conference on Sensors, Signal and Image Processing. https://doi.org/10.1145/3502814.3502820.
Mukherjee, A., Chakraborty, S., & Saha, S. K. (2019). Detection of loop closure in slam: A DeconvNet based approach. Applied Soft Computing, 80, 650–656. https://doi.org/10.1016/j.asoc.2019.04.041.
Saood, A., & Hatem, I. (2020). Covid-19 lung CT image segmentation using deep learning methods: UNET vs. segnet. https://doi.org/10.21203/rs.3.rs-56882/v2.
Wang, P., Xiong, L., & Dan, B. (2021). Surface vortex image segmentation in KR desulfurization based on improved SegNet model. 2021 China Automation Congress (CAC). https://doi.org/10.1109/cac53003.2021.9728231.
Son, S.-B., Jung, J.-U., Oh, H.-S., & Jung, Y.-chul. (2020). DeepMask: Face masking system using deep neural networks on real-time streaming. Journal of Institute of Control, Robotics and Systems, 26 (6), 423–428. https://doi.org/10.5302/j.icros.2020.20.0029.
Trullo, R., Petitjean, C., Ruan, S., Dubray, B., Nie, D., & Shen, D. (2017). Segmentation of organs at risk in thoracic CT images using a SharpMask architecture and conditional random fields. 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). https://doi.org/10.1109/isbi.2017.7950685.
Hu, X., & Yang, H. (2020). Dru-Net: A novel U-Net for biomedical image segmentation. IET Image Processing, 14(1), 192–200. https://doi.org/10.1049/ietipr.2019.0025.
Wang, K., Liew, J. H., Zou, Y., Zhou, D., & Feng, J. (2019). Panet: Few-shot image semantic segmentation with prototype alignment. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv.2019.00929.
Chen, X., Girshick, R., He, K., & Dollar, P. (2019). TensorMask: A foundation for dense object segmentation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv.2019.00215.
Ghosh A., et al., (2020) Fundamental Concepts of Convolutional Neural Network. En Balas V., Kumar R., Srivastava R. (eds) Recent Trends and Advances in Artificial Intelligence and Internet of Things. Intelligent Systems Reference Library. Springer. https://doi.org/10.1007/978-3-030-32644-9_36.
Qin, Z., et al., (2018). How convolutional neural networks see the world. A survey of convolutional neural network visualization methods. Mathematical Foundations of Computing, 1(2): 149-180. https://doi.org/10.3934/mfc.2018008.
