MACHINE LEARNING ALGORITHMS OPTIMIZATION METHODOLOGY FOR EMBEDDED CYBER-PHYSICAL SYSTEMS
DOI:
https://doi.org/10.32782/tnv-tech.2024.1.2Keywords:
embedded systems, machine learning, optimization, energy efficiency, neural network, cyber-physical systemAbstract
The development of digital transformation and artificial intelligence is accompanied by a growing need to implement embedded cyber-physical systems that use machine learning algorithms in critical infrastructure such as energy, transportation, manufacturing, and healthcare. However, this process is complicated by the need to develop optimization methods for machine learning algorithms that would ensure the efficient operation of embedded systems with limited computing resources and ensure resilience in critical environments. This poses the challenge for developers to adapt and optimize machine learning algorithms to the limitations associated with computing power, memory space, decision accuracy, and power consumption. That is why we have developed and evaluated a methodology for optimizing machine learning algorithms for use in embedded cyber-physical systems. The main focus of the study was on convolutional neural networks used in image recognition tasks. Using convolutional neural networks trained on the Street View House Numbers (SVHN) dataset, the study demonstrates how models can effectively perform real-time digit classification and recognition tasks while optimizing the use of limited resources in embedded systems. To evaluate the effectiveness of the proposed methodology, criteria such as execution time to ensure the accuracy of measurements in cyber-physical systems, power consumption of embedded systems, and minimization of disk space and RAM required to run the models were taken into account. In the process of researching the methodology, optimization methods such as weight reduction and quantization were applied, the combination of which allows to reduce the model size and power consumption without significant loss of accuracy. The optimized neural network models were tested on a typical ESP32 embedded system, demonstrating the ability to operate autonomously and recognize objects in real time. The study included data preparation, model development and training, application of different optimization options, and measurement of their impact on the system’s final metrics. The use of the TensorFlow Lite framework allowed us to adapt the models for effective use in embedded systems. The results of the study confirmed the effectiveness of the proposed optimization methodology, which ensures a decrease in model accuracy by only 2.1%, while increasing execution speed by 30% and significantly reducing power consumption.
References
A Cyber-Physical Systems Paper Survey About the Concept, Architecture and Challenges for the Deployment within the Concept of Industry 4.0 / M. Melicher et al. Research Papers Faculty of Materials Science and Technology Slovak University of Technology. 2019. Vol. 27, no. 45. P. 49–54.
Cyber-Physical Power System (CPPS): A Review on Modeling, Simulation, and Analysis With Cyber Security Applications / R. V. Yohanandhan et al. IEEE Access. 2020. Vol. 8. P. 151019–151064.
Conrad C., Al-Rubaye S., Tsourdos A. Intelligent Embedded Systems Platform for Vehicular Cyber-Physical Systems. Electronics. 2023. Vol. 12, no. 13. P. 2908.
Batzolis E., Vrochidou E., Papakostas G. A. Machine Learning in Embedded Systems: Limitations, Solutions and Future Challenges. 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 8–11 March 2023. 2023.
Wiedemann S., Muller K.-R., Samek W. Compact and Computationally Efficient Representation of Deep Neural Networks. IEEE Transactions on Neural Networks and Learning Systems. 2020. Vol. 31, no. 3. P. 772–785.
Tung F., Mori G. Deep Neural Network Compression by In-Parallel Pruning-Quantization. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2020. Vol. 42, no. 3. P. 568–579.
Compressing by Learning in a Low-Rank and Sparse Decomposition Form / K. Guo et al. IEEE Access. 2019. Vol. 7. P. 150823–150832.
LightweightNet: Toward fast and lightweight convolutional neural networks via architecture distillation / T.-B. Xu et al. Pattern Recognition. 2019. Vol. 88. P. 272–284.
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices / X. Zhang et al. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, 18–23 June 2018. 2018.
EIE: Efficient Inference Engine on Compressed Deep Neural Network / S. Han et al. Computer Architecture News. 2016. Vol. 44(3). P. 243–254.
Yang H., Yao H. Street View House Number Identification Based on Deep Learning. International Journal of Advanced Network, Monitoring and Controls. 2019. Vol. 4, no. 3. P. 47–52.
