APPROACHES TO REDUCING THE CARBON FOOTPRINT IN TRAINING LARGE ML MODELS
Keywords:
machine learning, energy efficiency, carbon footprint, renewable energy sources, sustainable development, data centres, algorithm optimisationAbstract
The study’s relevance is due to a significant increase in the use of machine learning, accompanied by an increase in the energy consumption required to train large models. This creates a significant carbon footprint that threatens environmental sustainability. To reduce the environmental impact of technological development, the problem requires the integration of energy-efficient approaches and the introduction of renewable energy sources.The study aims to develop effective approaches to optimising computing processes, with the aim of reducing energy consumption during the training of machine learning models.The study uses system analysis to assess current approaches to reducing the carbon footprint.It also compares technological solutions and summarises the results to develop comprehensive recommendations.It has been established that the main areas of optimisation are the introduction of energy- efficient algorithms, in particular Pruning and Quantisation, using specialised equipment (TPU, GPU, ASIC), distributed computing and energy consumption monitoring systems. It has been proven that integrating renewable energy sources, such as solar and wind energy, into data centre operations significantly reduces carbon emissions. It is revealed that the main problems remain the high cost of energy-efficient equipment, insufficient transparency of algorithms and limited access to green energy.The study results confirm that energy-efficient practices in machine learning can significantly reduce the environmental impact of computing processes. It is recommended that systematic energy consumption monitoring be introduced, infrastructure developed to support green computing, and international standards harmonised in this area.Prospects for further research are focused on improving artificial intelligence models’ performance in conditions of limited resources, developing adaptive algorithms, and minimising environmental risks associated with the introduction of modern technologies.
References
Strubell E., Ganesh A., McCallum A. Energy and policy considerations for modern deep learning research. In: Proceedings of the AAAI conference on artificial intelligence. 2020. P. 13693–13696. DOI: https://doi.org/10.1609/aaai.v34i09.7123 (date of access: 18.01.2025).
Lacoste A., Luccioni A., Schmidt V., Dandres T. Quantifying the carbon emissions of machine learning. arXiv preprint. 2019.arXiv:1910.09700. URL: https://arxiv.org/ abs/1910.09700 (date of access: 18.01.2025).
Patterson D., Gonzalez J., Le Q., Liang C., Munguia L.-M., Rothchild D., So D., Texier M., Dean J. Carbon emissions and large neural network training. 2021. arXiv preprint. arXiv:2104.10350. URL: https://arxiv.org/abs/2104.10350 (date of access: 18.01.2025).
Tmamna J., Ayed E. B., Fourati R., et al. Pruning Deep Neural Networks for Green Energy-Efficient Models: A Survey. Cognitive Computation. 2024. Vol. 16 (11). P. 2931–2952. DOI: https://doi.org/10.1007/s12559-024-10313-0 (date of access: 18.01.2025).
Henderson P., et al. Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research. 2020. Vol. 21 (248). P. 1–43. URL: https://www.jmlr.org/papers/v21/20-312.html (date of access: 18.01.2025).
Mehlin V., Schacht S., Lanquillon C. Towards energy-efficient deep learning: An overview of energy-efficient approaches along the deep learning lifecycle. 2023. arXiv preprint. arXiv:2303.01980. URL: https://arxiv.org/abs/2303.01980 (date of access: 18.01.2025).
Rolnick D., Donti P. L., Kaack L. H., Kochanski K., Lacoste A., Sankaran K., et al. Tackling climate change with machine learning. 2022. ACM Computing Surveys (CSUR). Vol. 55 (2). P. 1–96. DOI: https://doi.org/10.1145/348512 (date of access: 18.01.2025).
Dinh V. K. Q., Nguyen C., Dang V. A., Nguyen M. Q. Federated learning for green and sustainable 6G IIoT applications. Internet of Things. 2024. Vol. 25. Article ID: 101061. DOI: https://doi.org/10.1016/j.iot.2024.101061 (date of access: 18.01.2025).
Khan T., Tian W., Ilager S., Buyya R. Workload forecasting and energy state estimation in cloud data centres: ML-centric approach. Future Generation Computer Systems. 2022. Vol. 128. P. 320–332. DOI: https://doi.org/10.1016/j.future.2021.10.019 (date of access: 18.01.2025).
Bender E. M., Gebru T., McMillan-Major A., Shmitchell S. On the dangers of stochastic parrots: Can language models be too big? In: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 2021. P. 610–623. DOI: https://doi.org/10.1145/3442188.3445922 (date of access: 18.01.2025).
Zennaro F., Furlan E., Simeoni C., Torresan S., Aslan S., Critto A., Marcomini A. Exploring machine learning potential for climate change risk assessment. Earth-Science Reviews. 2021. Vol. 220. Article ID: 103752. DOI: https://doi. org/10.1016/j.earscirev.2021.103752 (date of access: 18.01.2025).
Daghero F., Pagliari D. J., Poncino M. Energy-efficient deep learning inference on edge devices. In: Advances in Computers. Elsevier. 2021. P. 247–301. DOI: https:// doi.org/10.1016/bs.adcom.2020.07.002 (date of access: 18.01.2025).
Narciso D. A. C., Martins F. G. Application of machine learning tools for energy efficiency in industry: A review. Energy Reports. 2020. Vol. 6. P. 1181–1199. DOI: https://doi.org/10.1016/j.egyr.2020.04.035 (date of access: 18.01.2025).
Demirci M. A Survey of Machine Learning Applications for Energy-Efficient Resource Management in Cloud Computing Environments. 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA). Miami, FL, USA. 2015. P. 1185–1190. DOI: https://doi.org/10.1109/ICMLA.2015.205 (date of access: 18.01.2025).
M. Pedram. Energy-Efficient Datacenters. in IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems. Vol. 31. № 10. P. 1465-1484. DOI: 10.1109/TCAD.2012.2212898. (date of access: 18.01.2025).
CodeCarbon: Official documentation. CodeCarbon: official website, 2024. URL: https://codecarbon.io? (date of access: 21.01.2025).
EnergyScope. EnergyScope: official documentation, 2024. URL: https:// energyscope.readthedocs.io/en/master/index.html (date of access: 21.01.2025). Google Sustainability: Data centers and renewable energy. Google: official website, 2024. URL: https://sustainability.google/ (date of access: 21.01.2025).
Energy management standard. ISO 50001: Energy management systems – Requirements with guidance for use. International Organization for Standardization: official website, 2024. URL: https://www.iso.org/standard/50001.html (date of access: 21.01.2025).
LEED Certification: Green building standards. LEED: official website, 2024. URL: https://leed.usgbc.org/ (date of access: 21.01.2025).
Google Sustainability: Data centers and renewable energy. Google: official website, 2024. URL: https://sustainability.google/ (date of access: 21.01.2025).
Microsoft and renewable energy: Building sustainable data centers. Microsoft: official website, 2024. URL: https://microsoft.com/sustainability (date of access: 21.01.2025).
Facebook’s sustainable data centers. Facebook: official website. URL: https:// sustainability.fb.com/ (date of access: 21.01.2025).
Amazon Web Services: Sustainability in cloud infrastructure. Amazon: official website. 2024. URL: https://aws.amazon.com/sustainability/ (date of access: 21.01.2025).
Environmental management standard. ISO 14001: Environmental management systems – Requirements with guidance for use. International Organization for Standardization: official website, 2024. URL: https://www.iso.org/standard/14001.html (date of access: 21.01.2025).
