COMPARATIVE ANALYSIS OF FEDERATED MACHINE LEARNING ALGORITHMS
DOI:
https://doi.org/10.37943/17BVCN7579Keywords:
federated learning, FedAvg , FedAdam , FedYogi , FedSparse , loss , accuracyAbstract
In this paper, the authors propose a new machine learning paradigm, federated machine learning. This method produces accurate predictions without revealing private data. It requires less network traffic, reduces communication costs and enables private learning from device to device. Federated machine learning helps to build models and further the models are moved to the device. Applications are particularly prevalent in healthcare, finance, retail, etc., as regulations make it difficult to share sensitive information. Note that this method creates an opportunity to build models with huge amounts of data by combining multiple databases and devices. There are many algorithms available in this area of machine learning and new ones are constantly being created. Our paper presents a comparative analysis of algorithms: FedAdam, FedYogi and FedSparse. But we need to keep in mind that FedAvg is at the core of many federated machine learning algorithms. Data testing was conducted using the Flower and Kaggle platforms with the above algorithms.
Federated machine learning technology is usable in smartphones and other devices where it can create accurate predictions without revealing raw personal data. In organizations, it can reduce network load and enable private learning between devices. Federated machine learning can help develop models for the Internet of Things that adapt to changes in the system while protecting user privacy. And it is also used to develop an AI model to meet the risk requirements of leaking client's personal data. The main aspects to consider are privacy and security of the data, the choice of the client to whom the algorithm itself will be directed to process the data, communication costs as well as its quality, and the platform for model aggregation.
References
Gabrielli, E., Pica, G., & Tolomei, G. (2023). A Survey on Decentralized Federated Learning. ArXiv, abs/2308.04604.
Li, T., Sahu, A., Talwalkar, A., & Smith, V. (2019). Federated Learning: Challenges, Methods, and Future Directions. IEEE Signal Processing Magazine, 37, 50-60.
Aledhari, M., Razzak, R., Parizi, R.M., & Saeed, F. (2020). Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications. IEEE Access, 8, 140699-140725.
Dinh, T.Q., Tran, T., & Le, T. (2021). Communication cost reduction using sparse ternary compression and encoding for FedAvg. 2021 International Conference on Information and Communication Technology Convergence (ICTC), 351-356.
Abhishek, V, A., Binny, S., JohanT, R., Raj, N., & Thomas, V. (2022). Federated Learning: Collaborative Machine Learning without Centralized Training Data. International journal of engineering technology and management sciences.
Dinh, T.Q., Tran, T., & Le, T. (2021). Communication cost reduction using sparse ternary compression and encoding for FedAvg. 2021 International Conference on Information and Communication Technology Convergence (ICTC), 351-356.
Pang, S., Peng, Y., Ban, T., Inoue, D., & Sarrafzadeh, A. (2015). A federated network online network traffics analysis engine for cybersecurity. 2015 International Joint Conference on Neural Networks (IJCNN), 1-8.
Mosharraf, M., & Taghiyareh, F. (2017). Federated Search Engine for Open Educational Linked Data.
Smith, V., Chiang, C., Sanjabi, M., & Talwalkar, A. (2017). Federated Multi-Task Learning. Neural Information Processing Systems.
Zhou, W., Li, Y., Chen, S., & Ding, B. (2018). Real-Time Data Processing Architecture for Multi-Robots Based on Differential Federated Learning. 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), 462-471.
Tyagi, S., Rajput, I.S., & Pandey, R. (2023). Federated learning: Applications, Security hazards and Defense measures. 2023 International Conference on Device Intelligence, Computing and Communication Technologies, (DICCT), 477-482.
Zhao, Y. (2023). Comparison of Federated Learning Algorithms for Image Classification. 2023 2nd International Conference on Data Analytics, Computing and Artificial Intelligence (ICDACAI), 613-615.
Cai, L., Chen, N., Wei, Y., Chen, H., & Li, Y. (2022). Cluster-based Federated Learning Framework for Intrusion Detection. 2022 IEEE 13th International Symposium on Parallel Architectures, Algorithms and Programming (PAAP), 1-6.
Panigrahi, M., Bharti, S., & Sharma, A. (2023). Federated Learning for Beginners: Types, Simulation Environments, and Open Challenges. 2023 International Conference on Computer, Electronics & Electrical Engineering & their Applications (IC2E3), 1-6.
Stano, M., Hluchý, L., Bobák, M., Krammer, P., & Tran, V.D. (2023). Federated Learning Methods for Analytics of Big and Sensitive Distributed Data and Survey. 2023 IEEE 17th International Symposium on Applied Computational Intelligence and Informatics (SACI), 000705-000710
Reddy, G.P., & Kumar, Y.V. (2023). A Beginner’s Guide to Federated Learning. 2023 Intelligent Methods, Systems, and Applications (IMSA), 557-562.
Michálek, J., Skorpil, V., & Oujezský, V. (2022). Federated Learning on Android - Highlights from Recent Developments. 2022 14th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), 27-30.
Obarafor, V., Qi, M., & Zhang, L. (2023). A Review of Privacy-Preserving Federated Learning, Deep Learning, and Machine Learning IIoT and IoTs Solutions. 2023 8th International Conference on Signal and Image Processing (ICSIP), 1074-1078.
Gu, Z., Shi, J., Yang, Y., & He, L. (2023). Defending against Poisoning Attacks in Federated Learning from a Spatial-temporal Perspective. 2023 42nd International Symposium on Reliable Distributed Systems (SRDS), 25-34.
Aliyev, S., & Ismayilova, N. (2023). FL2: Fuzzy Logic for Device Selection in Federated Learning. 2023 IEEE 17th International Conference on Application of Information and Communication Technologies (AICT), 1-6.
Reguieg, H., Hanjri, M. E., Kamili, M. E., & Kobbane, A. (2023). A Comparative Evaluation of FedAvg and Per-FedAvg Algorithms for Dirichlet Distributed Heterogeneous Data. 2023 10th International Conference on Wireless Networks and Mobile Communications (WINCOM), 1-6. https://doi.org/10.1109/WINCOM59760.2023.10322899
Bian, K., & Zheng, H. (2023). FedAvg-DWA: A Novel Algorithm for Enhanced Fraud Detection in Federated Learning Environment. 2023 4th International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), 13-17. https://doi.org/10.1109/ICBAIE59714.2023.10281317.
Shin, W., & Shin, J. (2022). FedVar: Federated Learning Algorithm with Weight Variation in Clients,”2022 37th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC), 1-4. https://doi.org/10.1109/ITCCSCC55581.2022.9894899.
Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S.J., Stich, S.U., & Suresh, A.T. (2019). SCAFFOLD: Stochastic Controlled Averaging for Federated Learning. International Conference on Machine Learning.
Krizhevsky, A. (2009). Learning Multiple Layers of Features from Tiny Images
Nilsback, M., & Zisserman, A. (2008). Automated Flower Classification over a Large Number of Classes. 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, 722-729.
Hayashi, T., Shimizu, T., & Fukami, Y. (2021). Collaborative Problem Solving on a Data Platform Kaggle. ArXiv, abs/2107.11929.
McMahan, H.B., Moore, E., Ramage, D., Hampson, S., & Arcas, B.A. (2016). Communication-Efficient Learning of Deep Networks from Decentralized Data. International Conference on Artificial Intelligence and Statistics.
Nilsson, A., Smith, S., Ulm, G., Gustavsson, E., & Jirstrand, M. (2018). A Performance Evaluation of Federated Learning Algorithms. Proceedings of the Second Workshop on Distributed Infrastructures for Deep Learning.
Reddi, S.J., Charles, Z.B., Zaheer, M., Garrett, Z., Rush, K., Konecný, J., Kumar, S., & McMahan, H.B. (2020). Adaptive Federated Optimization. ArXiv, abs/2003.00295.
Louizos, C., Reisser, M., Soriaga, J.B., & Welling, M. (2021). An Expectation-Maximization Perspective on Federated Learning. ArXiv, abs/2111.10192
Mohammadzadeh, A., Zhang, C., Alattas, K.A., El-Sousy, F.F., & Vu, M.T. (2023). Fourier-based type2 fuzzy neural network: Simple and effective for high dimensional problems. Neurocomputing, 547, 126316.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Articles are open access under the Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish a manuscript in this journal agree to the following terms:
- The authors reserve the right to authorship of their work and transfer to the journal the right of first publication under the terms of the Creative Commons Attribution License, which allows others to freely distribute the published work with a mandatory link to the the original work and the first publication of the work in this journal.
- Authors have the right to conclude independent additional agreements that relate to the non-exclusive distribution of the work in the form in which it was published by this journal (for example, to post the work in the electronic repository of the institution or publish as part of a monograph), providing the link to the first publication of the work in this journal.
- Other terms stated in the Copyright Agreement.