• Federated Learning
• Generative AI
• Internet of Things

Machine Learning (ML) is a research field that deals with the concept of extracting knowledge from data that is usually collected from external devices. The main goal of ML is to create a model that is able to predict the correct output based on certain input data. However, despite continuous advances in computing power and storage capabilities, modern technologies often struggle with the sheer volume of data collected, especially in the case of the Internet of Things (IoT). Transferring these massive amounts of data to central servers for processing can lead to network congestion and performance degradation.
In addition, the data collected may contain sensitive information, posing a significant risk to user privacy. In response to these challenges, Federated Learning (FL) has gained significant attention. FL is an innovative ML technique that aims to solve these problems by training an ML algorithm on a large number of decentralized devices on their own local data without sharing the raw data with the central server.
FL differs from traditional ML in several important ways. One notable difference is the inherent heterogeneity, both in the data itself and in the capabilities of the devices involved, and in the different network conditions on those devices.
For example, a major challenge is the communication of millions of model parameters, which can cause significant communication overhead.
In this thesis, we have investigated several new approaches to solve the various problems related to FL. This thesis includes three main objectives, the first of which focuses on developing a scalable communication infrastructure to support the FL process. This is achieved by implementing an architecture based on Information-Centric Networks (ICN) using Apache Kafka. In ICN, data is stored in intermediate nodes, which ensures efficient and reliable data transmission. This approach is particularly valuable in solving the problem of intermittent connectivity that is common with mobile nodes at the edge of the network.
While aggregating numerous local model updates from clients can improve the accuracy of the global model in FL, it can also lead to significant data traffic and thus congestion problems at the edge. To mitigate these challenges, our main focus is on developing client selection techniques. Since communication bottlenecks often occur in FL systems, it is important to develop strategies that prevent the selection of all or a large fraction of clients, thus preventing excessive increases in communication costs and congestion.
In this context, the process of client selection plays a crucial role in reducing bottlenecks, managing communication costs, and optimizing the overall use of computing resources. The proposed client selection algorithm is designed to select a subset of clients based on their available resources. It incorporates a time-based backoff system that considers the time-averaged accuracy of FL to effectively manage the traffic load. In addition, this algorithm improves client resource utilization, mitigates the risk of queue overflow, and ensures fairness in client selection.
The second goal of this thesis is to develop a model specific to resource-constrained devices. This model serves a dual purpose: first, it seeks to minimize communication costs within FL. Second, it seeks to reduce the deployment cost associated with each client device for model inference. To achieve these goals, we use a creative approach that involves integrating a cross-modal distillation technique into FL. The effectiveness of this approach depends on its ability to significantly reduce communication costs while maintaining strict privacy standards. In addition, the resulting model is much more robust to overfitting, leading to better generalization than other existing methods.
The third main objective of this thesis is to address the challenges related to data scarcity and missing data values while improving the resilience of the federated model. In this final section of the thesis, we propose to leverage the capabilities of generative models, specifically Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). Generative models exhibit remarkable versatility in various domains and offer a wide range of benefits to advance the field of distributed optimization.
Generative models are invaluable in creating synthetic datasets that can subsequently be used to train ML models. We have used generative models to create synthetic samples and fill gaps in datasets because they have a remarkable ability to capture the underlying data structure and generate values for missing data points. This capability proves to be indispensable in scenarios where data gaps are prevalent, such as time series.
Finally, our goal is to develop a universal, unsupervised generative model. The adaptability and versatility of this model make it a valuable asset in both IoT and distributed environments with the potential for a wide range of applications. The resulting model is capable of learning different data distributions and generating realistic patterns without the need for extensive data analysis prior to the training phase. In summary, the goal is to develop a general-purpose model that takes into account scalability, communication costs, and resource constraints, and aims to improve local data distributions by filling in missing values or generating synthetic data within a dataset.