• 5G
• 6G
• Artificial Intelligence
• GenAI
• NTN
• Reinforcement Learning
• Satellite Communication
• Wireless Communication

My research project is about the development of a learning-based scheduling system that uses Actor-Critic Reinforcement Learning (AC-RL) and Generative Artificial Intelligence (GenAI) to support Multi-connectivity (MC) in fifth generation (5G) and beyond (B5G) mobile networks. MC is a communication technology that enables simultaneous connection across multiple paths of the same network or different networks to improve reliability, throughput, data rates, load balancing and bandwidth aggregation by utilizing the different channel characteristics of each path.
The main challenge in MC is the selection of suitable paths and the determination of the required amount of traffic to be allocated to each path. Traffic scheduling poses a great challenge, due to the temporal and spatial channel variability and heterogeneity in wireless mobile networks. When multipath is coupled with traffic duplication for flow protection, redundancy estimation intensifies the challenge in traffic scheduling because too little redundancy cannot provide the protection required while too much redundancy wastes bandwidth. Unfortunately, most legacy schedulers such as Round-Robin (RR) and Weighted Round Robin (WRR) either emphasize one aspect and neglect the other or use static scheduling policies that do not take account of the heterogeneity and dynamic nature of wireless networks. Moreover, conventional traffic protection schemes such forward error correction (FEC) add computational requirements due to coding and decoding while automatic repeat request (ARQ) causes additional delays due to retransmissions which severely impact real-time traffic. In addition, the advent of fifth generation (5G) and the forthcoming Sixth Generation (6G) have stringent requirements in terms of delay and data rates. This has attracted research in cognitive and self-reconfigurable systems that can automatically and dynamically adapt to changing network conditions
In this research project, we develop a scheduling system that uses the AC-RL algorithm to learn in real time the channel conditions of the available network paths, such as packet loss rate, bandwidth, congestion, and delay, and select an appropriate subset of the paths to avoid traffic loss. The thesis focuses on MC in both terrestrial networks (TN), where a user equipment (UE) can simultaneously connect to multiple mobile networks such as 4G and 5G. It also addresses MC in the networks that integrate TN and non-terrestrial networks (NTN) such as satellites, High Altitude Platforms (HAPs) and Unmanned Aerial Systems (UAS) as proposed by the Third Generation Partnership Project (3GPP), in which the UE accesses the Internet via the 5G core by connecting simultaneously to multiple satellites with 5G gNB. The scheduling system can also be used to support Access Traffic Steering, Splitting and Switching (ATSSS), a 3GPP standard for MC that includes 3GPP access networks such as 4G or 5G and non-3GPP access networks such as NTN or Wireless Local Area Network (WLAN).
In the NTN scenarios considered, our scheduling system can predict the line-of-sight (LOS) of multiple satellites to which a UE is connected and distribute traffic to appropriate satellite links to avoid traffic loss due to LOS fluctuations. In satellite communications with moving satellites, such as Low Earth Orbit (LEO) satellites, the LOS probability varies with the elevation angle of the satellite, making the propagation channel non-stationary. Since the proposed scheduling system is based on RL, the learning agent is able to constantly retrain its model to track the changing satellite elevation angle. However, since the process of LOS estimation with RL is modelled as a partially observable Markov decision process (POMDP) for scalability reasons, the agent takes a very long time to converge in a system with many links compared to the satellite visibility time, which is usually very short, in the order of a few minutes. To solve this problem, we use GenAI, specifically Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), to generate synthetic LOS states of the links that are not accessed by the agent during the learning process, thus converting the POMDP into a fully observable process (FOMDP). This provides the agent with a complete view of all LOS states of the available connections and increases the convergence rate. In the development of this scheduling system, the AC-RL and the generative models were designed using fully multi-perceptron neural networks (NNs), TensorFlow, Keras, Python and its libraries such as Numpy, Pandas and Scikit-learn. Although single user communication was considered, this work can be extended to multi-user communication by using multi-agent AC-RL algorithm with multi-objective functions where multiple gNBs or UEs can compete or collaborate for joint-resource optimization.
Simulation results have shown that our scheduling system can reduce End-to-End Packet Loss Rate (E2E-PLR) for video streaming in MC wireless networks. It is also able to estimate efficiently the redundancy required to overcome traffic loss without excessive bandwidth usage. Since it is dynamic and learning based, it outperforms the legacy schedulers such as RR and WRR which are static and unable to handle dynamic and heterogeneous wireless networks. It is light weight suitable for resource constrained devices such as UAVs, outperforming traffic protection techniques like FEC which requires coding and decoding and ARQ which is unsuitable for real-time multimedia traffic due to delays as a result of retransmissions. The use of GenAI accelerates the convergence rate of a RL agent by about 89% in LOS estimation in TN-NTN integrated networks.