• Station-keeping
• permanence
• surveillance
• sounding balloon
• stratospheric platform
• artificial intelligence
• reinforcement learning
• AI

Sounding balloons, often called weather balloons, are the smallest members of the aerostat family. Compared to other aerial platforms, such as drones or aircraft, or their larger aerostat counterparts (zero-pressure or super-pressure balloons), sounding balloons stay afloat in the atmosphere for relatively short periods of time, are extremely cheap to manufacture and operate and are generally considered to be a cost-effective solution for various applications such as surveillance, communication, and atmospheric research.

The capability to perform station-keeping would be crucial for these balloons to perform specific missions. However, the control of sounding balloons is quite challenging due to the strong dependence on their dynamics on atmospheric conditions. Traditional control methods based on mathematical models are not efficient for controlling these balloons in complex and uncertain environments. Therefore, there is a need for a new approach to effectively control sounding balloons.

The use of Deep Reinforcement Learning (DRL) has shown significant potential in solving complex control problems in various fields, including robotics, gaming, and autonomous vehicles. DRL combines the advantages of deep learning and reinforcement learning to learn optimal control policies. As of today, there is limited research on applying DRL to the control of high-altitude balloons.

The main objective of this thesis is to explore the use of DRL for controlling a sounding balloon to ensure permanence over a specified area. Specifically, we aim to implement the deep Q-network (DQN) algorithm to learn a control policy for the balloon, by exploiting the wind direction at different altitudes, reached by dropping ballast or releasing helium. This work aims to conduct experiments using a simulation environment and evaluate the performance of the trained DQN model in real historical wind data. The goal is to demonstrate that the DQN algorithm can effectively learn a control policy that achieves station-keeping with a high success rate and outperforms the baseline approaches.

We then used an auto-encoder to analyze wind data to investigate the underlying structures of wind data and identifying features where wind patterns were particularly conducive to station-keeping. By reducing the data into a condensed, easily interpretable form, we were able to gain new insights into the atmospheric conditions that affect station-keeping.

Finally, we demonstrated the possibility of station-keeping for the first time in history through an actual launch carried out on 05/25/2023.