• artificial intelligence
• c
• convolutional neural networks inference
• deep learning
• PolarFire SoC FPGA
• python
• Vectorblox Microchip

The unstoppable growth of artificial intelligence in recent years has created new opportunities for several industries, including space. Several ML algorithms, in particular, have become major players in space missions, for example, to support operations of large satellite constellations, such as relative positioning, communications, and end-of-life management, but also to promote greater rover autonomy, such as in guidance. In particular, when AI is used directly on embedded systems, without the need to transfer data to a remote server for processing, it is more appropriate to speak of Edge AI. Edge computing is critical for satellite applications, in fact, real-time data processing is essential for those applications where an autonomous decision needs to be made quickly and the round-trip delay to Earth can be prohibitive for any real-time application and decision-making. The optimization and implementation of Edge Computing, which results first and foremost in lower latency but also in lower bandwidth required to transmit data and a consequent lower cost for spatial communication, requires the assistance of artificial intelligence algorithms. In this thesis work, the main focus is on neural networks (NNs), more specifically the convolutional ones (CNNs), which have emerged as one of the most important ML algorithms due to their power of generalization and versatility to various applications. Obviously, when it comes to the space environment, devices must not only survive the accelerations and vibrations of launch but also cope with extreme conditions in terms of temperature, high vacuum, and especially being hit by ionizing particles. Therefore, it is necessary to have space-qualified devices that are radiation tolerant. In the initial chapters of this thesis work, after a theoretical part on AI, ML, and particularly convolutional neural networks, which are the ones used, the different solutions available to run artificial intelligence algorithms directly on satellites are explored. Basically, there are two categories of components available for this purpose: COTS devices and space-qualified devices. COTS stands for "Commercial Off-The-Shelf” and refers to components designed not specifically for space applications, but for a generic commercial setting (Off The Shelf precisely), thus without additional requirements imposed by external users and organizations. Whether or not to use COTS devices in space applications is a choice that must be well thought out because, while it leads to many benefits, it can compromise the entire space mission. It greatly depends on what performance these components offer and the specific mission requirements. Sometimes, the reliability of radiation-tolerant devices is needed, so several ESA missions, mostly those directed further afield, still rely on space-qualified components. The introductory section then shows an overview of available solutions, trying to orientate between the state of the art of available space-qualified processors, such as the LEON series, and the state of art of AI cores, soft and hard. The former, while perfectly suited for space missions, often do not guarantee adequate performance for very complex AI and ML algorithms; the latter, on the other hand, are specifically designed for artificial intelligence but do not always offer adequate guarantees in terms of reliability. However, it is important to point out that solutions are already available to combine both of the above-mentioned needs, such as LEON's SPARROW extension or the Versal AI Core series. In this thesis, the accelerator used to infer neural networks and whose performance was evaluated is Microchip's hardware accelerator VectorBlox, a soft IP. The hardware used is the PolarFire SoC FPGA, part of a family of devices that makes low power consumption and thermal efficiency one of its main strengths, while the board containing it is the Icicle Kit. First, the inference times of some of the most common patterns of convolutional and fully connected neural networks were evaluated, varying the input shape and kernel size of the convolutional filters and the number of units of the dense layers, respectively. Next, again to evaluate the performance of VectorBlox, four neural networks were created, three of them by transfer-learning with some of the best-known Deep CNN architectures (MobileNet, ResNet, and Inception), and trained on the EuroSAT dataset from ESA's Sentinel 2 mission. A comprehensive analysis was conducted with these models, comparing not only inference times but also the size required to store them, the accuracy of the models before and after quantization, and the time necessary to prepare input/output buffers. Finally, models created specifically for satellite applications from companies (Airbus), universities of Pisa, and within ESA (Gitlab) were inferred, or at least an attempt was made. The code was run on Linux onboard and the programming languages used were C to infer the networks and test their performance with the VectorBlox API, Python to process the results (e.g., plotting them), and in particular, the Keras API to create, train, manipulate, and test the neural networks and some scripts in bash to evaluate the accuracy of the networks on the test set. The work done and the results obtained have also been uploaded to ESA's internal Gitlab, where they are added to a set of machine learning (ML) application-specific benchmarks.