• control
• data-driven control
• learning-based control
• legged robotics
• legged robots
• locomotion control
• quadrupedal robots
• quadrupeds
• reinforcement learning
• robotics

In recent years, tasks regarding autonomous mobility favored the use of legged robots rather than wheeled ones thanks to their higher mobility on rough and uneven terrains. This comes at the cost of more complex motion planners and controllers to ensure robot stability and balance. However, in the case of quadrupedal robots, balancing is simpler than it is for bipeds thanks to their larger support polygons. Until a few years ago, most scientists and engineers addressed the quadrupedal locomotion problem with model-based approaches, which require a great deal of modeling expertise. A new trend is the use of data-driven methods, which seem to be quite promising and have shown great results. These methods do not require any modeling effort, but they suffer from computational limitations dictated by the hardware resources used. However, only the design phase of these algorithms requires large computing resources (controller training); their execution in the operational phase (deployment), takes place in real time on common processors.
This dissertation explores novel Reinforcement Learning (RL)--based control methodologies for various quadrupedal robot platforms with distinct design characteristics and locomotion requirements. First, the design and the foundational locomotion control framework for an 8-degrees-of-freedom (8-DoF) quadrupedal robot is introduced. Subsequently, the approach is extended to the ANYmal-C robot, equipped with adaptive feet, to achieve advanced terrain adaptability. Next, the hybrid wheeled-legged configuration of the 8-DoF quadrupedal robot, equipped with four omniwheels at the feet location, is examined to enable enhanced mobility, increasing the system to 12 actuated DoF. Finally, the development of motor skills, including jumping and climbing, for the SOLO 12 robot is investigated. This dissertation contributes to advancing quadrupedal locomotion capabilities through reinforcement learning in increasingly complex and versatile robotic systems.
As said, at first, a framework for training neural network-based locomotion control policies for an 8-DoF quadrupedal robot using reinforcement learning is presented. Unlike typical 12-DoF quadrupeds, this design lacks hip adduction abduction, increasing the challenge for base rotation and angular twist control. This simplification reduces robot weight, enhancing its power-to-weight ratio and enabling more dynamic maneuvers, like high jumps, which are valuable for tasks such as environmental monitoring. RL-based policies were trained in GPU-accelerated simulations for stable, efficient movement despite mechanical constraints, then successfully transferred to the physical robot for tasks like trajectory following. These results highlight RL promise for controlling simplified robotic systems, advancing the field of underactuated quadrupedal robots.
Adaptive feet capable of sensing terrain profile information have been designed and have shown great performance. Still, no dynamic locomotion control method has been specifically designed to leverage the advantages and supplementary information provided by this type of adaptive feet. For these reasons, the use of different end-to-end control policies trained via reinforcement learning algorithms, specifically designed and trained to work on quadrupedal robots equipped with adaptive feet, is investigated and the performance evaluated for their dynamic locomotion control over a diverse set of terrains. Furthermore, we examine how the addition of haptic terrain perception affects locomotion performance.
Building on the design of the 8-DoF quadrupedal robot, an innovative hybrid wheeled-legged configuration is explored by incorporating omniwheels at each foot, resulting in a 12-DoF actuated platform. This hybrid design enhances robot maneuverability, combining traditional legged locomotion with wheeled mobility to achieve efficient and adaptive ground traversal. Reinforcement learning algorithms are employed to develop a control framework that can manage the complexity of the additional degrees of freedom, allowing seamless transitions between stepping and rolling motions. Simulations reveal that this approach significantly improves performance in tasks requiring fast adaptation, omnidirectional movement, and obstacle negotiation. These findings highlight the advantages of hybrid locomotion for quadrupedal robots, especially in scenarios where agility and versatility are paramount, and demonstrate RL efficacy in creating cohesive control policies for hybrid systems.
Lastly, the development of highly dynamic motor skills, specifically jumping and climbing, is investigated for the SOLO 12 quadrupedal robot. Reinforcement learning techniques are applied to create control policies that enable robust and precise high-energy movements, ensuring balance and stability during actions like vertical and lateral jumps, as well as climbing steep surfaces and obstacles. These skills are tested in diverse simulated environments, showcasing the adaptability of the policies across varied conditions and evaluating their transferability to real-world robotic platforms. This work underscores the feasibility of applying RL to achieve dynamic, high-agility tasks in quadrupedal robots, extending their applicability to challenging scenarios, such as unstructured terrain exploration and search-and-rescue missions.