• re-grasp execution
• grasp failure
• neural network online prediction

This thesis tackles the problem of predicting in real-time when a grasp, autonomously executed by a soft robotic hand, is going to fail, providing the most probable direction of sliding, and proposes a solution to compensate for the potential failure by activating reactive multi-arm primitives. For the prediction part deep learning techniques are implemented in real-time, while, regarding the compensation, primitives are based on human-like motion planning. Using a robotic manipulator, that moves like a human, allows the scenario to be more similar to reality, and in particular, the introduction of a second one, to compensate for grasp failure, is crucial for future applications of collaborative robots. In this work, starting from the studies developed by my partner Irene Valdambrini, who proposed some solutions to predict a grasp failure in off-line mode, first of all, a Neural Network is selected. Two main aspects have to be considered: the test accuracy parameter for detecting grasp failure and the robot velocity constraints to execute the re-grasp action. This network works online and takes as input a multidimensional continuum stream of raw signals coming with a frequency of 70 Hz from 15 Inertial Measurement Units. Data are published on two ROS topics and through a dynamic buffer are provided to Neural Network, which requires a longer time to do inference, about 5 Hz. The network is trained to predict the occurrence in the near future of a slippage event and in particular its direction. Before the 1st robot grasps the object the IMUs data are not significant to predict the correct output, so network output starts to be published on ROS topic when the robot starts to lift the object. If the output predicted is the same for five consecutive times the 2nd robot starts to plan and to execute the re-grasp motion. This strategy is adopted to find a trade-off between a reliable Neural Network output and the time that the 2nd robot takes to do the re-grasp action, about 2 seconds. Based on the output predicted, two different re-grasp primitives will be performed: bottom, if the object slips perpendicularly to the palm of the hand (type 1), or lateral if it slips parallelly (type 2). The grasp and re-grasp trajectories are planned through the MoveIt! software by implementing the RRT Connect algorithm, that finds these quickly, and are executed by robots using an impedance control. Therefore, based on training experiments and outcomes, to verify the Neural Network in real-time, 20 tests are conducted for each typology of grasp: success, failure type 1, and failure type 2. The results show that the network detects 78.33$\%$ of failures online while in off-line mode 83.61$\%$ and the re-grasp is executed successfully 87.5$\%$ times. Finally, to generalize the network, it is tested with daily life objects of different shapes. For each new object, 10 experiments are conducted and the result obtained is the detection of slipping with 77.05$\%$ of accuracy with 88.63$\%$ of successfully re-grasp. Starting from cylinder and sphere, the network is tested to detect slipping considering new shapes. This analysis has been possible using a soft hand that, thanks to its physical adaptability, is able to grasp objects with different shapes and surfaces, while grasping a pan, the contact surface between object and hand is smaller than the sphere case. The network generalization is also possible using IMUs data as sensor strategy. This work shows consistent results which could be improved by introducing a system of vision to detect the object to grasp and to develop autonomous grasping, or detecting failures but also studying hand-object contact to obtain useful information to provide to the network.