• Robot Grasp Softhand Neural-Network Deep-Learning

This paper addresses the challenge of predicting grasping errors and slip direction in soft hands before they happen by combining Deep Learning with a distributed Inertial Measurement Units based sensing strategy. We first propose two neural architectures that we have implemented and tested with data from previous work collected with an articulated soft hand, the Pisa/IIT SoftHand, and an IMUs glove. The first architecture (regressor) implements a prediction at time t of the Kth sensor data at time t+1. The second (classifier) classifies the entire data sequence as belonging to one of N classes and the choice falls on this model because it is computationally less expensive than the regressor. The general idea behind this is that the data is pruned at the time of failure and then the entire sequence is given as input to the neural network with a backward pruning from the last time. The final data corresponds to a series of experiments in which failure events have not yet occurred, in this way the classification of the data is equivalent to a prediction of the failure. In order to achieve a correspondence between the time of prediction and the backward cut, it is necessary to know the frequency of the experiments.
Previous works have some limitations that are improved in this work: a lack of data experiments, a non-generalized setup, the time of failure is detected manually with the possibility of human error and only one direction of slip, so it is necessary to collect new data.
The new experimental setup is created to collect more robust data to be sent to the network during training. The Sensus apparatus includes: an optical motion tracking 3D system for offline detection of fault timing and a glove equipped with 17 IMUs to provide information about slippage to the network. Two robotic platforms are involved in this work, the first, mentioned earlier, is Pisa/IIT SoftHand and the second is the robotic arm with 7 D.O.F. Panda from Franka Emika. The failures are caused by the weight added to the grasped object to obtain generalized failures, and the objects are built with a 3D printer to give them the best shape. The experiments collected in this setup are 1800, divided into rough and smooth surfaces, and are classified into three groups: Success, Lateral Slippage, and Central Slippage, and each is preprocessed before being input to the neural network.
During model selection, some neural networks are tested to predict failure events before they occur. The network that is best suited for a temporal sequence of data is a Recurrent Neural Network and the two gated networks are tested: LSTM(Long Short Term Memory) and GRU (Gated Recurrent Units). The falls are detected in a window of 1-3 seconds before they occur with a test accuracy between 86-76%. These results are good offline, but the real challenge is to test whether or not the percentage accuracy could be as high in real time.
The next goal will be to find a predictive model to test in real time and test the network with a second robot to retrieve objects that are about to fall.