• contact sensing
• deformable surface
• soft sensors
• tactile sensing
• TacTip

Tactile sensing is an essential component in autonomous grasping and haptic manipulation. Intrinsic Tactile Sensing (ITS) is a technique that relies on force/torque measurements and on a priori knowledge of the geometry of the exploring surface in order to address the contact sensing problem, i.e., the problem of resolving the location of a contact, the force and moment exerted at the interface. ITS is a well established technique with rigid surfaces but it is ill-suited in the case of soft deformable materials, although some solutions to tackle this case have been proposed.
In the state of art, a variety of soft optical sensors are available and suitable for real-world applications in tactile perception, exploration and manipulation. They exploit vision-based algorithms to gather rich geometric information about the contact point but they lack of an estimation of the contact forces.
As of today, the integration of ITS and soft optical sensors has never been attempted before. Such an integration could enrich the characterization of the contact with soft deformable fingertips.
This thesis proposes to integrate a vision-based sensing surface estimation relying on the TacTip soft optical sensor with the ITS technique in order to enhance the solution of the contact sensing problem. Firstly, the optical sensor images are converted into a new feature, the marker density, via Gaussian Kernel. The new feature is an approximation of the tip deformation and thus can be used to infer the tip displacement. Secondly, the estimated displacement is used to address the ITS on a shrunk geometry approximating the deformed surface. Two methods are presented: (i) a Closed-Form solution and (ii) an iterative solution that take the force-deformation behaviour of the tip materials into account.