ETD

• deep learning
• image-based contact force estimation
• wearable force sensing systems

Grasping objects is a complex activity through which humans interact with the world. Accurate characterization of grasping forces is essential for understanding grasp stability and force coordination. Sensing systems should capture the three-dimensional contact force vector without constraining where contact occurs or altering natural tactile stimuli.
Conventional sensing approaches often fail to meet these requirements. Early methods relied either on sensorized objects, which limited interaction to specific locations on the sensing surface, or on miniaturized fingertip sensors, which could interfere with the tactile cues used to regulate force. Vision-based methods have emerged as an unobtrusive alternative, estimating contact forces from colour variations in the fingernail and surrounding skin induced by fingertip deformation. However, existing approaches often rely on fixed camera setups or require subject-specific calibration, limiting their flexibility.
This study introduces a deep neural network to estimate the three-dimensional grasping force vector from images of the fingernail and surrounding skin. These images are acquired through a camera embedded in a custom-designed wearable device, enabling unconstrained and unobtrusive grasping during natural object interactions. By avoiding physical sensors at the contact interface, the system preserves natural tactile feedback and allows free selection of hand posture and contact locations.