• robotics
• computer vision
• pose estimation
• joint tracking

Pose estimation from RGB images is a challenging task and an ill-posed problem by itself. Its usefulness is clear in situation where joint tracking is hard, like in cable-driven surgical robots. At the same time, data collection in 3D is incredibly expensive, and no open datasets are available in the surgical robotics field.

In this work we address the problem with a deep learning based approach inspired by ideas used in the human pose estimation research line. We developed a streamlined approach to generate synthetic data from CAD files and collected real images manually with 2D labels.
We dug deeper in order to understand how multiple views could benefit our processing and optimized the prediction over multiple frames.

Our approach requires no additional hardware and solves the error drifting in the joint predictions marking an improvement over the currently used joint tracking method.