• Generative models
• Machine Learning
• Robot
• Vision

This thesis investigates the coupling of action and perception in soft robots, looking at the influence of robot actions on the evolution of visual information to construct a perceptual predictive model. The study explores a neural network architecture based on conditional variational autoencoders (CVAEs) for predicting images, given the current image and an applied action affecting the depicted system. First, model selection is conducted to determine the optimal VAE architecture for image projection in a compact representation vector. Next, the conditioning action is applied to the input image representation vector and serves as input to a recurrent network that performs regression to the target image representation vector. This methodology is benchmarked against a feedforward network, demonstrating its superior performance in prediction accuracy and image reconstruction. Finally, an ablation study is conducted on this model to analyze its capabilities in greater depth.