• semantic segmentation
• artificial neural networks
• deep learning
• computer vision
• virtual reality

Boosting the quality of semantic understanding of images and videos has recently become a key process in computer vision. Artificial neural networks have played an important role in defining new ways to extract valuable information from large and diverse sets of data. This capability allows them to be applied to a wide range of applications, including Virtual Reality: Object Detection, Semantic Segmentation and Human Pose Estimation can be vital in Virtual Environments reconstruction based on monocular images and videos. In this work we focus on the task of Semantic Segmentation, which deals with a pixel-level classification, and we will compare some of the state-of-the-art methods addressing this task. The evolution of such networks is strongly related with both the quantity and quality of the data used for learning. We study this problem by testing the models over two brand new datasets, which address music-related environments. The goal is to obtain an efficient network able to segment live concert videos from which we want to build the Virtual Environment.
The models are evaluated in terms of mean Intersection over Union index (mIoU) achieved on evaluation sets, and in terms of efficiency analyzing resource utilization and processing time during training and inference. We also discuss the fact that using Generative Adversarial Networks to produce new data cannot be used as aid to enrich a small dataset.