• deep learning
• ECB communication
• multimodal analysis
• stock market forecasting

Predicting the movements of a stock index is a challenging task that has been extensively researched in the literature. In this thesis, we have two objectives. The first is to explore the possibility of successfully predicting the surprises of the SX7E banking index after the European Central Bank (ECB) press conferences through a multimodal approach that exploits their audios and transcripts. The second, functional to the first, is creating the first Multimodal Aligned database of ECB Press Conferences (ECB - MAPC) to lay the foundations for further multimodal research on central bank communication. For the predictions, we use a two-stage approach based on a first extraction of context-dependent speech features and a second stage, where the actual forecast takes place. We then compare the predictive performance of different machine and deep learning models, finding that good levels of accuracy can be achieved. The results also show that multimodality is crucial for decreasing prediction error.