• pre-trained language models
• neural language model
• metonymy
• Transformers
• probing task
• BERT
• NLP

Pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers) have achieved state of the art performances in NLP. The success of such models is due to the contextualized vector representations of language, also known as embeddings, they are able to generate and that have attracted much attention from researchers. In fact, pre-trained language models suffer from low interpretability, meaning that is it difficult to identify the pieces of information encoded in their embeddings, and where and how they are encoded. The aim of this work is to probe the vector representations extracted from BERT to understand whether it captures some information related to one linguistic phenomenon in particular, metonymy.