Tipo di tesi
Tesi di laurea magistrale
Titolo
Regularizing Transformers by symbolic knowledge and Deep Graph Networks
Corso di studi
INFORMATICA
Parole chiave
- Bert
- Deep Graph Networks
- embeddings regression
- Graph Attention Network
- language modeling
- node-embeddings
- symbolic knowledge
- Transformers
- word-embeddings
Data inizio appello
22/04/2022
Consultabilità
Tesi non consultabile
Riassunto (Italiano)
In recent years, Deep Graph Networks (DGNs) have proven to be one of the state-of-art for representation learning for graphs. This thesis focuses on using Graph Attention Network embeddings to regularize the Transformers model, specifically BERT. The word-embeddings learned during BERT's Masked Language Modeling are regularized to combine symbolic knowledge of Knowledge Graphs. This work evaluates the quality of graph-regularized word-embeddings concerning the baseline through word-embeddings probing tasks.
