ETD

Archivio digitale delle tesi discusse presso l'Università di Pisa

Tesi etd-03222022-131206


Tipo di tesi
Tesi di laurea magistrale
Autore
MEDIOLI, MATTEO
URN
etd-03222022-131206
Titolo
Regularizing Transformers by symbolic knowledge and Deep Graph Networks
Dipartimento
INFORMATICA
Corso di studi
INFORMATICA
Relatori
relatore Bacciu, Davide
correlatore Valenti, Andrea
supervisore Passaro, Lucia C.
Parole chiave
  • embeddings regression
  • symbolic knowledge
  • node-embeddings
  • word-embeddings
  • Graph Attention Network
  • Bert
  • Deep Graph Networks
  • Transformers
  • language modeling
Data inizio appello
22/04/2022
Consultabilità
Tesi non consultabile
Riassunto
In recent years, Deep Graph Networks (DGNs) have proven to be one of the state-of-art for representation learning for graphs. This thesis focuses on using Graph Attention Network embeddings to regularize the Transformers model, specifically BERT. The word-embeddings learned during BERT's Masked Language Modeling are regularized to combine symbolic knowledge of Knowledge Graphs. This work evaluates the quality of graph-regularized word-embeddings concerning the baseline through word-embeddings probing tasks.
File