ETD system

Electronic theses and dissertations repository

 

Tesi etd-04152019-100924


Thesis type
Tesi di laurea magistrale
Author
PANDELEA, VLAD ALEXANDRU
URN
etd-04152019-100924
Title
Audio-Augmented Dialogue Systems
Struttura
INFORMATICA
Corso di studi
INFORMATICA
Supervisors
relatore Bacciu, Davide
relatore Cambria, Erik
Parole chiave
  • Multi-modality
  • Language generation
  • Audio features
  • Dialogue systems
Data inizio appello
03/05/2019;
Consultabilità
Secretata d'ufficio
Data di rilascio
03/05/2089
Riassunto analitico
Research on building dialogue systems able to converse with humans naturally has recently attracted a lot of attention. Most work on this area assumes text-based conversation, where the user message is modeled as a sequence of words in a vocabulary. Real-world human conversation, in contrast, involves other modalities, such as voice, facial expression and body language, which in certain scenarios can have a significant influence on the conversation.

In this work, we explore the impact of incorporating the audio features of the user message into the dialogue system. Specifically, we first design an auxiliary response classification task to refine raw audio features. Then we use word-level modality fusion to incorporate the audio features as additional context in our main generative model. Experiments show that our audio-augmented model outperforms the audio-free counterpart on perplexity, response diversity and human evaluation.
File