• benchmark
• large language models
• question-answering
• RAG
• retrieval augmented generation
• semi-structured documents

The main objective of this thesis is to explore the state of the art to develop a system capable of handling structured and unstructured data, answering in seconds with accuracy and without “hallucinations”. In this context, a specific benchmark was created for the evaluation of Question Answering tasks, combining text, metadata and qualitative language feature analysis. In addition, new architectures based on Retrieval-Augmented Generation (RAG) were proposed to improve performance on the benchmark.
The results obtained showed that the current state of the art offers solid performance even in complex scenarios, but new approaches have also been identified that are capable of improving on the initial state-of-the-art baseline. However it was highlighted that the choice of the best solution depends on the application context. This work could be used as a starting point for new studies, such as developing a multi-modal system, exploring the potential of a hybrid RAG and fine-tuning approach, or designing new architectures that further improve natural language understanding and generation capabilities.