• Deep Learning
• Domain Adaptation
• Machine Learning
• Multi-task
• Neural Network
• NLP
• Software Development Domain

Nowadays, almost every aspect of life depends on reliable high-quality software so there is a high demand for software tools that could help this development process. One direction of improvement consists on the use of Deep Learning techniques but we might face problems related to limited labeled datasets available, model overfitting that prevents the effective generalization, and energy consumption for the training process. In this thesis, we investigate how Multi-task Deep Learning can tackle these issues in the software development domain applying it to tasks that involve the manipulation of English and four programming languages namely Python, SQL, C#, and Java. We adapt the Transformer model architecture, actual state-of-the-art for sequence-to-sequence manipulation problems, to seven supervised tasks and the self-supervised language model and we explore whether we get benefits on the training of single tasks compared to multiple tasks together. We show the performance of our models and we compare our results with state-of-the-art counterparts that solved the tasks with the same datasets. We conclude that, given enough computing resources, Multi-task Deep Learning with the Transformer architecture is a promising framework to deal with software development domain tasks. To the best of our knowledge, this is the first work that applies large-scale multi-task models to software development tasks that involve source code and self-supervised and supervised tasks.