The last two years see the great advance in training general purpose language representation models using the enormous amount of unannotated text on the web, known as pre-training. The pre-trained model can then be fine-tuned on small-data NLP tasks, resulting in substantial accuracy improvements compared to training on these datasets from scratch. In this work we provide an overview of the challenging fake news detection problems viewed as a range of computational linguistic tasks and present the improved results of Fake News Detection Challenge (2017)
obtained by leveraging publicly available pre-trained transformers like BERT, RoBERTa and XLNet.