Summary of the project

Assessing the veracity of claims is a vital capability in the modern world, but it is a task that the public is often ill-equipped to do. This is evidenced, for example, in debates around human contribution to climate change, public health policies and political elections. As a result of targeted disinformation campaigns, many users are inadvertently spreading misinformation, without critically reflecting on its sources, as the information is often presented without further context. As experts cannot provide contextualising explanations about the validity of a claim instantaneously, this gives the opportunity to research automated claim and fact verification approaches.

State-of-the-art research on fact verification mostly focuses on the capability to identify misleading claims. However, for end-users, it is important to provide explanations why exactly a claim was identified as wrong. These explanations serve both as a context for the claim and as an insight into the reasoning process that led to the veracity decision. Existing fact verification approaches rely on deep learning-based models optimised on large static datasets to automatically classify whether a claim is true based on retrieved supporting evidence. It is however unclear, whether end-users will accept these verdicts without further context. This is further problematic, as these models have been shown to exhibit biases inferred from the datasets they were optimised upon, for example relying on the appearance of specific keywords.

Instead, we take a novel approach to developing AI-driven fact verification systems, with explainability at their core. We will leverage the large availability of high-quality human-written claim verification articles from specialised journalist outlets such as Full Fact to create a dataset for the task of explainable fact verification. We will use the data to develop a fact verification system underpinned by deep learning-based, generative language models that will generate human-understandable explanations that contextualise their verdicts.

The dataset collection will follow the paradigm of iterative crowdsourcing, where annotators not only annotate data to train neural networks but also rate the quality of the predictions of models optimised on the previous iteration. Thus, we will shift the focus to an evaluation methodology that revolves around the explainability requirements of humans. This will allow us to iteratively refine the developed tool and subsequently learn to judge the quality of explanations from human annotations.