Evidence Retrieval for Explainable Question Answering

Abstract

Explainability in machine learning remains a critical unsolved challenge that slows the adoption of machine learning systems in real-world applications. Machine learning approaches are widely applied to complex natural language processing tasks such as question answering (QA) where explainability directly impacts end users understanding and trust. This thesis is particularly focused on improving the explainability of question answering systems via textual evidence retrieval and explaining learned representations within QA systems. Evidence retrieval in question answering (QA) is necessary not only to explain the decisions but also improve QA performance. We present 3 simple but effective unsupervised techniques for retrieving evidence texts necessary for explaining the QA inference process : Relevance-Overlap-Coverage Retriever (ROCC), Alignment Retriever and Alignment based Iterative Retriever ({\bf AIR}). ROCC (a) maximizes the relevance of the selected sentences, (b) minimizes the overlap between the selected facts, and (c) maximizes the coverage of both question and answer. Alignment retriever computes similarity between query and evidence facts by computing cosine similarity of individual tokens in embedding space. AIR extends the alignment retriever by combining two techniques: (a) adding an iterative process that reformulates queries focusing on terms that are not covered by existing justifications, which (b) stops when the terms in the given question and candidate answers are covered by the retrieved justifications. We show that, when evidence retrieved by ROCC, Alignment retriever and {\bf AIR} are fed to state-of-the-art transformer based QA methods, we substantially improve the state-of-the-art QA performance on multiple QA datasets. We further improve the state-of-the-art performance by adding supervision for evidence retrieval and show several representational analyses of the supervised retrieval based QA model.