Detecting Word Transfer in Open Domain Question Answering

• Author / Creator

  Yadegari, Mostafa

• There is essential information in the underlying structure of sentences and the relationships between words and phrases in natural language questions, and the use of this information has been extensively studied. This thesis studies the problem of word transfer from questions to answer passages in the context of open-domain question answering. Word transfer happens for both terms that are explicitly mentioned in questions and those that may be implied. On the same basis, this thesis is broken down to two parts.

  In the first part of the thesis, we study one particular structure, referred to as {\em frozen phrases}, that is highly expected to transfer as a whole from questions to answer passages. Frozen phrases, if detected, can be helpful in open-domain Question Answering (QA) where identifying the localized context of a given input question is crucial. To identify those phrases, we cast the problem as a sequence-labeling task and create synthetic data from existing QA datasets to train a model. We further plug this model into a sparse retriever that is made aware of the detected phrases. Our experiments reveal that detecting frozen phrases whose presence in answer documents are highly plausible yields significant improvements in retrievals as well as in the end-to-end accuracy of open-domain QA models.

  In the second part, a query expansion method is introduced to predict the terms that fall outside of a question but are expected to be found in the answer passages. For this task, we explore the capacity of modern language models under few-shot in-context learning. Our evaluation reveals that the proposed method is quite effective, achieving a new state-of-the-art unsupervised query expansion on Natural Questions dataset.

• Subjects / Keywords

  • Open-domain question answering
  • Information retrieval
  • Large language models
  • Query expansion
  • Unsupervised retrieval
  • Frozen phrases
  • Sparse retriever
  • Question paraphrasing

• Graduation date

  Fall 2023

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-k2pp-yj07

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.