Assessing and Mitigating Social Bias in Text and Natural Language Processing Systems

• Author / Creator

  Ding, Lei

• This thesis presents a comprehensive exploration of social biases embedded within texts and Natural Language Processing (NLP) models. It develops innovative algorithms to evaluate and mitigate these biases, thereby enhancing the fairness and effectiveness of NLP applications. The initial phase of the research introduces a novel method for reducing gender bias in static word embeddings, meticulously designed to preserve maximum semantic integrity and explainability. This approach not only achieves state-of-the-art results in gender debiasing tasks but also enhances performance in word similarity evaluations and various downstream NLP tasks.

  Expanding the scope, subsequent sections delve into broader evaluations of social biases. A new evaluation framework employing Masked Language Models is introduced, which quantitatively assesses social bias using validated inventories of social cues and words, enabling a systematic linguistic analysis. This framework was applied in a large-scale evaluation of the ChatGPT model in high-stakes environments such as the job market. Our findings reveal how the increasing use of generative AI by both employers and job seekers can reinforce gender and social disparities through biased language.

  The final section proposes a statistical hypothesis-testing framework to detect biases in texts generated by MLMs. This unsupervised approach uses sentence perturbation techniques to facilitate effective bias testing across various linguistic contexts. Empirical validation confirms its ability to identify subtle biases, enhancing the framework's practical utility and effectiveness.

  Together, these investigations provide a series of comprehensive, effective, and efficient algorithms for studying social bias in textual contexts. They offer valuable insights and practical tools for future researchers and significantly advance the state of the art in NLP research. This thesis contributes to academic knowledge and represents a crucial step toward creating more equitable technological solutions in language processing.

• Subjects / Keywords

  • NLP
  • Social Bias

• Graduation date

  Fall 2024

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/r3-s1rm-bc07

• License

  This thesis is made available by the University of Alberta Library with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.