Skip to Main Content

Search

Skip to Search Results
Filter
Close
Subject / Keyword
Collections
Supervisors
Author / Creator / Contributor
Year
to
Languages
Item type
Departments
Subject: BERT Collections: Graduate and Postdoctoral Studies (GPS), Faculty of Supervisors: Barbosa, Denilson (Computing Science)

  • Improving Bengali and Hindi Large Language Models

    Download

    Spring 2024

    Shahriar, Arif

    Bengali and Hind are two widely spoken yet low-resource languages. The state-of-the-art in modeling such languages uses BERT and the Wordpiece tokenizer. We observed that the Wordpiece tokenizer often breaks words into meaningless tokens, failing to separate roots from affixes. Moreover,...

1 - 1 of 1