Unsupervised fine-tuning data selection for ASR using self-supervised speech models

Simple item page
dc.contributor.advisorHarwath, David
dc.creatorGoudy, Reem A.
dc.creator.orcid0000-0001-7680-8928
dc.date.accessioned2023-04-06T21:26:01Z
dc.date.available2023-04-06T21:26:01Z
dc.date.created2022-12
dc.date.issued2022-12-01
dc.date.submittedDecember 2022
dc.date.updated2023-04-06T21:26:02Z
dc.description.abstractSelf-supervised learning (SSL) has been able to leverage unlabeled data to boost the performance of automatic speech recognition (ASR) models when we have access to only a small amount of transcribed speech data. However, this raises the question of which subset of the available unlabeled data should be selected for transcription. Our work investigates different unsupervised data selection techniques for fine-tuning the HuBERT model under a limited transcription budget. We investigate the impact of speaker diversity, gender bias, and topic diversity on the downstream ASR performance. We also devise two novel techniques for unsupervised data selection: pre-training loss based data selection and the perplexity of byte pair encoded clustered units (PBPE) and we show how these techniques compare to pure data selection. Finally, we analyze the correlations between the inherent characteristics of the selected fine-tuning subsets as how these characteristics correlate with the resultant word error rate. We demonstrate the importance of token diversity, speaker diversity, and topic diversity in achieving the best performance in terms of WER.
dc.description.departmentComputer Science
dc.format.mimetypeapplication/pdf
dc.identifier.urihttps://hdl.handle.net/2152/117812
dc.identifier.urihttp://dx.doi.org/10.26153/tsw/44691
dc.language.isoen
dc.subjectSelf-supervised learning
dc.subjectUnsupervised data selection
dc.titleUnsupervised fine-tuning data selection for ASR using self-supervised speech models
dc.typeThesis
dc.type.materialtext
thesis.degree.departmentComputer Sciences
thesis.degree.disciplineComputer Science
thesis.degree.grantorThe University of Texas at Austin
thesis.degree.levelMasters
thesis.degree.nameMaster of Science in Computer Sciences

Access full-text files

Original bundle

Now showing 1 - 1 of 1
Thumbnail Image
Name:
GOUDY-THESIS-2022.pdf
Size:
1.09 MB
Format:
Adobe Portable Document Format
Download

License bundle

Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
4.45 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
LICENSE.txt
Size:
1.84 KB
Format:
Plain Text
Description:
Download

Collections

UT Electronic Theses and Dissertations