TR2021-103

Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition


Abstract:

Pseudo-labeling (PL) has been shown to be effective in semi-supervised automatic speech recognition (ASR), where a base model is self-trained with pseudo-labels generated from unlabeled data.
We present momentum pseudo-labeling (MPL),
a simple yet effective strategy for semi-supervised ASR.
MPL consists of a pair of online and offline models that interact and learn from each other, inspired by the mean teacher method.

 

  • Related Publication

  •  Higuchi, Y., Moritz, N., Le Roux, J., Hori, T., "Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition", arXiv, June 2021.
    BibTeX arXiv
    • @article{Higuchi2021jun,
    • author = {Higuchi, Yosuke and Moritz, Niko and Le Roux, Jonathan and Hori, Takaaki},
    • title = {Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition},
    • journal = {arXiv},
    • year = 2021,
    • month = jun,
    • url = {https://arxiv.org/abs/2106.08922}
    • }