Transformer 네트워크를 이용한 음성신호 변환

국문 초록 (Abstract)

음성 변환은 다양한 음성 처리 응용에 적용될 수 있으며, 음성 인식을 위한 학습 데이터 증강에도 중요한 역할을 할 수 있다. 기존의 방법은 음성 합성을 이용하여 음성 변환을 수행하는 구조를 사용하여 멜 필터뱅크가 중요한 파라미터로 활용된다. 멜 필터뱅크는 뉴럴 네트워크 학습의 편리성 및 빠른 연산 속도를 제공하지만, 자연스러운 음성파형을 생성하기 위해서는 보코더를 필요로 한다. 또한, 이 방법은 음성 인식을 위한 다양한 데이터를 얻는데 효과적이지 않다. 이 문제를 해결하기 위해 본 논문은 원형 스펙트럼을 사용하여 음성 신호 자체의 변환을 시도하였고, 어텐션 메커니즘으로 스펙트럼 성분 사이의 관계를 효율적으로 찾아내어 변환을 위한 자질을 학습할 수 있는 transformer 네트워크 기반 딥러닝 구조를 제안하였다. 영어 숫자로 구성된 TIDIGITS 데이터를 사용하여 개별 숫자변환 모델을 학습하였고, 연속 숫자 음성 변환 디코더를 통한 결과를 평가하였다. 30명의 청취 평가자를 모집하여 변환된 음성의 자연성과 유사성에 대해 평가를 진행하였고, 자연성 3.52±0.22 및 유사성 3.89±0.19 품질의 성능을 얻었다.

다국어 초록 (Multilingual Abstract)

Voice conversion can be applied to various voice processing applications. It can also play an important role in data augmentation for speech recognition. The conventional method uses the architecture of voice conversion with speech synthesis, with Mel filter bank as the main parameter. Mel filter bank is well-suited for quick computation of neural networks but cannot be converted into a high-quality waveform without the aid of a vocoder. Further, it is not effective in terms of obtaining data for speech recognition. In this paper, we focus on performing voice-to-voice conversion using only the raw spectrum. We propose a deep learning model based on the transformer network, which quickly learns the voice conversion properties using an attention mechanism between source and target spectral components. The experiments were performed on TIDIGITS data, a series of numbers spoken by an English speaker. The conversion voices were evaluated for naturalness and similarity using mean opinion score (MOS) obtained from 30 participants. Our final results yielded 3.52±0.22 for naturalness and 3.89±0.19 for similarity.

목차 (Table of Contents)

• Abstract
• 1. 서론
• 2. 관련 연구
• 3. Transformer 네트워크 기반 음성대 음성 변환
• 4. 실험 결과 및 분석
• 5. 결론
• References
• 국문초록

참고문헌 (Reference)

• 1 Morise, M., "WORLD : A vocoder-based high-quality speech synthesis system for real-time applications" 99 (99): 1877-1884, 2016
• 2 Huang, W. C., "Voice transformer network: Sequence-to-sequence voice conversion using transformer with text-to-speech pretraining"
• 3 Liu, R., "Voice conversion with transformer Network" 7759-7759, 2020
• 4 Kim, J. W., "Vocoder-free end-to-end voice conversion with transformer Network"
• 5 Leonard, R. G., "Tidigits speech corpus" Texas Instruments 1993
• 6 Griffin, D., "Signal estimation from modified short-time Fourier transform" 32 (32): 236-243, 1984
• 7 Sutskever, I., "Sequence to sequence learning with neural networks" 3104-3112, 2014
• 8 Kwon, S., "Preprocessing for elderly speech recognition of smart devices" 36 : 110-121, 2016
• 9 Biadsy, F., "Parrotron: An end-to-end speech-to-speech conversion model and its applications to hearing-impaired speech and speech separation"
• 10 Li, N., "Neural speech synthesis with transformer network" 33 : 6706-6713, 2019
• 11 Shen, J., "Natural tts synthesis by conditioning wavenet on MEL spectrogram predictions" 4779-4783, 2018
• 12 McAuliffe, M., "Montreal forced aligner: Trainable text-speech alignment using Kaldi" 2017 : 498-502, 2017
• 13 Hochreiter, S., "Long short-term memory" 9 (9): 1735-1780, 1997
• 14 Radford, A., "Improving language understanding by generative pre-training"
• 15 Lee, J., "Fully character-level neural machine translation without explicit segmentation" 5 : 365-378, 2017
• 16 Chung, J., "Empirical evaluation of gated recurrent neural networks on sequence modeling"
• 17 Jia, Y., "Direct speech-to-speech translation with a sequence-to-sequence model"
• 18 Kim, Y., "Convolutional neural networks for sentence classification" 1746-1751, 2014
• 19 Schuster, M., "Bidirectional recurrent neural networks" 45 (45): 2673-2681, 1997
• 20 Devlin, J., "BERT:Pre-training of deep bidirectional transformers for language understanding"
• 21 Potamianos, A., "Automatic speech recognition for children" 2371-2374, 1997
• 22 Vaswani, A., "Attention is all you need" 5998-6008, 2017
• 23 Tanaka, K., "AttS2S-VC: Sequence-to-sequence voice conversion with attention and context preservation mechanisms" 6805-6809, 2019
• 24 Kingma, D. P., "Adam: A method for stochastic optimization"