인공지능을 이용한 국악 멜로디 생성기에 관한 연구 = Study on the Korean Traditional Music Melody Generator Using Artificial Intelligence

다국어 초록 (Multilingual Abstract)

Machine learning research on image recognition and processing has been actively conducted over the past decade. In the field of music, various methods using machine learning have been recently attempted, showing the possibility of creating melody and harmony that humans cannot make. However, most of the research on artificial intelligence composition has been centered on Western music, and little research has been done on Korean traditional music. In particular, there were many difficulties in research because even a data set for research was not created.

Therefore, in this paper, we will create a data set of Korean traditional music, create a Korean traditional music melody using three algorithms based on the data set, and compare the results. The purpose of this thesis is to better understand the structure and characteristics of Korean traditional music through big data analysis of Korean traditional music, and to create a Korean traditional music melody generator model using artificial intelligence based on it.

The criterion for the selection of algorithms and architectures was how well the algorithms exhibited excellent performance in deriving results suitable for creating Korean traditional music melodies. As a result of reviewing the existing research algorithms, it was found that a deep learning algorithm based on text is suitable for generating musical melodies. As a result, LSTM, Music Transformer and Self Attention 3 models based on the similarity between language and music were selected.

It is necessary to pre-process the data in order to learn Korean traditional music data by inputting it into a neural network. To this end, Korean songs were converted into MIDI, and they were converted into vectors using the Music21 library.

Using each of the three models, a Korean traditional music melody generator was modeled and trained to generate a Korean traditional music melody. When comparing the three methods, the self-attention model was the model that produced the melody that was closest to the feeling of Korean traditional music. The LSTM model produced a melody that resembles Arirang, one of the training data, too repetitively, and it was not able to properly express the pros and cons of Korean traditional music. The music transformer model used 10 tone scales, which used a lot of semitones, unlike traditional traditional music, which mainly used 5 tone scales, and the resulting melody was reminiscent of the piano sonata of Western classical music.

Since the evaluation of music is not objective but depends on individual preferences, it is necessary to perform user evaluation rather than accuracy evaluation, so a user survey was conducted. The results of the melodies of Korean traditional music of the three models were converted into sound sources and played to people, and the score was evaluated on a 5-point scale to determine which songs best used the feeling of Korean music. As a result of user evaluation, a difference of 0.76 points compared to the Self Attention method from LSTM method and 0.67 points from the Music transformer method showed high preference.

Data representation and training data are very important in artificial intelligence composition. For this, a basic Korean music data set was created, and artificial intelligence composition was attempted with various algorithms, and this is expected to be helpful in future research on artificial intelligence composition for Korean traditional music.

목차 (Table of Contents)

• 1. 서 론 1
• 1.1 연구의 배경 1
• 1.2 연구의 목적 2
• 2. 기초연구 4
• 2.1 신경망에 대하여(Neural Networks) 4
• 2.1.1 Feedforward Neural Network (FFNN) 6
• 2.1.2 Simple Recurrent Network (SRN) 7
• 2.1.3 Recurrent Neural Network (RNN) 8
• 2.1.4 Long Short-Term Memory (LSTM) 10
• 2.1.5 Restricted Boltzmann Machine (RBM) 11
• 2.1.6 Autoencoder (AE) 13
• 2.1.5 Variational Autoencoder (VAE) 13
• 2.1.6 Convolutional Neural Network (CNN) 14
• 2.1.7 Generative Adversarial Network (GAN) 15
• 2.2 국악기초 이론 16
• 2.2.1 율명(律名) 16
• 2.2.2 장단 17
• 2.2.3 음계 18
• 2.2.4 선법 19
• 2.2.5 조(調) 20
• 2.2.6 농현과 요성 21
• 2.2.7 조직(組織) 22
• 3. 기존 알고리즘 작곡 연구 23
• 3.1 알고리즘 작곡의 역사 24
• 3.2 컴퓨터를 이용한 알고리즘 작곡 26
• 3.2.1 알고리즘 작곡 26
• 3.2.2 신경망 이외의 방식 27
• 3.2.2.1 단순 방식 27
• 3.2.2.2 유전 알고리즘(Genetic Algorithms) 28
• 3.2.2.3 마르코프 모델(Markov Model) 29
• 3.2.3 신경망을 이용한 방식 30
• 3.2.3.1 방식 및 목적에 따른 분류 32
• 3.2.3.2. 범주(Domain)에 따른 분류 35
• 3.2.3.3 음악적 범주(Musical Domain)에 따른 분류 37
• 3.2.3.4 음악 장르(Genre)에 따른 분류 39
• 3.2.3.5 입력 표현(Input Representation)에 따른 분류 40
• 4. 기존 인공지능 작곡 모델 및 아키텍쳐 43
• 4.1 신경망 아키텍처의 종류 43
• 4.1.1 Feedforward Neural Network (FFNN) 모델 43
• 4.1.2 Simple Recurrent Neural Network (SRN) 모델 44
• 4.1.3 Recurrent Neural Network (RNN) 모델 45
• 4.1.4 Autoencoder (AE) 모델 46
• 4.2 기존 연구의 한계점 48
• 5. 방법론 49
• 5.1 알고리즘 및 아키텍쳐 선택 49
• 5.1.1 Long Short-Term Memory (LSTM) 모델 50
• 5.1.2 Music Transformer 모델 51
• 5.1.3 Self Attention 모델 53
• 5.2 국악 데이터 전처리 (Data Preprocessing) 55
• 5.2.1 MIDI 변환 56
• 5.2.2 토큰화(Tokenization) 58
• 5.2.3 데이터 가공 및 표현 59
• 5.3 학습 및 모델 생성 61
• 5.3.1 LSTM을 이용한 국악 멜로디 생성기 모델링 61
• 5.3.2 Music Transformer 을 이용한 국악 멜로디 생성기 모델링 68
• 5.3.3 Self Attention을 이용한 국악 멜로디 생성기 모델링 71
• 6. 실험 및 결과 79
• 6.1 국악 데이터 세트 분석 79
• 6.1.1 국악 데이터의 히스토그램 분석 79
• 6.1.2 Pitch Class 분석 83
• 6.2 AI 국악 멜로디 생성기 모델별 결과 비교 86
• 6.2.1 LSTM 모델 86
• 6.2.2. Music Transformer 모델 89
• 6.2.3 Self Attention 모델 93
• 6.3 모델별 정확도 및 손실률 비교 96
• 6.4 모델별 청취자 선호도 평가 97
• 6.4.1 MIDI 파일 음원화 작업 97
• 6.4.2 청취자 평가 결과 99
• 7. 결론 및 향후 과제 101
• 7.1 결론 101
• 7.2 향후 과제 103
• 참 고 문 헌 105
• ABSTRACT 112