무작위 표본에 대한 코퍼스 언어학적 연구 = A Corpus-linguistic Approach to Random Samples

다국어 초록 (Multilingual Abstract)

In quantitative studies, a randomsample is supposed to be randomly selected by probability sampling in sucha way that it represents a population. The statistical analysis of corpus frequencydata is based on a random sample model, which assumes that the corpus wasrandomly selected from the language. However, Kilgarriff (2005), Evert (2006),Goh (2011) show that typical corpus data severely violate the randomnessassumption. This paper aims to evaluate random sampling methods for corpuslinguistics and to explore their characteristics and applicability. They are evaluatedon the relative frequencies of 30 morphemes and the frequencies of all morphemetypes which occur in each sample observed from 1,000 resampling trials basedon how close each random sample is to the normal distribution and theZipf-Mandelbrot (Mandelbrot 1977) law. The present study creates three findings. First, systematic sampling at the unit of measurement, i.e. individual words froman entire corpus is a best way to construct random samples for corpus linguistics. Second, the closer the relative frequencies of 30 morphemes in a sample lieto the normal distribution, the closer the frequency distribution of all morphemetypes to the Zipf-Mandelbrot distribution. Third, It is an effective way to utilizerandom samples for solving problems that stem from different sample size anddata sparseness. Moreover, using them facilitates detecting rather big differencein word frequencies obtained from different corpora.

참고문헌 (Reference)

• 1 고광윤, "참조 코퍼스와 키워드 목록의 구성내용" 한국영어학회 11 (11): 787-810, 2011
• 2 남윤진, "언어 정보의 탐구 1" 연세대학교 언어정보개발연구원 1999
• 3 정영미, "국어 어휘의 통계적 특성과 이의 응용" 연세대학교 언어정보개발원 5 : 134-163, 1995
• 4 Baayen, R. H., "Word Frequency Distributions" Kluwer 2001
• 5 Rodgers, J, "The bootstrap, the jackknife, and the randomization test: A sampling taxonomy" 34 (34): 441-456, 2000
• 6 Mandelbrot, B., "The Fractal Geometry of Nature" W. H. Freeman 1977
• 7 Evert, S., "Testing the extrapolation quality of word frequency models" 2006
• 8 Biber, D, "Representativeness in corpus design" 8 (8): 243-257, 1993
• 9 Kilgarriff, A, "Language is never, ever, ever, random" 1 (1): 263-276, 2005
• 10 Mouillot, D., "Introduction of relative abundance distribution (RAD) indices, estimated from the rank-frequency diagrams (RFD), to assess changes in community diversity" 63 (63): 279-295, 2000
• 11 Zipf, G., "Human Behaviour and the Principles of Least Effort" Addison-Wesley 1949
• 12 Tweedie, F., "How variable may a constant be? Measures of lexical richness in perspective" 32 : 323-352, 1998
• 13 Hinneburg, A., "How to handle small samples: Bootstrap and Bayesian methods in the analysis of linguistic change" 22 (22): 137-150, 2007
• 14 Evert, S, "How random is a corpus? The library metaphor" 54 (54): 177-190, 2006
• 15 Ha, L. Q., "Extension of Zipf's law to words and phrases" 315-320, 2003
• 16 Gries, S, "Exploring variability within and between corpora: Some methodological considerations" 1 (1): 109-151, 2007
• 17 Gries, S, "Dispersions and adjusted frequencies in corpora" 13 (13): 403-437, 2008
• 18 Baroni, M., "Corpus Linguistics: An International Handbook" Mouton de Gruyter 2009
• 19 Gries, S., "Converging evidence II: More on the association of verbs and constructions" CSLI 16 (16): 1-15, 2008
• 20 Kilgarriff, A, "Comparing corpora" 6 (6): 1-37, 2001
• 21 Evert, S., "A simple LNRE model for random character sequences" Louvain-la-Neuve 411-422, 2004