关键词: LDA, 向量空间模型, 数据挖掘, 文本聚类, K-Means

Abstract:

In order to overcome the shortcomings and limitations of traditional clutering algorithms in dealing with largescale and high dimension text clustering, a text clustering method is presented based on weighted LDA(latent dirichlet allocation)model. Two distributions are obtained by LDA: the topic distribution and the word distribution of different topics hidden in the text, which are then combined as the text feature to obtain the final text similarity. Using the classic K-Means algorithm in both English and Chinese corpus, the experimental results show that compared with the pure VSMcombined with K-Means, this algorithm has better clustering effect.

Key words: LDA, vector space model, data mining, text clustering, K-means