Verifying the proximity and size hypothesis for self-organizing maps

Abstract

The Kohonen Self-Organizing Map (SOM) is an unsupervised learning technique for summarizing high-dimensional data so that similar inputs are, in general, mapped close to one another. When applied to textual data, SOM has been shown to be able to group together related concepts in a data collection and to present major topics within the collection with larger regions. Research in which properties of SOM were validated, called the Proximity and Size Hypotheses,is presented through a user evaluation study. Building upon the previous research in automatic concept generation and classification, it is demonstrated that the Kohonen SOM was able to perform concept clustering effectively, based on its concept precision and recall7 scores as judged by human experts. A positive relationship between the size of an SOM region and the number of documents contained in the region is also demonstrated.