A preliminary investigation of image indexing: The influence of domain knowledge, indexer experience and image characteristics

Abstract

This study concerns image indexing and the impact of indexer experience levels and subject expertise on interindexer consistency and term selection. While the inherent complexities of applying terms to images are broadly acknowledged few studies have addressed interindexer consistency of visual materials. Two studies to investigate this topic are those of Markey (1984) and Wells-Angerer (2005). Markey's investigation looked at the indexing terms applied by thirty-nine individuals to one hundred images of medieval works on three different categories (objects, expressional, events). A low percentage of agreement of terms was reported by Markey, with an average of seven percent for exact term matches, and thirteen percent for conceptual matches in indexing terms. In a study assessing the influence of indexer subject knowledge on image retrieval rates of online museum collections Wells-Angerer (2005) investigated the terms applied to ten works of art by thirty participants falling into three categories of image indexers (expert, knowledgeable, novice). Wells-Angerer found the terms applied by indexers with the highest level of knowledge about the objects in the collections (scholars, curators and collection staff) had retrieval success rates of approximately sixteen percent. Indexer retrieval rates for those who had less subject knowledge were considerably lower, at approximately five percent (Wells-Angerer, 2005). The results of this investigation indicate that indexer experience and subject expertise ought to be considered in discussions of interindexer consistency. Markey's study has been used on several occasions to support the hypothesis that image indexing produces low returns for the effort involved in the work. This is remarkable as Markey (1984) states that '[t]he use of inexperienced indexers and non-subject specialists in this study may have diminished interindexer consistency scores.' The limited number of studies investigating the practices of image indexers and the conflicting results of these two studies indicate additional research is warranted in the area of image indexing. Thus, the present study was undertaken in order to explore some of the issues at work which influence image indexing. Using a Web-based questionnaire 140 participants provided demographic data and indexing terms for eight images. Images of cultural works formed the focus of the study. Several documentary style photographs were included, however, to assess the influence of an image's subject accessibility and mode of representation on the terms chosen by the study's participants. For data analysis purposes the participants were divided into several groups according to their subject expertise (2 or less courses or 11 or more courses with an art/cultural focus) or their professional experience (self identification as an image indexer). The data collected from the participants has been analyzed using qualitative and descriptive statistics. Further analysis of the data using quantitative methods is in process. Subject expertise and indexing experience were found to have an impact on the terms applied to images. The number of terms applied and the co-occurrence of terms was related to the level of indexing experience and subject expertise of participants. On the most basic level of analysis, the experienced image indexers provided on average the highest number of terms per image, with the subject experts supplying a slightly reduced number and the subject novice participants the fewest. Co-occurrence of applied terms among participant groups also followed this pattern. In addition, the images themselves were found to have an influence on the number of terms applied and the interindexer consistency achieved by the indexers of these images. The legibility of images with easily accessible subjects and realistic representation, while scoring well in terms of interindexer consistency were found to receive fewer term applications by the image indexers and the subject experts. This finding suggests that while interindexer consistency might be highest among skilled indexers and those with solid domain knowledge, a broader range of terms were sometimes applied to images with readily accessible subjects by those individuals who lacked training or subject expertise. Other interesting findings of the study point to the various kinds of terms applied by the three groups. The subject novices applied a greater number of generic terms to the images with the indexers and subject experts providing a higher number of terms which identified specific features of the image. Finally, while the number of emotive or interpretive terms applied to the images was found to be very low across all three groups the subject novices applied these terms more often than the other participant groups. The results of this study provide a preliminary account of the influence of subject knowledge and indexing experience on image indexing.