FACTORS AFFECTING THE REDUCTION OF NARRATIVE DATA.

Abstract

Narrative data enable evaluators to understand other people's viewpoints without predetermining those viewpoints by using preselected questionnaire categories. Narrative data yield rich detail, insight, and information. However, reducing narrative data into meaningful conclusions is difficult and time consuming, and requires attention, commitment, and skill on the part of trained coders. The personal and situational characteristics of the coders (called value inertia and cognitive limitation biases) affect data reduction. The effects of coder exposure to expected project outcomes and the level of coder research methodology sophistication were investigated. Coders considered either sophisticated or naive in research methodology were exposed to positive, ambiguous, or negative project outcome expectations. The coders reduced, or categorized, 25 open-ended interview response sets into previously established positive, negative, and ambiguous statement-type, content-code categories. The effectiveness of coder training was also explored by computing generalizability (reliability) coefficients. High generalizability coefficients were found, regardless of level of exposure to project outcome expectations. This indicates that coders were able to code the same statements the same way and is an indication of the coders' ability to reproduce the results. Results of this study also indicate that evaluators should use sophisticated coders for the reduction of narrative data, given that option. Sophisticated coders appear more resistant to the effect of exposure to project outcome expectations, coding narrative data more positively with less variability than naive coders when exposed to positive outcome expectations.