Social Semiotics and Literacy: How Refugee-Background Adult Second Language Learners With Emerging Literacy Make Meaning in Multimodal Assessment Texts

Abstract

This dissertation contributes to the fields of applied linguistics and literacy studies by considering the complex meaning-making processes of adults from refugee backgrounds as they navigate new textual, linguistic, and educational landscapes. Meaning-making as it is understood here involves both perception and production; it is inherently dialogical, and bound in social semiotic systems, which are not only linguistic but multimodal (Kress, 1994). Making meaning from multimodal texts requires understanding headings, directions, images, graphic devices, top/down and left/right organization, and the relationships among such elements. Taking these complications as a starting point, this research focuses on refugee-background adult second language learners, specifically, those with emerging literacy or who (have) experienced interruptions in their formal, school- based education. Such learners are becoming literate while simultaneously learning the language their literacy is developing in. For these reasons, the texts that are central to their experiences as learners of a new language – particularly language and literacy assessments – are of considerable importance for understanding the intersecting dimensions at play when people learn how to make meaning in a new language. While there is a growing body of research that has examined the psycholinguistic aspects of adult second language learners’ literacy development (e.g., Kurvers, 2002; Tarone et al., 2009; Young-Scholten & Naeb, 2010), many questions remain about the social semiotics of literacy – the interplay of context, culture, history, text, and meaning-making – for adults with emerging literacy or interruptions in their education. Moreover, little research to-date investigates the connections between social semiotics and the visual and multimodal literacies of this population (Altherr Flores, 2017; Bruski, 2012; Whiteside, 2008). This is problematical because many materials designed for beginning second language learners rely heavily on visual cues; without a comprehensive understanding of how such cues are being interpreted, the field’s understanding of how diverse populations make meaning from multimodal texts is compromised. Such knowledge is crucial for designing tests and other materials that aim to support learning. Building on prior scholarship, including an earlier pilot study by the author (Altherr Flores, 2017), this dissertation focuses on the role of visual literacy, language, and lived experience in multimodal texts that are used with adult second language learners with emerging literacy or interrupted education backgrounds. The core data for this research were: 1) English language and English literacy assessments – both in-house assessments used by a local program, and two experimental versions of assessments, created through iterative design as part of the research, 2) textual artifacts, and 3) semi-structured interviews with participants enrolled in community language and literacy classes. The analyses use a critical multimodal social semiotic approach (Kress, 2010; Kress & van Leeuwen, 2006; Pennycook, 2001) to examine the underlying assumptions presented in key texts’ visual and linguistic design, and investigate how this population understands and engages with these multimodal texts. The findings showed assumptions of multimodal design and visual literacy, and assumed content and referential background schemata in the design of the original assessment texts. In particular, the study exposed tensions between participants’ responses to textual and visual prompts and the expectations of test designers. The interview data further revealed the self-articulated strategies participants use to make meaning in multimodal texts, often relying on their lived experiences. By approaching the participants’ meaning-making practices as creative and complex, the research was able to show that the participants often relied on multimodal aspects of the test design that were taken for granted by the test designers. The participants often drew from their lived experiences, and also approached the assessments as a dialogue with the instructor, thus bringing shared frames of reference into play that would potentially be missed by an external evaluator. This study provides insight into how beginning language and literacy learners from refugee backgrounds make meaning from the verbal and visual aspects of assessment materials, and demonstrates how both textual composition and assessment practices may be inadvertently biased against individuals with vastly different literacy experiences. In addition to potentially helping test and materials developers to rethink their design choices, this study expands understandings of what it means to be literate by laying bare the creative and multimodal dimensions of engagement with even the most quotidian types of texts. The study’s results are beneficial for multimodal materials development and assessment practices in learning environments. The results also highlight sociopolitical issues in assessment of this population, and raise questions to be considered concerning assessment in higher stakes environments such as the U.S. naturalization test.