LARGE LANGUAGE MODELS TO IMPROVE HEALTH LITERACY WITH IMAGES

Abstract

A lack of health literacy is a systematic problem in health care that increases costs for patients and hospitals while resulting in increased negative health outcomes. Medical text comprehension and recall have become a paramount issue, especially with the advent of widely accessible medical content online. Text with accompanying explanatory images has been shown to significantly increase reader understanding (Houts et al. 2006). Unfortunately, a vast majority of medical text does not have corresponding images(Agrawal et al. 2011). This thesis first explores Large Language and Diffusion text-image models' ability to generate effective explanatory images for text. Two types of characteristics were evaluated for their impact on automated image generation: sentence concreteness and sentence length. It was found that both of these variables are significant determinants of image quality. Second, a pilot study created an initial framework that improved on the first approach with better models, actionable prompts, and a feedback loop for automating image refinement. It was found that the image quality was improved, however, automated, existing metrics, e.g., ROUGE and BLUE, were not good indicators of image quality.