The utility of the propensity score in enhancing comparisons across multiple years of surveys

Abstract

The primary aim of the study was to determine if the propensity score data analytic procedure was sufficient to remove the sampling bias that was thought to be present in a large multiple-year smoking prevalence survey. The survey under examination was the Adult Tobacco Survey (ATS) administered in the state of Arizona in 1996, 1999, and 2002 by the Tobacco Education and Prevention Program. Information provided by the ATS is used to obtain estimates of smoking prevalence and determine attitudes and beliefs about smoking. In an attempt to obtain more relevant information, the ATS was revised between the 1999 and 2002 survey administrations; questions were added or removed, the order in which questions were presented was modified, and question wording was altered. In addition, slight changes in research protocol throughout the years had resulted in different sampling procedures and inclusion/exclusion criteria. It was suspected that these differences between the survey years had resulted in sampling bias or selection error. In an attempt to control for the sampling bias, two propensity score analyses were conducted that examine differences between survey years as well as differences due to variations in the surveys. It was anticipated that examining participants that had been stratified and matched using the calculated propensity scores would facilitate the comparison of groups that were originally nonequivalent. After stratifying and matching the participants, the two propensity analyses resulted in statewide prevalence estimates that were similar for the three years and revealed a steady decline in smoking prevalence. Based upon the results obtained with this investigation, it was concluded that the propensity score performs adequately to remove sampling bias in large multiple-year surveys.