Assessing the predictive ability of a deterministic model and a stochastic model

Abstract

Formulary decision-makers must make choices based upon the safety, efficacy, and projected budgetary impact of medications. Models used to predict cost impacts are rarely assessed to determine how accurately they predict treatment cost changes. The purpose of this research was to assess the ability of a decision analytic based deterministic model and a regression analytic based stochastic model to predict the average diabetes-specific costs incurred by a managed care organization during the 12 month period following the addition of metformin to an HMO formulary. The ability of the stochastic model to predict the average diabetes-related costs and total health care costs was also assessed. The deterministic model, a decision tree, was constructed within an equilibrium framework using literature-based probabilities and internal costs to predict the expected diabetes-specific costs. The estimate of the total diabetes-specific cost impact came within 5% of the actual costs. The model underestimated the diabetes-specific medical costs (predicted was 73% of actual) and overestimated the diabetes-specific pharmacy costs (predicted was 258% of actual). A regression model was constructed using medical and pharmacy claims data to predict the expected diabetes-specific, diabetes-related and total health care costs. The average total cost estimates produced by the total health care cost model were within 7% of the actual average costs incurred. The diabetes-related and diabetes-specific cost models produced estimates that were within 12% and 18% of the actual costs incurred, respectively. The total, diabetes-related, and diabetes-specific average medical costs produced by the regression models were within 6%, 50%, and 46% of the actual costs respectively. The total, diabetes-related, and diabetes-specific average pharmacy costs were within 20%, 45%, and 49% of the actual costs respectively. Further research is needed to determine the best way to construct a model to estimate the economic impact of adding a medication to the formulary. A decision tree constructed with internal data should be used to predict the disease-specific economic impact of adding a medication to the formulary when only medical and pharmacy claims data from the previous year are available. A regression model should be used to predict the total health care cost impact.