A Quantitative-Qualitative Evaluation Framework for Multi-Level Data-Driven Agent-Based Simulation

Abstract

Dynamic data driven application system (DDDAS) is a paradigm, whereby executing application simulations and real-time measurements form a symbiotic feedback control system. DDDAS enables dynamic data incorporation for online optimization, which facilitates the accuracy and reliability of analysis and prediction. For data fusion and multi-scale model development, the agent-based model (ABM) is an effective and flexible method to integrate both micro-level details and meso-level or macro-level mechanisms. The ABM model allows researchers to develop a model based on both the bottom-up individual behavioral data and the top-down policy or assumptions. While the ABM has been widely used in various fields, such as public health, transportation, economics, and sociology, some common challenges and limitations have been identified: 1) model parameter calibration, 2) validation and interpretation of simulation results, and 3) heavy computational load. By emphasizing the internal model structure and underlying statistical mechanisms, the goal of this research is to develop a novel evaluation framework for the model developer to prepare the model at the model building stage by 1) optimally deciding the model complexity and the connection among different model components, 2) analyzing the value and range of appropriate input parameters to obtain convincing results with fewer simulation replications, by avoiding unreliable extreme cases, and 3) generating interpretable results or outcomes for intervention or scenario settings. We mainly consider two aspects of ABM (viz. parameter configuration and model structure) and, by focusing on them, develop a quantitative-qualitative evaluation framework for ABM users to build a logic-rigorous and computation-friendly model with expert knowledge and empirical data, especially for complex systems involving multiple components and multiple (e.g., micro-macro) scales. While the proposed evaluation framework is intended to be generic and applicable to various applications, we illustrate the proposed framework for 1) Nowcasting: breath-cycle-to-long-term respiratory and physiological response simulation model, 2) Forecasting: classroom- mobility-to-campus COVID transmission model, and 3) Scenario simulation: individual-touch-to-classroom MRSA transmission model in this research.