Discrete Recurrent Event Data Analytics - An Intensity Decomposition Approach

Abstract

In complex natural or social systems, events of interest to specific stakeholders may occur randomly and repeatedly over time and/or space, which is termed discrete recurrent events. These events are often generated based on complicated and latent mechanisms, where the occurrence of one event can influence the likelihood of future events. Given all of the known factors, these mechanisms make the events that occurred at different times and locations not independent, which violates the common independent assumption of many existing models and cannot be modeled by these models. This research aims to establish a unified, quantifiable, and interpretable intensity decomposition modeling framework to analyze the underlying mechanisms that drive the occurrences of the events. The ultimate goal is to improve the accuracy of event occurrence predictions and provide actionable insights for better system management. In particular, this research makes efforts to solve the following three research problems: Generic, Quantifiable, and Interpretable Intensity Decomposition. This research is motivated by the study of autonomous vehicle (AV) reliability, where errors occurring in one module can propagate to interconnected modules, potentially leading to system failures. This phenomenon, referred to as a triggering mechanism, manifests as error propagation (EP) within AV and significantly affects the reliability of AV. Accurately modeling and quantifying the triggering mechanism is essential, yet remains challenging due to the latent nature of EP. To address this challenge, this research formulates a generic intensity decomposition framework comprising two components: (i) the baseline intensity, which captures the primary error rate stemming from imperfect performance in the current module, and (ii) the triggering intensity, which accounts for the propagated error rate induced by errors from other modules. In order to quantify the latent EP, a modified expectation-maximization (EM) algorithm is proposed for model estimation. Covariate-Adjusted Learning of the Latent Triggering Mechanism. In addition to the triggering mechanism, event occurrences may also be affected by a wide range of covariates. In the context of healthcare, individuals with opioid use disorder are at risk of experiencing an overdose event, which is often associated with various factors such as age, sex, and the amount of substance used. Moreover, due to the chronic nature of the condition, a past overdose can increase the likelihood of subsequent overdoses. This work extends the intensity decomposition framework by incorporating covariate effects into the baseline intensity, allowing for the simultaneous modeling of both covariate and event-triggering dependency effects. Computationally Efficient and Theoretically Guaranteed Learning of the Latent Triggering Mechanism. While the methods developed above have demonstrated effectiveness in relatively small-scale datasets, they face estimation challenges in large-scale datasets. For instance, the AV processes large volumes of data within a short time period, leading to the generation of extensive errors during their operation. Estimating the latent EP from extensive errors is challenging. The third work aims to develop a computationally efficient and theoretically guaranteed estimation algorithm to learn the latent triggering mechanism from large-scale datasets.