High-Dimensional Spatial-Temporal Data Analytics via Knowledge-Informed and Interpretable Decomposition

Abstract

Over the past decade, rapid advancements in sensing and data storage technologies have dramatically increased the availability and scale of spatial-temporal (ST) data, offering new opportunities while presenting significant challenges. ST data, acquired from diverse sources, often exhibit complex patterns that can be decomposed into several interpretable components: (i) a global component remaining stable, (ii) a locally distinct feature component reflecting system-specific variations, and (iii) a random noise component accounting for unpredictable fluctuations. These decomposition principles are demonstrated across multiple application domains. In water distribution systems (WDSs), for example, pipeline bursts cause abrupt deviations in pressure that differ significantly from normal operating conditions. To detect such anomalies, an ST decomposition model is proposed that decomposes hydraulic measurements into a regular consumption component, a noise component, and a burst-induced anomaly component. Domain-specific knowledge, such as expected pressure behavior under normal and fault conditions, is integrated through information fusion to guide the decomposition, enhancing detection accuracy and robustness in real-world scenarios. In surveillance systems, the objective is to identify moving foreground objects, such as pedestrians and vehicles, from surveillance video frames. To achieve this, a foreground detection model is proposed that decomposes video frames into background, foreground, and noise components, each regularized by distinct ST properties. For instance, static backgrounds under fixed cameras, continuously moving foreground objects, and random noise are explicitly modeled based on their characteristics, leading to improved detection effectiveness without the need for labeled data. In structural health monitoring (SHM), data from wireless sensor networks (WSNs) deployed on infrastructure systems frequently suffer from missing values due to sensor faults or battery life constraints, compromising monitoring reliability. To address this issue, a graph-regularized decomposition method is proposed that models the spatial relationships between sensors based on their dynamic behavior, as captured by structural mode shapes. In this framework, edge weights in the graph represent inter-sensor similarity derived from mode shape correlation, allowing the imputation of missing data using only structurally relevant sensors. This targeted approach avoids interference from unrelated data and significantly enhances the missing data imputation performance and robustness of SHM systems.