High-Dimensional Data Analytics Based on Spatial-Temporal Decomposition

Abstract

In the last decade, significant progress in sensing and data storage technologies has ushered in a new era of spatiotemporal data analysis. This progress has dramatically increased the availability and scale of spatiotemporal data, presenting both exciting opportunities and complex challenges in this field. Spatiotemporal data from various sources exhibit unique patterns, but they share two common characteristics: (i) anomalies in spatiotemporal data are typically sparse, meaning that the number of anomalies is significantly less than the number of normal data, and (ii) anomalies cause deviations from the normal patterns of the data. To illustrate the practical application of these principles, consider a water distribution system (WDS). A sudden burst in the system can lead to a substantial drop in water pressure compared to normal conditions. To efficiently detect such anomalies, a penalized regression model based on basis expansion is introduced. This model effectively captures the features of hydraulic measurements through basis coefficients. It encourages sparsity in the anomalies (such as bursts) by applying an $L_1$ penalty. Furthermore, it encourages normal measurements to align closely with the sample mean through an $L_2$ regularization term. The model is solved using an optimization algorithm, and its performance is evaluated through a simulated case study. In the context of additive manufacturing, where images capture the manufacturing process, the profile of objects being produced must be extracted accurately. This is particularly challenging due to variations in pixel intensities between the cured profile and the background, which are caused by differences in optical properties. To address this, a tensor decomposition-based method is introduced. Difference matrices are employed to penalize variations in pixel intensities both vertically and horizontally, promoting smoothness in the background. Simultaneously, an $L_1$ regularization term enforces sparsity in the cured profile. The optimization model is solved to estimate the profile, with the effectiveness of this approach demonstrated through both simulated and real-world case studies. In the context of a surveillance system, the primary goal is to detect moving targets, especially when dealing with a moving camera. To achieve this, a novel optical flow-based method is proposed. Beyond considerations for background smoothness and foreground sparsity, this method introduces a total-variance regularization mechanism based on patches. This ensures that the optical flow associated with the foreground moves consistently. Real-world case studies are used to validate the proposed model's performance in detecting moving objects.