Intelligent Federated Cyber-physical Systems Experimental Testbeds: Design, Analysis, and Evaluation

Abstract

The advancement of computer information technology, Internet services, and Internet of Things (IoT) have made their deployments touch all aspects of our life. In fact, almost all modern engineering systems integrate software components with physical systems (Cyber-Physical Systems) such as washing machines, cooking appliances, autonomous vehicles, drones, power grids, medical devices, and industrial control systems. Cyber-Physical System (CPS) research is active in nearly every industrial field including agriculture, energy management, automotive, smart city, medical devices, and military equipment. CPS produces new functionality in traditional physical systems while at the same time they are adding vulnerabilities and complexities when they are integrated with cyber components. On one hand, CPS makes a huge contribution to national economic growth. For example, the number of self-driving cars, a promising CPS application in transportation, has reached 20.3 million units in 2021. On the other hand, the interconnectivity of cyberspace and physical components face the threats of cyberattacks. Cybercrime has caused more than 6 trillion US dollars loss in 2016 and it is expected to reach 10.5 trillion US dollars annually by 2025. CPS Design requires a comprehensive testing and evaluation methodology to ensure the security, performance, and reliability of the system and it requires financial investment in hardware and software support. A testbed with real-world applications can accelerate the research to increase the security, reliability, and performance of CPS applications and consequently lead to rapid acceptance and deployment of CPS services. Currently, there are many isolated CPS testbeds; however, little research has focused on methods to automatically build a federated CPS testbed. There is a lack of fundamental research on designing methodologies that address the challenges of seamlessly and efficiently composing isolated CPS testbeds and scheduling experiments on heterogeneous CPS testbeds that can be managed by different organizations. We critically need the capabilities to collect and analyze data of a federated CPS testbed to experiment with and evaluate different algorithms for securing and protecting cyber-physical systems and their services under normal and abnormal conditions. Abnormal events can be caused by natural causes, accidents, or malicious actions. Hence, there is an urgent need for designing a federated CPS testbed platform that helps users to experiment with, validate and evaluate different CPS designs that meet their security and performance requirements. The objective of this research is to investigate the design of a cloud platform that can seamlessly compose federated CPS testbeds to facilitate CPS experimentations and data analysis.In this dissertation, we developed an efficient federated CPS testbed platform that we refer to as Federated Cybersecurity Testbed as a Service (FCTaaS). The FCTaaS provides the following capabilities: 1) An efficient method to build a federated testbed using experiment manager service, policy and security services, interoperability service, and web services; 2) Data preparation and machine learning services to support data standardization and provide machine learning algorithms to uncover the hidden pattern of CPS system from CPS traffic; 3) An explainable machine learning service and model-agonistic interpretation approaches that can significantly improve the trustworthiness of ML algorithms. We validated the effectiveness of the FCTaaS by conducting cybersecurity experimentations such as Denial-of-Service attacks and Man-in-the-middle attacks. We evaluated the federated testbed performance by measuring the data transmission latency between geographically distributed testbeds and the integration overhead with the CPU and memory usage in the federated testbed. For validating the federated testbed data preparation and machine learning services, we developed an intelligent video surveillance system and our experimental results show that our model achieves comparable results when compared to the state-of-the-art approaches in spite of using a lower complexity model.