Multi-Layer Data-Plane Telemetry Collection Design for Systems With Multiple Observers

Abstract

Existing telemetry-related technical discussions focus on how to efficiently collect data from multiple systems for a single observer. In this paper, the focus is on a generic telemetry design for a single system observed by multiple distinct observers. For example, in a server, a management application collects capacity usage-related data while the vendor's telemetry application collects hardware-related parameters, license-related data, etc. Multiple observers can have both common and unique data set used for observability. As the frequency of collection is varied across observers, telemetry collection design needs to have approaches to a) avoid “observer effect”, b) use hybrid data collection methods like streaming and pull. We present and discuss the pros and cons of different architectures for the use case of multiple observers for telemetry collection. Our proposed design uses REST API -based telemetry architecture with multi-layer data planes.