Automated Generation of Telemetry Formats

Abstract

The process of generating a telemetry format is currently more of an ad-hoc art than a science. Telemetry stream formats conform to traditions that seem to be obsolete given today's computing power. Most format designers would have difficulty explaining why they use the development heuristics they use and even more difficulty explaining why the heuristics work. The formats produced by these heuristics tend to be inefficient in the sense that bandwidth is wasted. This paper makes an important step in establishing a theory on which to base telemetry format construction. In particular it describes an O(nlog n) algorithm for automatically generating telemetry formats. The algorithm also has the potential of efficiently filling a telemetry stream without wasting bits.