EFFICIENT DATA STORAGE, SAMPLING, AND RETRIEVAL BY LEVERAGING OPEN SOURCE TECHNOLOGIES
AffiliationAvionics Test & Analysis Corporation
MetadataShow full item record
RightsCopyright © held by the author; distribution rights International Foundation for Telemetering
Collection InformationProceedings from the International Telemetering Conference are made available by the International Foundation for Telemetering and the University of Arizona Libraries. Visit http://www.telemetry.org/index.php/contact-us if you have questions about items in this collection.
AbstractThis paper demonstrates the use of open-source software tools to manage large data sets. Advances in technology have greatly reduced the cost of data storage and processing systems. The ability to handle large amounts of data efficiently while retaining fine-grain control of the data retrieval process becomes a challenge. In particular, traditional data processing applications are inadequate to handle the large data sets typically encountered in IRIG-106 Chapter 10 data files. We answer this challenge by using readily available, open-source tools that efficiently store and retrieve IRIG-106 Chapter 10 data to/from a file-based database. We will demonstrate a method that facilitates a separation between the parsing of raw input data and the display of desired information at a user-defined sample rate. This open-source based solution provides a low-cost, reliable, and efficient means for handling large amounts of data at a high rate of speed.
SponsorsInternational Foundation for Telemetering