Show simple item record

dc.contributor.authorOgden, Eddge
dc.contributor.authorCall, Kenneth
dc.contributor.authorMyers, Isaac J.
dc.contributor.authorMa, Zayd L.
dc.contributor.authorLowe, Daniel
dc.date.accessioned2022-11-24T01:32:51Z
dc.date.available2022-11-24T01:32:51Z
dc.date.issued2022-10
dc.identifier.citationOgden, E., Call, K., Myers, I. J., Ma, Z. L., & Lowe, D. (2022). OPAL: Leveraging Open Source. International Telemetering Conference Proceedings, 57.
dc.identifier.issn1546-2188
dc.identifier.issn0884-5123
dc.identifier.issn0074-9079
dc.identifier.urihttp://hdl.handle.net/10150/666979
dc.description.abstractThe current explosion of test and evaluation data being collected from various systems has exposed a strong need for low-cost digital infrastructure to facilitate scalable analytics across all available data. Private industry and academic research have built such systems utilizing Open-Source Software (OSS) with tremendous success. The 309th Software Egineering Group (SWEG) developed OPAL (Open Platform for Advanced Learning) platform is a government owned and developed solution to address this gap and provide data discovery, analytics, and warehousing all license free. OPAL leverages best-in-breed Open-Source Software including JupyterLab (Python analysis environment), MinIO (S3-compliant, redundant and object-versioning data backend), Postgres (Data cataloging), and Dask (scalable compute), among others. In addition to Open-Source tooling, custom integration and software piping are used to further lower the analysts’ barrier to available data: custom Chapter 10 parsing and translating at high speed (10GB/min) into Apache Parquet format, a web-based data catalog for discovery, and lightweight arbitrary object storage organization. This paper will delineate design choices, our DevOps paradigm, benchmarking numbers, and results against a publicly available commercial flight dataset.
dc.description.sponsorshipInternational Foundation for Telemetering
dc.language.isoen
dc.publisherInternational Foundation for Telemetering
dc.relation.urlhttp://www.telemetry.org/
dc.rightsCopyright © held by the author; distribution rights International Foundation for Telemetering
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/
dc.titleOPAL: Leveraging Open Source
dc.typeProceedings
dc.typetext
dc.contributor.department309th SWEG
dc.contributor.departmentAir Force Materiel Command Hill AFB, UT
dc.contributor.departmentAvionics Test and Analysis Corporation
dc.identifier.journalInternational Telemetering Conference Proceedings
dc.description.collectioninformationProceedings from the International Telemetering Conference are made available by the International Foundation for Telemetering and the University of Arizona Libraries. Visit https://telemetry.org/contact-us/ if you have questions about items in this collection.
dc.eprint.versionFinal published version
dc.source.journaltitleInternational Telemetering Conference Proceedings
refterms.dateFOA2022-11-24T01:32:52Z


Files in this item

Thumbnail
Name:
ITC_2022_22-15-03.pdf
Size:
477.1Kb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record