Revealing Earth science code and data-use practices using the Throughput Graph Database

Abstract

The increased use of complex programmatic workflows and open data within the Earth sciences has led to an increase in the need to find and reuse code, whether as examples, templates, or code snippets that can be used across projects. The “Throughput Graph Database” project offers a platform for discovery that links research objects by using structured annotations. Throughput was initially populated by scraping GitHub for code repositories that reference the names or URLs of data archives listed on the Registry of Research Data Repositories (https://re3data.org). Throughput annotations link the research data archives to public code repositories, which makes data-relevant code repositories easier to find.

Linking code repositories in a queryable, machine-readable way is only the first step to improving discoverability. A better understanding of the ways in which data is used and reused in code repositories is needed to better support code reuse. In this paper, we examine the data practices of Earth science data reusers through a classification of GitHub repositories that reference geology and paleontology data archives. A typology of seven reuse classes was developed to describe how data were used within a code repository, and it was applied to a subset of 129 public code repositories on GitHub. Code repositories could have multiple typology assignments. Data use for Software Development dominated (n = 44), followed by Miscellaneous Links to Data Archives (n = 41), Analysis (n = 22), and Educational (n = 20) uses. GitHub repository features show some relationships to the assigned typologies, which indicates that these characteristics may be leveraged to systematically predict a code repository’s category or discover potentially useful code repositories for certain data archives.