Advancing Standards for Machine Learning in the Biosciences

Abstract

The data revolution in the life sciences has brought on new challenges and opportunities. Large-scale and complex data from novel sensing mechanisms challenges existing infrastructure and methods of modeling and drawing inferences. Increasingly, researchers turn to machine learning (ML) methods to detect patterns and draw inferences from these datasets. While these methods have brought new insights and capabilities to process these data, new pitfalls have emerged. In many cases, the reproducibility of these studies is challenging as standards continue to evolve. Further, ML methods are being developed at a rapid rate, making it difficult for researchers to assess the utility and limitations of new methods. New standards are needed to improve the reproducibility, reusability, and comparison of ML and other computational methods. The aim of this dissertation is to advance those standards and drive the comparison of tools and analytics. Specifically, three works are presented, each of which aims to further reproducibility standards regarding the use of ML in life science research. The first chapter is a review paper that explores the use of ML for biosensing using bioreceptor-free biosensors. In traditional biosensors, the bioreceptor provides specificity and sensitivity to the sensor. The bioreceptor, being of biological origin (e.g., an enzyme) degrades over time and under certain ambient conditions. To circumvent this, sensors have been developed which attempt to imitate the bioreceptor and biological sensing organs. In many cases, the loss of performance due to the lack of bioreceptor is compensated with robust data analysis techniques, particularly ML for pattern detection. Given the rapid pace of ML development, the methods used for each sensing modality varies, as does the performance of those methods. Additionally, the methods used for different sensing modalities are disjoint, even when similar data structures and problems are faced in various subfields. In this review, the methods that have been used for the different sensing modalities, and the ones that have been the most performant are highlighted. By addressing the field as a whole, this review provides guidance to researchers regarding comparisons between candidate models and points them toward future directions.The second chapter goes beyond a comparison of reported performance and conducts a thorough benchmarking of computational tools developed for detecting bacteriophage (phage) sequences in metagenomes. Sequence classification, particularly for detecting phage, is a difficult task. Phages rapidly mutate, lack a conserved biomarker, and can integrate into the host genome as a prophage. By July of 2021, at least 19 computational tools have been published for detecting phage sequences in metagenomes, each taking a different approach to the problem. Many of the tools build ML models using features related to homology (e.g., viral hallmark genes, lack of bacterial genes) or features of the sequences themselves (e.g., k-mers). When attempting to run these tools, less than half (9 / 19) could be run at scale, if at all. Of the tools compared, performance on several metrics varied significantly. The strengths and weaknesses of the tools are evaluated to inform future research. All datasets used for benchmarking the tools are made available to facilitate further benchmarking efforts. Finally, the last chapter is an opinion paper presenting the methods used and rationale for implementing a ML-enabled workflow that reaches a high level of reproducibility. In this work, considerations for reproducibility, code, and data standards are discussed. The paper describes how standards were implemented, including which tools and software packages were used. This work serves as an example for implementing a reproducible ML research project and opens further discussion on how the implementation could be further improved.