devtools::install_github("SydneyBioX/BenchHub")
BenchHub is a data storage framework implemented in R language to facilitate living benchmarks. It aims to enhance reproducibility and accessibility of benchmarking studies by making it easier to store, analyse and share benchmarking data.
The three components currently in BenchHub are:
BenchHub aims to make benchmarking easier for multiple groups of users in the community:
Trio is built around three key components:
Trio is implemented as R6 object with fields to store each of the components.
BenchmarkInsight serves as a visualisation and analysis tool for benchmarking results. It contains multiple visualisation techniques to help researchers analyse benchmarking results in terms of data, methods and metrics.
Once results are evaluated using Trio, the output can be directly passed into the benchmarkInsight object. BenchmarkInsight currently supports the following plot types.
Benchmarking often involve large amount of datasets and processing scripts. BenchmarkStudy serves as the organisation framework for such objects. It stores references to collections of Trio objects and helper functions that standardise method outputs into a common form to evaluation.
We provide a comprehensive list of vignettes for every key step of BenchHub.
Please refer to the website to see all vignettes listed below:
| Vignette | Description |
|---|---|
| 1 Introduction to the Trio Class | |
| 2 Evaluation using Trio | |
| 3 Introduction to BenchmarkInsight Class | |
| 4 Uploading a Trio to Curated Trio Datasets | |
| 5 Working with BenchmarkStudy |