Introduction

biodb provides access to chemical, biological and mass spectra databases, and offers a development framework that facilitates the writing of new connectors.

Numerous public databases are available for scientific research, but few are easily accessible from a programming environment, making it hard for most of researchers to use their content. Developing a code to access databases and keep it up to date with the evolutions of these databases are two time consuming tasks. It is thus greatly preferable to use an already developed package.

In R, packages with public database connectors, most often propose to connect to one single database with a specific API, and do not offer a development framework (Szöcs et al. 2020; Guha 2016; Tenenbaum and Volkening 2020; Carey 2020; Soudy et al. 2020; Carlson and Ortutay 2020; Wolf 2019; Stravs et al. 2013; Drost and Paszkowski 2017; Winter, Chamberlain, and Guangchun 2020). When a package does not offer the services the scientific programmer requests, or when no package exists for the targeted database, a homemade solution is implemented. In such a case, the effort spent is often lost and never capitalized for sharing with the community.

biodb has been designed and implemented as a unified API to databases and a development framework. The unified API allows to access the databases in a standardized way, while allowing original database services to be accessed directly. The development framework has for goal to help scientific programmers to capitalize on their effort to improve connection to databases and share it with the community. The framework lowers the effort needed by the developer to improve an existing connector or implement a new one. Most biodb connectors are distributed inside separated packages, that are automatically recognized by the main package. This system of extensions gives more independence for developing new connectors and distributing them, since developers do not need to request any modification inside the main package code.

The database services provided by the unified API of biodb: retrieval of entries, chemical and biological compound search by mass and name, mass spectra annotation, MSMS matching, read and write of in-house local databases. Alongside the unified API, connectors to public databases furnishes also access to specific web services through dedicated methods. See table @ref(tab:features) for a list of available features.

In this vignette we will introduce you to the basic features of biodb, allowing you to be quickly productive. Pointers toward other documents are included along the way, for going into details or learning advanced features.

For a complete list of features, see vignette Details on biodb for a more more information of biodb with other packages.

Installation

Install using Bioconductor:

if (!requireNamespace("BiocManager", quietly=TRUE))
    install.packages("BiocManager")
BiocManager::install('biodb')

Initialization

The first step in using biodb, is to create an instance of the main class BiodbMain. This is done by calling the constructor of the class:

mybiodb <- biodb::newInst()

During this step the configuration is set up, the cache system is initialized and extension packages are loaded.

We will see at the end of this vignette that the biodb instance needs to be terminated with a call to the terminate() method.

Connecting to a database

In biodb the connection to a database is handled by a connector instance that you can get from the factory. Here we create a connector to a CSV file database (see @ref(tab:compTable) for content) of chemical compounds:

compUrl <- system.file("extdata", "chebi_extract.tsv", package='biodb')
compdb <- mybiodb$getFactory()$createConn('comp.csv.file', url=compUrl)

## Loading required package: biodb

The two parameters passed to the createConn() are the identifier of the Compound CSV File connector class and the URL (i.e.: the path) of the TSV file. With this connector instance you are now able to get entries and search for them by either name or mass. By default biodb will use the TAB character as separator for the CSV file, and the standard biodb entry field names for the column names of the file. To load a CSV file with a different separator and custom column names, you have to define them inside the connector instance. Please see vignette Details on biodb for learning how to define the character separator and the column names of your file inside the CSV database connector.

To get a list of all connector classes available with their names, call an instance of BiodbDbsInfo:

mybiodb$getDbsInfo()

## Biodb databases information instance.
## The following databases are defined:
##   comp.csv.file: Compound CSV File connector class.
##   comp.sqlite: Compound SQLite connector class.
##   mass.csv.file: Mass spectra CSV File connector class.
##   mass.sqlite: Mass spectra SQLite connector class.

To get available informations on these database connectors, use the get() method:

mybiodb$getDbsInfo()$get(c('comp.csv.file', 'mass.csv.file'))

## $comp.csv.file
## Compound CSV File class.
##   Class: comp.csv.file.
##   Package: biodb.
##   Description: A connector to handle a compound database stored inside a CSV file. It is possible to choose the separator for the CSV file, as well as match the column names with the biodb entry fields..
##   Entry content type: tsv.
## 
## $mass.csv.file
## Mass spectra CSV File class.
##   Class: mass.csv.file.
##   Package: biodb.
##   Description: A connector to handle a mass spectra database stored inside a CSV file. It is possible to choose the separator for the CSV file, as well as match the column names with the biodb entry fields...
##   Entry content type: tsv.

Here we must stop a moment to explain the use of the $ operator. This operator is the call operator for the object oriented programming (OOP) model R5. This OOP model is different from S4. While in S4 the generic methods and their specialization are defined apart from the classes, in R5 the two are defined together and a method definition is necessarily part of a class. Each method being part of a class, it is also part of each object of the class, hence the use of a call operator ($) on a object. In the code line above, the call mybiodb$getFactory() means to call getFactory() method onto biodb instance. This call will return another object (of class BiodbFactory) on which we call the method createConn(). Note that while in R Studio, you will benefit from the autosuggestion system to find all methods available for an instance. See vignette Details on biodb for explanations about the OOP model chosen for biodb.

Accessing entries

The main goal of the connector is to retrieve entries. The two main generic ways to retrieve entries with a connector are: getting entries using their identifiers (accession numbers), and searching for entries by name. For compound databases, there is also the possibility to search for entries by mass.

In this section we will show how to get entries, convert them into a data frame and search for entries by name. For advanced features about entries, please see the vignette entries.

entries <- compdb$getEntry(c('1018', '1549', '64679'))

## INFO  [03:11:22.607] Loading file database "/tmp/Rtmp3r5jwV/Rinst14cbaba7141/biodb/extdata/chebi_extract.tsv".

entries

## [[1]]
## Biodb Compound CSV File entry instance 1018.
## 
## [[2]]
## Biodb Compound CSV File entry instance 1549.
## 
## [[3]]
## Biodb Compound CSV File entry instance 64679.

entries[[1]]$getFieldsAsJson()

## {
##   "accession": "1018",
##   "formula": "C2H8AsNO3",
##   "monoisotopic.mass": 168.97201,
##   "molecular.mass": 169.012,
##   "kegg.compound.id": "C07279",
##   "name": "2-Aminoethylarsonate",
##   "smiles": "NCC[As](O)(O)=O",
##   "description": "",
##   "comp.csv.file.id": "1018"
## }

fields <- entries[[1]]$getFieldNames()
fields

## [1] "accession"         "comp.csv.file.id"  "description"      
## [4] "formula"           "kegg.compound.id"  "molecular.mass"   
## [7] "monoisotopic.mass" "name"              "smiles"

mybiodb$getEntryFields()$get(fields)

## $accession
## Entry field "accession".
##   Description: The accession number of the entry.
##   Class: character.
##   Case: sensitive.
##   Cardinality: one.
##   Aliases: NA.
## 
## $comp.csv.file.id
## Entry field "comp.csv.file.id".
##   Description: Compound CSV File ID
##   Class: character.
##   Case: insensitive.
##   Type: id.
##   Cardinality: many.
##   Duplicates: forbidden.
##   Aliases: NA.
## 
## $description
## Entry field "description".
##   Description: The decription of the entry.
##   Class: character.
##   Case: sensitive.
##   Cardinality: one.
##   Aliases: protdesc.
## 
## $formula
## Entry field "formula".
##   Description: Empirical chemical formula of a compound.
##   Class: character.
##   Case: sensitive.
##   Cardinality: one.
##   Aliases: NA.
## 
## $kegg.compound.id
## Entry field "kegg.compound.id".
##   Description: KEGG Compound ID
##   Class: character.
##   Case: insensitive.
##   Type: id.
##   Cardinality: many.
##   Duplicates: forbidden.
##   Aliases: NA.
## 
## $molecular.mass
## Entry field "molecular.mass".
##   Description: Molecular mass (also called molecular weight), in u (unified atomic mass units) or Da (Dalton). It is computed from the atomic masses of each nuclide present in the molecule, taking into account the various possible isotops of each atom. See https://en.wikipedia.org/wiki/Molecular_mass.
##   Class: double.
##   Type: mass.
##   Cardinality: one.
##   Aliases: mass, molecular.weight, compoundmass.
## 
## $monoisotopic.mass
## Entry field "monoisotopic.mass".
##   Description: Monoisotopic mass, in u (unified atomic mass units) or Da (Dalton). It is computed using the mass of the primary isotope of the elements including the mass defect (mass difference between neutron and proton, and nuclear binding energy). Used with high resolution mass spectrometers. See https://en.wikipedia.org/wiki/Monoisotopic_mass.
##   Class: double.
##   Type: mass.
##   Cardinality: one.
##   Aliases: exact.mass.
## 
## $name
## Entry field "name".
##   Description: The name of the entry.
##   Class: character.
##   Case: insensitive.
##   Type: name.
##   Cardinality: many.
##   Duplicates: forbidden.
##   Aliases: fullnames, synonyms.
## 
## $smiles
## Entry field "smiles".
##   Description: SMILES.
##   Class: character.
##   Case: sensitive.
##   Cardinality: one.
##   Aliases: NA.

entries[[1]]$getFieldValue('formula')

## [1] "C2H8AsNO3"

entryDf <- entries[[1]]$getFieldsAsDataframe()

entriesDf <- mybiodb$entriesToDataframe(entries)

compdb$searchForEntries(list(name='deoxyguanosine'))

## [1] 40304

compdb$searchForEntries(list(name='guanosine', monoisotopic.mass=list(value=283.0917, delta=0.1)))

## [1] 16750 40304

mybiodb$getEntryFields()$getFieldNames(type='mass')

## [1] "average.mass"      "molecular.mass"    "monoisotopic.mass"
## [4] "nominal.mass"

mybiodb$getEntryFields()$get('nominal.mass')

## Entry field "nominal.mass".
##   Description: Nominal mass, in u (unified atomic mass units) or Da (Dalton). It is computed using the mass number of the most abundant isotope of each atom. Typically used with low resolution mass spectrometers. See https://en.wikipedia.org/wiki/Monoisotopic_mass.
##   Class: integer.
##   Type: mass.
##   Cardinality: one.
##   Aliases: NA.

compdb$isSearchableByField('average.mass')

## [1] FALSE

compdb$getSearchableFields()

## [1] "name"              "monoisotopic.mass" "molecular.mass"

Mass spectra

Another feature of biodb is the ability to annotate an LCMS spectra or to search for an MSMS spectra matching. In this section we will see the annotation of LCMS spectra and matching of MSMS spectra.

ms.tsv <- system.file("extdata", "ms.tsv", package='biodb')
mzdf <- read.table(ms.tsv, header=TRUE, sep="\t")

annotMz <- compdb$annotateMzValues(mzdf, mz.tol=1e-3, ms.mode='neg')

massUrl <- system.file("extdata", "massbank_extract_lcms_3.tsv", package='biodb')
massDb <- mybiodb$getFactory()$createConn('mass.csv.file', url=massUrl)

input <- data.frame(mz=c(73.01, 116.04, 174.2), rt=c(79, 173, 79))

annotMzRt <- massDb$searchMsPeaks(input, mz.tol=0.1, rt.unit='s', rt.tol=10, match.rt=TRUE, prefix='match.')

## INFO  [03:11:23.657] Loading file database "/tmp/Rtmp3r5jwV/Rinst14cbaba7141/biodb/extdata/massbank_extract_lcms_3.tsv".

msmsUrl <- system.file("extdata", "massbank_extract_msms.tsv", package='biodb')
msmsdb <- mybiodb$getFactory()$createConn('mass.csv.file', url=msmsUrl)

input <- data.frame(mz=c(286.1456, 287.1488, 288.1514), rel.int=c(100, 45, 18))

matchDf <- msmsdb$msmsSearch(input, precursor.mz=286.1438, mz.tol=0.1, mz.tol.unit='plain', ms.mode='pos')

## INFO  [03:11:24.128] Loading file database "/tmp/Rtmp3r5jwV/Rinst14cbaba7141/biodb/extdata/massbank_extract_msms.tsv".

Creating and improving connectors

A powerful feature of biodb is its architecture as a development framework. Connectors can be extended dynamically by created new rules to parse field values, or creating new fields. New connectors can also be defined. This feature has been used to create connectors to public databases like: KEGG, ChEBI, HMDB or UniProt.

See the vignettes Creating a new field for entries. and Creating a new connector. for details about connector creation and defining new entry fields.

Existing biodb extension packages

Several extension packages for biodb exist today on GitHub. See table @ref(tab:extensions) for a list of those extension and a short description.

For installing them, please first make sure that you have the package devtools installed and run:

devtools::install_github('pkrog/biodbChebi', dependencies=TRUE, build_vignettes=TRUE)

Replace 'pkrog/biodbChebi' by the appropriate repository.

The extensions whose status is marked as Functional are in working order and can be installed and used safely with biodb. They may still need some updates in the documentation or the tests, thus do not hesitate to contact us if you have doubts on the API, the behaviour or if you would like to improve the extension. The extensions whose status is marked as In maintenance are currently non functional due to the refactoring of biodb into a development framework, but will be upgraded as soon as possible. If have the need to re-enable a currently in maintenance extension, do not hesitate to contact us, we may be able to accelerate the upgrade or propose you with our support to upgrade it yourself. If you have the desire to develop a new extension, please contact us, as we will be able accompany you in the process.

Sources of documentation

Several vignettes are available. Among them you will find help for creating a new connector, adding an entry field to an existing connector, searching for compounds by mass or name, merging entries from different databases into a local database, annotation of a mass spectrum, etc. See table @ref(tab:vignettes) for a full list of available vignettes.

You will also find documentation inside the R manual of the package. All biodb public classes have a help page. On each help page you will find a description of the class as well as a list of all its public methods with a description of their parameters. For instance you can get help on BiodbEntry class with ?BiodbEntry.

Closing biodb instance

When done with your biodb instance you have to terminate it, in order to ensure release of resources (file handles, database connection, etc):

mybiodb$terminate()

## INFO  [03:11:24.309] Closing BiodbMain instance...
## INFO  [03:11:24.310] Connector "comp.csv.file" deleted.
## INFO  [03:11:24.311] Connector "mass.csv.file" deleted.
## INFO  [03:11:24.311] Connector "mass.csv.file.1" deleted.

Session information

sessionInfo()

## R version 4.4.2 (2024-10-31)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 24.04.1 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=C              
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: Etc/UTC
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] biodb_1.15.0     BiocStyle_2.35.0
## 
## loaded via a namespace (and not attached):
##  [1] rappdirs_0.3.3       sass_0.4.9           utf8_1.2.4          
##  [4] generics_0.1.3       stringi_1.8.4        RSQLite_2.3.8       
##  [7] hms_1.1.3            digest_0.6.37        magrittr_2.0.3      
## [10] evaluate_1.0.1       fastmap_1.2.0        blob_1.2.4          
## [13] plyr_1.8.9           jsonlite_1.8.9       progress_1.2.3      
## [16] DBI_1.2.3            BiocManager_1.30.25  httr_1.4.7          
## [19] fansi_1.0.6          XML_3.99-0.17        jquerylib_0.1.4     
## [22] cli_3.6.3            rlang_1.1.4          chk_0.9.2           
## [25] crayon_1.5.3         dbplyr_2.5.0         bit64_4.5.2         
## [28] withr_3.0.2          cachem_1.1.0         yaml_2.3.10         
## [31] tools_4.4.2          memoise_2.0.1        dplyr_1.1.4         
## [34] filelock_1.0.3       curl_6.0.1           buildtools_1.0.0    
## [37] vctrs_0.6.5          R6_2.5.1             BiocFileCache_2.15.0
## [40] lifecycle_1.0.4      stringr_1.5.1        bit_4.5.0           
## [43] pkgconfig_2.0.3      pillar_1.9.0         bslib_0.8.0         
## [46] glue_1.8.0           Rcpp_1.0.13-1        lgr_0.4.4           
## [49] xfun_0.49            tibble_3.2.1         tidyselect_1.2.1    
## [52] sys_3.4.3            knitr_1.49           htmltools_0.5.8.1   
## [55] rmarkdown_2.29       maketools_1.3.1      compiler_4.4.2      
## [58] prettyunits_1.2.0    askpass_1.2.1        openssl_2.2.2