| Title: | Serialize and restore Spectra objects in interoperable file formats |
|---|---|
| Description: | The serialization mechanism of R allows to save and load R data objects in a binary format, that can however not be read by other programming languages or software. The SpectraStash package implements the serialization methods from the MsStash package for Spectra objects and its MsBackend classes. A variety of different file formats and types, including HDF5 and JSON-based formats defined by the Bioconductor *alabaster* package are supported. The file type can be defined and configured through a second argument `param` of the save/read methods. |
| Authors: | Johannes Rainer [aut, cre] (ORCID: <https://orcid.org/0000-0002-6977-7147>), Philippine Louail [aut] (ORCID: <https://orcid.org/0009-0007-5429-6846>), Laurent Gatto [ctb] (ORCID: <https://orcid.org/0000-0002-1520-2268>), Gabriele Tomè [aut] (ORCID: <https://orcid.org/0000-0002-3976-6068>, fnd: MetaRbolomics4Galaxy project (CUP: D53C25001030003) co-funded by the Autonomous Province of Bolzano under the Joint Projects South Tyrol–Germany 2025 program.) |
| Maintainer: | Johannes Rainer <[email protected]> |
| License: | GPL-3 |
| Version: | 0.99.7 |
| Built: | 2026-11-17 12:54:33 UTC |
| Source: | https://github.com/bioc/SpectraStash |
MsBackendCached
The Spectra::MsBackendCached backend keeps a cache of spectra metadata
(spectra variables) in an internal data.frame adding thus support for
(temporarily) changing or adding spectra variables for purely read-only
MsBackend classes. This backend is used e.g. by the
MsBackendSql::MsBackendSql backend allowing to add or replace spectra
variables without affecting the content of the underlying data base used by
the MsBackendSql.
The stash of a MsBackendCached contains therefore the local spectra data
cache (if present), the names of the available spectra variables and the
total number of spectra.
Any classes extending MsBackendCached must implement their own
saveMsObject() and readMsObject() methods and call callNextMethod() to
trigger execution of the respective method from MsBackendCached storing
respectively reading any cached data to (or from) the stash.
Supported stash formats are listed in the sections below.
## S4 method for signature 'MsBackendCached,PlainTextParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendCached,PlainTextParam' readMsObject(object, param, ...) ## S4 method for signature 'MsBackendCached' saveObject(x, path, ...) ## S4 method for signature 'MsBackendCached,AlabasterParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendCached,AlabasterParam' readMsObject(object, param, ...)## S4 method for signature 'MsBackendCached,PlainTextParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendCached,PlainTextParam' readMsObject(object, param, ...) ## S4 method for signature 'MsBackendCached' saveObject(x, path, ...) ## S4 method for signature 'MsBackendCached,AlabasterParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendCached,AlabasterParam' readMsObject(object, param, ...)
object |
An |
param |
Either a |
... |
Currently ignored. |
x |
An |
path |
For |
Notes for stash-functionality for MsBackend objects extending
MsBackendCached:
saveMsObject() and saveObject() will fail if the stash directory
already exist. Thus, stash functions of backend implementations extending
MsBackendCached should first call the MsBackendCached's
saveMsObject() or saveObject() (e.g. through callNextMethod())
before exporting their respective content to the stash directory.
readMsObject() returns a Spectra::MsBackendCached object.
AlabasterParam
The content from all slots of the MsBackendCached are stored using
functionality from the alabaster.base package into separate sub-folders
of the base stash directory. These are:
local_data: for the data.frame with the locally cached spectra
variables (slot @localData).
nspectra: (integer(1)) with the number of spectra.
spectra_variables: (character) with the names of the object's spectra
variables.
PlainTextParam
The data files written into the stash are:
ms_backend_data.txt: tabulator separated text file with the content of
the @localData slot.
ms_backend_nspectra.txt: the number of spectra.
ms_backend_spectra_variables.txt: the names of the object's spectra variables (tabulator separated).
Johannes Rainer
library(Spectra) ## Create an empty `MsBackendCached` object be <- MsBackendCached() ## Stash the object in alabaster-format in a temporary directory ap <- AlabasterParam(file.path(tempdir(), "cache-stash")) saveMsObject(be, ap) ## The content of the stash folder: dir(file.path(tempdir(), "cache-stash")) ## Restore the object res <- readMsObject(MsBackendCached(), ap) reslibrary(Spectra) ## Create an empty `MsBackendCached` object be <- MsBackendCached() ## Stash the object in alabaster-format in a temporary directory ap <- AlabasterParam(file.path(tempdir(), "cache-stash")) saveMsObject(be, ap) ## The content of the stash folder: dir(file.path(tempdir(), "cache-stash")) ## Restore the object res <- readMsObject(MsBackendCached(), ap) res
MsBackendHdf5Peaks classes can be stashed to (or read from) plain text
file-based or alabaster-based formats using the saveMsObject() and
readMsObject() functions combined with the PlainTextParam and
AlabasterParam parameter objects, respectively. In both cases, data files
are stored into the specified directory. By default, only the backend's
spectra metadata is stored in that folder. Setting parameter
consolidate = TRUE will also copy the HDF5-format peaks data files
(containing the m/z and intensity values) of the backend into the folder
generating a self-consistent stash. The paths to the data storage files are
also updated to relative paths enabling to directly restore the object from
the stash when the stash folder was copied to another computer or location
on the file system (i.e., without the use of parameter spectraPath).
Details on the stored files are provided in the sections below.
## S4 method for signature 'MsBackendHdf5Peaks,PlainTextParam' saveMsObject(object, param, consolidate = FALSE) ## S4 method for signature 'MsBackendHdf5Peaks,PlainTextParam' readMsObject(object, param, spectraPath = character()) ## S4 method for signature 'MsBackendHdf5Peaks' saveObject(x, path, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendHdf5Peaks,AlabasterParam' saveMsObject(object, param, consolidate = FALSE) ## S4 method for signature 'MsBackendHdf5Peaks,AlabasterParam' readMsObject(object, param, spectraPath = character())## S4 method for signature 'MsBackendHdf5Peaks,PlainTextParam' saveMsObject(object, param, consolidate = FALSE) ## S4 method for signature 'MsBackendHdf5Peaks,PlainTextParam' readMsObject(object, param, spectraPath = character()) ## S4 method for signature 'MsBackendHdf5Peaks' saveObject(x, path, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendHdf5Peaks,AlabasterParam' saveMsObject(object, param, consolidate = FALSE) ## S4 method for signature 'MsBackendHdf5Peaks,AlabasterParam' readMsObject(object, param, spectraPath = character())
object |
An |
param |
Either a |
consolidate |
|
spectraPath |
For |
x |
An |
path |
For |
... |
Currently ignored. |
A MsBackendHdf5Peaks stash will by default only contain the spectra
metadata (spectra variables) but no m/z and intensity values. The reference
to the original HDF5-format peaks data files are stored as spectra variable
dataStorage. If these files were moved or if the stash was copied to a
different computer, the path to these original data files needs to be
provided to the readMsObject() function with parameter spectraPath.
Alternatively, with consolidate = TRUE, it is also possible to copy the
peaks data files to the stash directory generating a self-contained data
stash. Note however that in this case two copies of all data files exist (in
the original location and the stash directory).
readMsObject() returns an Spectra::MsBackendHdf5Peaks' object.
AlabasterParam
With AlabasterParam, the spectra metadata will be exported (or imported)
through the alabaster framework. Similar to the PlainTextParam,
consolidate = TRUE will also copy the HDF5-format peaks data files to the
stash directory.
PlainTextParam
The saveMsObject() function with the PlainTextParam stores the spectra
metadata (spectra variables) of an MsBackendHdf5Peaks to a plain tabulator
delimited text file with the name ms_backend_spectra_data.txt in the
directory specified with parameter path of the PlainTextParam object.
Depending on parameter consolidate also the peaks data files (in HDF5
format) will be copied to the stash folder (with consolidate = TRUE).
Johannes Rainer
library(Spectra) library(SpectraStash) ## Create an example MsBackendHdf5Peaks backend from a single mzML file library(MsDataHub) tmp <- backendInitialize(MsBackendMzR(), X20171016_POOL_POS_1_105.134.mzML()) be_h5 <- backendInitialize(MsBackendHdf5Peaks(), data = spectraData(tmp), hdf5path = file.path(tempdir(), "h5_backend")) be_h5 d <- file.path(tempdir(), "example_hdf5") ptp <- PlainTextParam(path = d) ## Store the object into the stash, including the peaks data files. saveMsObject(be_h5, ptp, consolidate = TRUE) ## List the content of the folder: ms_backend_spectra_data.txt file ## with the spectra metadata and an HDF5 file with the peaks data: dir(d) ## Restore the stashed object res <- readMsObject(MsBackendHdf5Peaks(), ptp) ## Store the object in an alabaster-format stash d <- file.path(tempdir(), "example_hdf5_2") ap <- AlabasterParam(d) saveMsObject(be_h5, ap) ## Check the content of the stash; with the default (`consolidate = FALSE`) ## no HDF5 data file was moved. dir(d) ## Restore the object again res <- readMsObject(MsBackendHdf5Peaks(), ap) reslibrary(Spectra) library(SpectraStash) ## Create an example MsBackendHdf5Peaks backend from a single mzML file library(MsDataHub) tmp <- backendInitialize(MsBackendMzR(), X20171016_POOL_POS_1_105.134.mzML()) be_h5 <- backendInitialize(MsBackendHdf5Peaks(), data = spectraData(tmp), hdf5path = file.path(tempdir(), "h5_backend")) be_h5 d <- file.path(tempdir(), "example_hdf5") ptp <- PlainTextParam(path = d) ## Store the object into the stash, including the peaks data files. saveMsObject(be_h5, ptp, consolidate = TRUE) ## List the content of the folder: ms_backend_spectra_data.txt file ## with the spectra metadata and an HDF5 file with the peaks data: dir(d) ## Restore the stashed object res <- readMsObject(MsBackendHdf5Peaks(), ptp) ## Store the object in an alabaster-format stash d <- file.path(tempdir(), "example_hdf5_2") ap <- AlabasterParam(d) saveMsObject(be_h5, ap) ## Check the content of the stash; with the default (`consolidate = FALSE`) ## no HDF5 data file was moved. dir(d) ## Restore the object again res <- readMsObject(MsBackendHdf5Peaks(), ap) res
MsBackendMemory and MsBackendDataFrame
The stash for the in-memory MS backends Spectra::MsBackendMemory and
Spectra::MsBackendDataFrame contains both the spectra variables
(spectra metadata) as well as the peaks data (m/z and intensity values).
Backend objects can be stashed and restored with the saveMsObject() and
readMsObject(), respectively. In addition, the saveObject() and
readObject() functions from alabaster.base are supported.
## S4 method for signature 'MsBackendInMemory,PlainTextParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendInMemory,PlainTextParam' readMsObject(object, param, ...) ## S4 method for signature 'MsBackendInMemory' saveObject(x, path, ...) ## S4 method for signature 'MsBackendInMemory,AlabasterParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendInMemory,AlabasterParam' readMsObject(object, param, ...)## S4 method for signature 'MsBackendInMemory,PlainTextParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendInMemory,PlainTextParam' readMsObject(object, param, ...) ## S4 method for signature 'MsBackendInMemory' saveObject(x, path, ...) ## S4 method for signature 'MsBackendInMemory,AlabasterParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendInMemory,AlabasterParam' readMsObject(object, param, ...)
object |
An |
param |
Either a |
... |
Currently ignored. |
x |
An |
path |
For |
To support storing spectra and peaks variables, the MsBackendMemory is
converted to a Spectra::MsBackendHdf5Peaks object with the full peaks data
stored into a peaks.h5 HDF5 file within the stash directory. Due to the
nested directory structure it is possible to load a MsBackendMemory stash
either as a MsBackendMemory or a MsBackendHdf5Peaks.
readMsObject() returns a restored backend.
AlabasterParam
The full data is stored as an alabaster-format MsBackendHdf5PeaksStash
in a directory backend within the stash folder. Due to this nesting it is
possible to load the data either as MsBackendMemory or
MsBackendHdf5Peaks.
PlainTextParam
The MsBackendMemory stash contains a text-file format
MsBackendHdf5PeaksStash within a sub-folder backend. When loading,
the data is first imported as a MsBackendHdf5Peaks which is then converted
to the resulting MsBackendMemory (with all data materialized into memory).
Currently only peaks variables "mz" and "intensity" are supported.
Johannes Rainer
library(Spectra) ## Create an example `MsBackendMemory` object. d <- data.frame(msLevel = c(1L, 2L, 1L, 1L), rtime = c(12.1, 12.2, 13.1, 13.4)) d$mz <- list( c(14.4, 155.2, 186.4), c(144.3, 231.3, 345.3, 453.1), c(111.2, 142.4, 143.1), c(143.3, 144.3, 153.3, 532.3, 641.5) ) d$intensity <- list( c(323.2, 53.2, 35.5), c(43.3, 54.1, 33.1, 53.1), c(435.3, 35312.3, 5432.5), c(433.5, 55434.2, 43.4, 54362.1, 24435.3) ) be <- backendInitialize(MsBackendMemory(), data = d) be ## Stash the object in alabaster-format to a temporary folder p <- AlabasterParam(file.path(tempdir(), "mem_stash")) saveMsObject(be, p) ## Show directory content of the stash dir(file.path(tempdir(), "mem_stash")) ## Restore the object from the stash res <- readMsObject(MsBackendMemory(), p) res ## The in-memory backends store their full MS data through an ## [Spectra::MsBackendHdf5Peaks] backend: their peaks data is stored as a ## (single) HDF5 file within the stash. A `MsBackendMemory` stash contains ## therefore a complete stash of a `MsBackendHdf5Peaks`, which allows to ## read the data also as a `MsBackendHdf5Peaks` backend: res <- readMsObject(MsBackendHdf5Peaks(), AlabasterParam(file.path(tempdir(), "mem_stash", "backend"))) res ## In addition to the `saveMsObject()` and `readMsObject()` functions, also ## the `saveObject()` and `readObject()` functions from *alabater.base* are ## supported (for stashes in alabaster format). library(alabaster.base) res <- readObject(file.path(tempdir(), "mem_stash")) res ## Storing and restoring a `MsBackendDataFrame` backend or using a stash in ## plain text file-based format works analogously.library(Spectra) ## Create an example `MsBackendMemory` object. d <- data.frame(msLevel = c(1L, 2L, 1L, 1L), rtime = c(12.1, 12.2, 13.1, 13.4)) d$mz <- list( c(14.4, 155.2, 186.4), c(144.3, 231.3, 345.3, 453.1), c(111.2, 142.4, 143.1), c(143.3, 144.3, 153.3, 532.3, 641.5) ) d$intensity <- list( c(323.2, 53.2, 35.5), c(43.3, 54.1, 33.1, 53.1), c(435.3, 35312.3, 5432.5), c(433.5, 55434.2, 43.4, 54362.1, 24435.3) ) be <- backendInitialize(MsBackendMemory(), data = d) be ## Stash the object in alabaster-format to a temporary folder p <- AlabasterParam(file.path(tempdir(), "mem_stash")) saveMsObject(be, p) ## Show directory content of the stash dir(file.path(tempdir(), "mem_stash")) ## Restore the object from the stash res <- readMsObject(MsBackendMemory(), p) res ## The in-memory backends store their full MS data through an ## [Spectra::MsBackendHdf5Peaks] backend: their peaks data is stored as a ## (single) HDF5 file within the stash. A `MsBackendMemory` stash contains ## therefore a complete stash of a `MsBackendHdf5Peaks`, which allows to ## read the data also as a `MsBackendHdf5Peaks` backend: res <- readMsObject(MsBackendHdf5Peaks(), AlabasterParam(file.path(tempdir(), "mem_stash", "backend"))) res ## In addition to the `saveMsObject()` and `readMsObject()` functions, also ## the `saveObject()` and `readObject()` functions from *alabater.base* are ## supported (for stashes in alabaster format). library(alabaster.base) res <- readObject(file.path(tempdir(), "mem_stash")) res ## Storing and restoring a `MsBackendDataFrame` backend or using a stash in ## plain text file-based format works analogously.
Backend classes for public repository mass spectrometry data, such as
MsBackendMetaboLights from the
MsBackendMetaboLights
package can be stored and restored from stash formats extending the
MsBackendMzRStash. Backends can be saved with saveMsObject() and restored
with readMsObject(). At present only the alabaster-based stash format is
supported.
## S4 method for signature 'MsBackendMetaboLights' saveObject(x, path, ...) ## S4 method for signature 'MsBackendMetaboLights,AlabasterParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendMetaboLights,AlabasterParam' readMsObject(object, param, offline = FALSE, ...)## S4 method for signature 'MsBackendMetaboLights' saveObject(x, path, ...) ## S4 method for signature 'MsBackendMetaboLights,AlabasterParam' saveMsObject(object, param, ...) ## S4 method for signature 'MsBackendMetaboLights,AlabasterParam' readMsObject(object, param, offline = FALSE, ...)
x |
For |
path |
For |
... |
Ignored. |
object |
An |
param |
An |
offline |
For |
readObject() and readMsObject() return a MsBackendMetaboLights
backend.
AlabasterParam
Calling saveMsObject() on a MsBackendMetaboLights with an
AlabasterParam parameter object stores the backend data in a directory
specified with parameter path of the parameter object. The stash folder
content is identical to the MsBackendMzRStash with only the class
definition of the OBJECT file being specific for the stored backend class.
Restoring a MsBackendMetaboLights from the stash folder imports the
metadata and, if not available in the local cache, downloads the mass
spectrometry data files from the MetaboLights repository. Synchronizing
the locally cached data requires an active internet connection, even if no
data needs to be downloaded. Setting parameter offline = TRUE avoids the
synchronization, but requires the mass spectrometry data files to be
present in the local cache.
The alabaster-format stashes can also be created and restored using the
saveObject() and readObject() functions.
saveMsObject() for MsBackendMetaboLights does not support (and does not
require) the consolidate parameter that most other backends implement to
create portable stashes. MsBackendMetaboLights backends retrieve and cache
the mass spectrometry data files if needed from the public repository and
are thus inherently portable.
Johannes Rainer
library(Spectra) library(SpectraStash) library(MsBackendMetaboLights) ## Retrieve mass spectrometry data files matching a specified file pattern ## for the data set MTBLS39 from MetaboLights s <- Spectra(mtblsId = "MTBLS39", source = MsBackendMetaboLights(), filePattern = "73A.cdf") ## Stash the `Spectra` object to a temporary folder. d <- file.path(tempdir(), "metabolights_stash") saveMsObject(s, AlabasterParam(d)) ## List the content of the SpectraStash dir(d) ## List the content of the backend stash (located within the *backend* ## sub-folder) dir(file.path(d, "backend")) ## Restore the `Spectra` object (with the respective backend); using ## `offline = TRUE` to avoid the lookup of the data set on the MetaboLights ## repository - all mass spectrometry data files of the backend must then be ## already present in the local cache. res <- readMsObject(Spectra(), AlabasterParam(d), offline = TRUE) reslibrary(Spectra) library(SpectraStash) library(MsBackendMetaboLights) ## Retrieve mass spectrometry data files matching a specified file pattern ## for the data set MTBLS39 from MetaboLights s <- Spectra(mtblsId = "MTBLS39", source = MsBackendMetaboLights(), filePattern = "73A.cdf") ## Stash the `Spectra` object to a temporary folder. d <- file.path(tempdir(), "metabolights_stash") saveMsObject(s, AlabasterParam(d)) ## List the content of the SpectraStash dir(d) ## List the content of the backend stash (located within the *backend* ## sub-folder) dir(file.path(d, "backend")) ## Restore the `Spectra` object (with the respective backend); using ## `offline = TRUE` to avoid the lookup of the data set on the MetaboLights ## repository - all mass spectrometry data files of the backend must then be ## already present in the local cache. res <- readMsObject(Spectra(), AlabasterParam(d), offline = TRUE) res
MsBackendMzR classes can be stashed to (or read from) plain text
file-based or alabaster-based formats using the saveMsObject() and
readMsObject() functions combined with the PlainTextParam and
AlabasterParam parameter objects, respectively. Setting parameter
consolidate = TRUE in the saveMsObject() or saveObject() function
will copy also the original MS data files into the folder generating a
self-consistent stash. This stash folder can also be copied to another
computer or location in the file system without the need to use parameter
spectraPath when restoring the data.
Additional properties of the stash formats are described in detail in the sections below.
## S4 method for signature 'MsBackendMzR,PlainTextParam' saveMsObject(object, param, consolidate = FALSE) ## S4 method for signature 'MsBackendMzR,PlainTextParam' readMsObject(object, param, spectraPath = character()) ## S4 method for signature 'MsBackendMzR' saveObject(x, path, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendMzR,AlabasterParam' saveMsObject(object, param, consolidate = FALSE) ## S4 method for signature 'MsBackendMzR,AlabasterParam' readMsObject(object, param, spectraPath = character())## S4 method for signature 'MsBackendMzR,PlainTextParam' saveMsObject(object, param, consolidate = FALSE) ## S4 method for signature 'MsBackendMzR,PlainTextParam' readMsObject(object, param, spectraPath = character()) ## S4 method for signature 'MsBackendMzR' saveObject(x, path, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendMzR,AlabasterParam' saveMsObject(object, param, consolidate = FALSE) ## S4 method for signature 'MsBackendMzR,AlabasterParam' readMsObject(object, param, spectraPath = character())
object |
An |
param |
Either a |
consolidate |
|
spectraPath |
For |
x |
An |
path |
For |
... |
Currently ignored. |
MsBackendMzR objects don't contain any peaks data (i.e., m/z and
intensity values) but retrieve these from the original MS data files (in
mzML, mzXML or CDF format). Unless consolidate = TRUE is used, a
MsBackendMzR stash will therefore only contain the spectra metadata
(i.e., the spectra variables) but no peaks data. The reference to the
original MS data files is stored as spectra variable dataStorage and if
the files are no longer available in the directory specified by
dataStorage the restored object will not be valid, unless the new location
is provided with parameter spectraPath.
readMsObject() returns an Spectra::MsBackendMzR object.
AlabasterParam
The saveMsObject() with an AlabasterParam parameter object stashes the
provided MsBackendMzR object in an alabster-based format into the
directory defined with argument param of the AlabasterParam.
readMsObject() with AlabasterParam restores a previously stashed
MsBackend object. Optional parameter spectraPath allows to specify the
storage path of the MS data files referenced by the MsBackendMzR (in case
they are no longer in the same directory when saving the object).
In addition, the alabaster methods saveObject() and readObject() can
be used to save and read MsBackendMzR objects.
PlainTextParam
The saveMsObject() function with the PlainTextParam stores the spectra
metadata (spectra variables) of an MsBackendMzR to a plain tabulator
delimited text file with the name ms_backend_spectra_data.txt in the
directory specified with parameter path of the PlainTextParam object.
Importantly, the peaks data (the m/z and intensity values) are not
exported with saveMsObject().
readMsObject() restores a previously stashed MsBackendMzR object from
the directory specified with parameter path of the PlainTextParam.
The additional parameter spectraPath of readMsObject() allows to define
the path to the MS data files containing the full MS data (i.e., the mzML,
mzXML or CDF files referred to by the MsBackendMzR).
Philippine Louail, Johannes Rainer
library(SpectraStash) library(Spectra) library(MsDataHub) ## Create a MsBackendMzR from test data be <- backendInitialize(MsBackendMzR(), PestMix1_DDA.mzML()) be ## Define a folder where to stash the object pth <- file.path(tempdir(), "mzr_stash") ## Stash the object to this folder in a plain text-based format saveMsObject(be, PlainTextParam(pth)) ## Restore the stashed object res <- readMsObject(MsBackendMzR(), PlainTextParam(pth)) res ## Clean-up unlink(pth, recursive = TRUE) ## Store the data in *alabaster* format including also the original MS ## data files (`consolidate = TRUE`) saveMsObject(be, AlabasterParam(pth), consolidate = TRUE) ## Get the directory content of the stash folder: dir(pth) ## Restore the object res <- readMsObject(MsBackendMzR(), AlabasterParam(pth)) res ## If the data is exported with `consolidate = FALSE` (the default), the ## new location of MS data files could be provided with parameter ## `spectraPath` of the `readMsObject()` function in case they are no ## longer in the path referenced by the stashed object. ## Clean-up unlink(pth, recursive = TRUE)library(SpectraStash) library(Spectra) library(MsDataHub) ## Create a MsBackendMzR from test data be <- backendInitialize(MsBackendMzR(), PestMix1_DDA.mzML()) be ## Define a folder where to stash the object pth <- file.path(tempdir(), "mzr_stash") ## Stash the object to this folder in a plain text-based format saveMsObject(be, PlainTextParam(pth)) ## Restore the stashed object res <- readMsObject(MsBackendMzR(), PlainTextParam(pth)) res ## Clean-up unlink(pth, recursive = TRUE) ## Store the data in *alabaster* format including also the original MS ## data files (`consolidate = TRUE`) saveMsObject(be, AlabasterParam(pth), consolidate = TRUE) ## Get the directory content of the stash folder: dir(pth) ## Restore the object res <- readMsObject(MsBackendMzR(), AlabasterParam(pth)) res ## If the data is exported with `consolidate = FALSE` (the default), the ## new location of MS data files could be provided with parameter ## `spectraPath` of the `readMsObject()` function in case they are no ## longer in the path referenced by the stashed object. ## Clean-up unlink(pth, recursive = TRUE)
MsBackendOfflineSql and MsBackendSql backend classes defined in the
MsBackendSql
Bioconductor package can be stashed (or read from) plain text file or
alabaster-based stash formats using the saveMsObject() and
readMsObject() functions configuring the stash with a PlainTextParam or
AlabasterParam parameter object. The Alabaster format is the preferred
stash format.
Additional properties of the stash formats are described in detail in the sections below.
## S4 method for signature 'MsBackendOfflineSql,PlainTextParam' saveMsObject(object, param, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendOfflineSql,PlainTextParam' readMsObject(object, param, password = character(), ...) ## S4 method for signature 'MsBackendSql,PlainTextParam' saveMsObject(object, param, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendSql,PlainTextParam' readMsObject(object, param, password = character()) ## S4 method for signature 'MsBackendOfflineSql' saveObject(x, path, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendOfflineSql,AlabasterParam' saveMsObject(object, param, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendOfflineSql,AlabasterParam' readMsObject(object, param, password = character()) ## S4 method for signature 'MsBackendSql' saveObject(x, path, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendSql,AlabasterParam' saveMsObject(object, param, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendSql,AlabasterParam' readMsObject(object, param, password = character())## S4 method for signature 'MsBackendOfflineSql,PlainTextParam' saveMsObject(object, param, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendOfflineSql,PlainTextParam' readMsObject(object, param, password = character(), ...) ## S4 method for signature 'MsBackendSql,PlainTextParam' saveMsObject(object, param, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendSql,PlainTextParam' readMsObject(object, param, password = character()) ## S4 method for signature 'MsBackendOfflineSql' saveObject(x, path, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendOfflineSql,AlabasterParam' saveMsObject(object, param, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendOfflineSql,AlabasterParam' readMsObject(object, param, password = character()) ## S4 method for signature 'MsBackendSql' saveObject(x, path, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendSql,AlabasterParam' saveMsObject(object, param, consolidate = FALSE, ...) ## S4 method for signature 'MsBackendSql,AlabasterParam' readMsObject(object, param, password = character())
object |
An |
param |
Either a |
consolidate |
|
... |
Currently ignored. |
password |
Optional |
x |
For |
path |
For |
readMsObject() returns a MsBackendSql::MsBackendOfflineSql
object or MsBackendSql::MsBackendSql object.
AlabasterParam
saveMsObject() with an AlabasterParam() stores the backend to a stash
folder in alabaster format. Individual components of the backend are
stored using base functionality of the alabaster.base Bioconductor package
into specific sub-folders within the main stash directory. These are:
dbinfo directory: contains a data.frame saved with alabaster.base
with the database-specific information such as database name, host etc.
spectra_ids directory: contains an integer vector saved with
alabaster.base with the IDs (indices) of the individual spectra.
The remaining directories hold the content of the MsBackendCached object
the MsBackendOfflineSql extends (i.e., all locally cached data). Using
parameter consolidate = TRUE will also store the database file within
the folder (if supported).
In addition to the saveMsObject() and readMsObject() also the
alabaster.base saveObject() and readObject() methods are directly
supported.
PlainTextParam
saveMsObject() with a PlainTextParam stores the content of a
MsBackendOfflineSql as text files into the stash folder. This creates
two text files:
ms_backend_dbinfo.txt: a two-column tabulator delimited text file with
columns "key" and "value" (first line being the header) with each
row representing a key-value pair with the SQL database connection
information. This includes "dbname" (the name of the database - for
SQLite databases the SQLite database file name), "dbdriver" the database
driver class (e.g. "RSQLite::SQLiteDriver" for SQLite databases),
"user" with the database user, "host" with the host running the
database and "port" with the respective port.
ms_backend_spectra_ids.txt: a single column text file (first row being
the header of the table) with the spectrum identifiers (of
type integer).
Remaining content, such as eventually present locally cached data, are
stored through the saveMsObject() function of the parent MsBackendCached
object (see MsBackendCachedStash for details).
For some SQL database systems it is possible to store also the database into
the stash folder (by setting consolidate = TRUE in the saveMsObject()
call). This is currently supported for:
SQLite-based databases (using the RSQLite package)
MsBackendSql backends are converted to a MsBackendOfflineSql before
saving into the stash, because database connections can not be serialized.
Thus, both classes use the same MsBackendSqlStash format (and both can
hence be restored from the same stash).
Johannes Rainer
## Create an example `Spectra` object library(Spectra) library(MsDataHub) s <- Spectra(MsDataHub::X20171016_POOL_POS_1_105.134.mzML()) ## Store the full data into a SQLite SQL database library(RSQLite) library(MsBackendSql) dbname <- tempfile() s <- setBackend(s, MsBackendOfflineSql(), dbname = dbname, drv = SQLite()) ## Stash the `Spectra` object to a temporary folder in alabaster format d <- file.path(tempdir(), "spectra_sql") saveMsObject(s, AlabasterParam(d)) ## List the content of the SpectraStash dir(d) ## List the content of the MsBackendOfflineSql stash, which is located ## within the *backend* folder dir(file.path(d, "backend")) ## Restore the object from the stash res <- readMsObject(Spectra(), AlabasterParam(d)) res ## Verify a SQL backend is used class(res@backend) ## Repeat using the plain text file format. We need to first remove ## the previously generated stash, as overwriting an existing stash ## is not supported. unlink(d, recursive = TRUE) ## Use `consolidate = TRUE` to store also the database into the stash saveMsObject(s, PlainTextParam(d), consolidate = TRUE) ## List the content of the SpectraStash dir(d) ## Restore the object from the stash res <- readMsObject(Spectra(), PlainTextParam(d)) res ## Verify a SQL backend is used class(res@backend)## Create an example `Spectra` object library(Spectra) library(MsDataHub) s <- Spectra(MsDataHub::X20171016_POOL_POS_1_105.134.mzML()) ## Store the full data into a SQLite SQL database library(RSQLite) library(MsBackendSql) dbname <- tempfile() s <- setBackend(s, MsBackendOfflineSql(), dbname = dbname, drv = SQLite()) ## Stash the `Spectra` object to a temporary folder in alabaster format d <- file.path(tempdir(), "spectra_sql") saveMsObject(s, AlabasterParam(d)) ## List the content of the SpectraStash dir(d) ## List the content of the MsBackendOfflineSql stash, which is located ## within the *backend* folder dir(file.path(d, "backend")) ## Restore the object from the stash res <- readMsObject(Spectra(), AlabasterParam(d)) res ## Verify a SQL backend is used class(res@backend) ## Repeat using the plain text file format. We need to first remove ## the previously generated stash, as overwriting an existing stash ## is not supported. unlink(d, recursive = TRUE) ## Use `consolidate = TRUE` to store also the database into the stash saveMsObject(s, PlainTextParam(d), consolidate = TRUE) ## List the content of the SpectraStash dir(d) ## Restore the object from the stash res <- readMsObject(Spectra(), PlainTextParam(d)) res ## Verify a SQL backend is used class(res@backend)
Spectra objects can be stashed to (or read from) plain text file-based or
alabaster-based formats using the saveMsObject() or readMsObject()
functions configured with the PlainTextParam or AlabasterParam parameter
objects, respectively. In both cases, the data from the Spectra object is
stored into the stash directory defined with the parameter object.
Depending on the used MS backend (and parameters used), the stash can also
contain the MS data files.
At present, Spectra objects using one of the following MS data backends
are supported:
MsBackendMzR: see MsBackendMzRStash for details and options.
MsBackendHdf5Peaks: see MsBackendHdf5PeaksStash for details and
options.
MsBackendMemory: see MsBackendMemoryStash for details and options.
MsBackendDataFrame: see MsBackendMemoryStash for details and options.
MsBackendOfflineSql (defined in the
MsBackendSql package):
see MsBackendSqlStash for details and options.
The data backend of any Spectra backend can eventually be changed to one
of the above backends using the Spectra::setBackend() method to support
saving the object into a Spectra stash. Support for additional backends
respectively data representations might also be provided by separate R
packages.
Details on the stash formats are provided in the respective sections below.
## S4 method for signature 'Spectra,PlainTextParam' saveMsObject(object, param, ...) ## S4 method for signature 'Spectra,PlainTextParam' readMsObject(object, param, ...) ## S4 method for signature 'Spectra' saveObject(x, path, ...) ## S4 method for signature 'Spectra,AlabasterParam' saveMsObject(object, param, ...) ## S4 method for signature 'Spectra,AlabasterParam' readMsObject(object, param, ...)## S4 method for signature 'Spectra,PlainTextParam' saveMsObject(object, param, ...) ## S4 method for signature 'Spectra,PlainTextParam' readMsObject(object, param, ...) ## S4 method for signature 'Spectra' saveObject(x, path, ...) ## S4 method for signature 'Spectra,AlabasterParam' saveMsObject(object, param, ...) ## S4 method for signature 'Spectra,AlabasterParam' readMsObject(object, param, ...)
object |
A |
param |
Either a |
... |
additional arguments passed to the |
x |
A |
path |
For |
readMsObject() returns a Spectra::Spectra object.
AlabasterParam
With AlabasterParam, the Spectra object will be exported (or imported)
through the alabaster framework as a set of JSON and/or HDF5 files.
The content of each slot is stored to a separate file with the name
matching the slot name (converted to snake_case). The object's MsBackend
is stored into a sub-folder backend within the stash folder.
For information on the MS backend's data stash see the respective documentation.
PlainTextParam
For this format, the data content of a Spectra object is stored into the
files:
spectra_slots.txt: plain text file containning the processing queue
variables (separated by a "|"), the processing log messages (separated
by a "|"), the processing chunk size and the MsBackend class used.
spectra_processing_queue.json: the object's processing queue serialized
in JSON format. It can be unserialized using jsonlite::unserializeJSON().
For information on the MS backend's data see the respective documentation.
Johannes Rainer, Philippine Louail
## Create a Spectra object from two example MS data files (from MsDataHub) library(Spectra) library(MsDataHub) s <- Spectra( c(X20171016_POOL_POS_1_105.134.mzML(), X20171016_POOL_POS_3_105.134.mzML())) s ## Filter the intensities of the Spectra removing peaks with an intensity ## below 100 s <- filterIntensity(s, intensity = 100) ## Define the format and location of the `Spectra` stash: use the ## *alabaster*-based format and store the stash in a folder named ## *spectra_stash* in a temporary directory d <- file.path(tempdir(), "spectra_stash") ap <- AlabasterParam(d) ## Stash the `Spectra` object copying in addition the MS data files into the ## stash (`consolidate = TRUE`). saveMsObject(s, ap, consolidate = TRUE) ## Show the content of the stash dir(d) ## Read the `Spectra` object from the stash: res <- readMsObject(Spectra(), ap) res ## It is also possible to read individual contents from the stash. The ## directory *backend* contains for example the stashed `MsBackend` of the ## `Spectra` object. To read only the `MsBackend`: ap2 <- AlabasterParam(file.path(d, "backend")) b <- readMsObject(MsBackendMzR(), ap2) b ## Alternatively, that data can also be read directly with the `readObject()` ## method from the *alabaster.base* package: library(alabaster.base) b <- readObject(file.path(d, "backend")) b## Create a Spectra object from two example MS data files (from MsDataHub) library(Spectra) library(MsDataHub) s <- Spectra( c(X20171016_POOL_POS_1_105.134.mzML(), X20171016_POOL_POS_3_105.134.mzML())) s ## Filter the intensities of the Spectra removing peaks with an intensity ## below 100 s <- filterIntensity(s, intensity = 100) ## Define the format and location of the `Spectra` stash: use the ## *alabaster*-based format and store the stash in a folder named ## *spectra_stash* in a temporary directory d <- file.path(tempdir(), "spectra_stash") ap <- AlabasterParam(d) ## Stash the `Spectra` object copying in addition the MS data files into the ## stash (`consolidate = TRUE`). saveMsObject(s, ap, consolidate = TRUE) ## Show the content of the stash dir(d) ## Read the `Spectra` object from the stash: res <- readMsObject(Spectra(), ap) res ## It is also possible to read individual contents from the stash. The ## directory *backend* contains for example the stashed `MsBackend` of the ## `Spectra` object. To read only the `MsBackend`: ap2 <- AlabasterParam(file.path(d, "backend")) b <- readMsObject(MsBackendMzR(), ap2) b ## Alternatively, that data can also be read directly with the `readObject()` ## method from the *alabaster.base* package: library(alabaster.base) b <- readObject(file.path(d, "backend")) b