Package 'MSstatsBig'

Convert out-of-memory DIANN files to MSstats format.

Description

Convert out-of-memory DIANN files to MSstats format.

Usage

bigDIANNtoMSstatsFormat(
  input_file,
  annotation = NULL,
  output_file_name,
  backend,
  MBR = TRUE,
  quantificationColumn = "FragmentQuantCorrected",
  global_qvalue_cutoff = 0.01,
  qvalue_cutoff = 0.01,
  pg_qvalue_cutoff = 0.01,
  max_feature_count = 100,
  filter_unique_peptides = FALSE,
  aggregate_psms = FALSE,
  filter_few_obs = FALSE,
  remove_annotation = FALSE,
  calculateAnomalyScores = FALSE,
  anomalyModelFeatures = c(),
  connection = NULL
)

Arguments

input_file	name of the input text file in 10-column MSstats format.
annotation	name of 'annotation.txt' data which includes Condition, BioReplicate, Run.
output_file_name	name of an output file which will be saved after pre-processing
backend	"arrow" or "sparklyr". Option "sparklyr" requires a spark installation and connection to spark instance provided in the 'connection' parameter.
MBR	True if analysis was done with match between runs
quantificationColumn	Use 'FragmentQuantCorrected'(default) column for quantified intensities for DIANN 1.8.x. Use 'FragmentQuantRaw' for quantified intensities for DIANN 1.9.x. Use 'auto' for quantified intensities for DIANN 2.x where each fragment intensity is a separate column, e.g. Fr0Quantity.
global_qvalue_cutoff	The qvalue cutoff for the Q.Value column, i.e. the run-specific precursor q-value. Default is 0.01.
qvalue_cutoff	If MBR is false, the qvalue cutoff for the Global.Q.Value column, i.e. global precursor q-value. If MBR is true, the qvalue cutoff for the Lib.Q.Value column, i.e. the q-value for the library created after the first MBR pass. Default is 0.01.
pg_qvalue_cutoff	If MBR is false, the qvalue cutoff for the Global.PG.Q.Value column, i.e. the global q-value for the protein group. If MBR is true, the qvalue cutoff for the Lib.PG.Q.Value column, i.e. the protein group q-value for the library created after the first MBR pass. Default is 0.01.
max_feature_count	maximum number of features per protein. Features will be selected based on highest average intensity.
filter_unique_peptides	If TRUE, shared peptides will be removed. Please refer to the 'Details' section for additional information.
aggregate_psms	If TRUE, multiple measurements per PSM in a Run will be aggregated (by taking maximum value). Please refer to the 'Details' section for additional information.
filter_few_obs	If TRUE, feature with less than 3 observations across runs will be removed. Please refer to the 'Details' section for additional information.
remove_annotation	If TRUE, columns BioReplicate and Condition will be removed to reduce output file size. These will need to be added manually later before using dataProcess function. Only applicable to sparklyr backend.
calculateAnomalyScores	If TRUE, will carry anomaly model features through pipeline
anomalyModelFeatures	Character vector of column names to be carried through the pipeline
connection	Connection to a spark instance created with the 'spark_connect' function from 'sparklyr' package.

Value

either arrow object or sparklyr table that can be optionally collected into memory by using dplyr::collect function.

---

Convert out-of-memory FragPipe files to MSstats format.

Description

Convert out-of-memory FragPipe files to MSstats format.

Usage

bigFragPipetoMSstatsFormat(
  input_file,
  output_file_name,
  backend,
  max_feature_count = 100,
  filter_unique_peptides = FALSE,
  aggregate_psms = FALSE,
  filter_few_obs = FALSE,
  remove_annotation = FALSE,
  connection = NULL
)

Arguments

input_file	name of the input text file in 10-column MSstats format.
output_file_name	name of an output file which will be saved after pre-processing
backend	"arrow" or "sparklyr". Option "sparklyr" requires a spark installation and connection to spark instance provided in the 'connection' parameter.
max_feature_count	maximum number of features per protein. Features will be selected based on highest average intensity.
filter_unique_peptides	If TRUE, shared peptides will be removed. Please refer to the 'Details' section for additional information.
aggregate_psms	If TRUE, multiple measurements per PSM in a Run will be aggregated (by taking maximum value). Please refer to the 'Details' section for additional information.
filter_few_obs	If TRUE, feature with less than 3 observations across runs will be removed. Please refer to the 'Details' section for additional information.
remove_annotation	If TRUE, columns BioReplicate and Condition will be removed to reduce output file size. These will need to be added manually later before using dataProcess function. Only applicable to sparklyr backend.
connection	Connection to a spark instance created with the 'spark_connect' function from 'sparklyr' package.

Value

either arrow object or sparklyr table that can be optionally collected into memory by using dplyr::collect function.

Examples

converted_data <- bigFragPipetoMSstatsFormat(
  system.file("extdata", "fgexample.csv", package = "MSstatsBig"),
  "output_file.csv",
  backend = "arrow")
converted_data <- dplyr::collect(converted_data)
head(converted_data)

---

Convert out-of-memory Spectronaut files to MSstats format.

Description

Convert out-of-memory Spectronaut files to MSstats format.

Usage

bigSpectronauttoMSstatsFormat(
  input_file,
  output_file_name,
  backend,
  intensity = "F.NormalizedPeakArea",
  filter_by_excluded = FALSE,
  filter_by_identified = FALSE,
  filter_by_qvalue = FALSE,
  qvalue_cutoff = 0.01,
  max_feature_count = 100,
  filter_unique_peptides = FALSE,
  aggregate_psms = FALSE,
  filter_few_obs = FALSE,
  remove_annotation = FALSE,
  calculateAnomalyScores = FALSE,
  anomalyModelFeatures = c(),
  annotation = NULL,
  connection = NULL,
  block_size = 16L * 1024L * 1024L
)

Arguments

input_file	name of the input text file in 10-column MSstats format.
output_file_name	name of an output file which will be saved after pre-processing
backend	"arrow" or "sparklyr". Option "sparklyr" requires a spark installation and connection to spark instance provided in the 'connection' parameter.
intensity	Name of the intensity column to be used in Spectronaut
filter_by_excluded	if TRUE, will filter by the 'F.ExcludedFromQuantification' column.
filter_by_identified	if TRUE, will filter by the 'EG.Identified' column.
filter_by_qvalue	if TRUE, will filter by EG.Qvalue and PG.Qvalue columns.
qvalue_cutoff	cutoff which will be used for q-value filtering.
max_feature_count	maximum number of features per protein. Features will be selected based on highest average intensity.
filter_unique_peptides	If TRUE, shared peptides will be removed. Please refer to the 'Details' section for additional information.
aggregate_psms	If TRUE, multiple measurements per PSM in a Run will be aggregated (by taking maximum value). Please refer to the 'Details' section for additional information.
filter_few_obs	If TRUE, feature with less than 3 observations across runs will be removed. Please refer to the 'Details' section for additional information.
remove_annotation	If TRUE, columns BioReplicate and Condition will be removed to reduce output file size. These will need to be added manually later before using dataProcess function. Only applicable to sparklyr backend.
calculateAnomalyScores	If TRUE, will carry anomaly model features through pipeline
anomalyModelFeatures	Character vector of column names to be carried through the pipeline
annotation	Optional data.frame with columns 'Run', 'BioReplicate', 'Condition' (plus any additional annotation columns). If supplied, the converter merges it onto the output on 'Run' and overrides any 'Condition' / 'BioReplicate' values that came from Spectronaut's 'R.Condition' / 'R.Replicate' columns. Required when the experimental design cannot be expressed in Spectronaut's own annotation — most notably paired designs, where 'BioReplicate' must encode the pairing structure rather than per-sample IDs.
connection	Connection to a spark instance created with the 'spark_connect' function from 'sparklyr' package.
block_size	Arrow CSV reader block size in bytes: the amount of raw CSV that is read and parsed at a time (roughly one record batch per block). Defaults to 16 MiB ('16L * 1024L * 1024L'). Two things matter when tuning it: It must be larger than your single widest row. If you see 'Invalid: straddling object straddles two block boundaries' on extra-wide exports, raise it (e.g. '64L * 1024L * 1024L'). Larger blocks reduce parsing overhead, but peak memory grows roughly linearly with 'block_size', so it is also the main lever on out-of-memory risk. Gains taper off quickly because per-batch time is dominated by row cleaning, not by block size. To speed up processing without risking a crash, double the value (32 -> 64 -> 128 MiB) while watching memory use, and stop once throughput stops improving. Keep it comfortably below free RAM – and below free RAM divided by the number of jobs if you run several conversions in parallel.

Details

'bigSpectronauttoMSstatsFormat' consumes the following Spectronaut export columns.

Always required:

• 'R.FileName', 'R.Condition', 'R.Replicate' -> Run, Condition, BioReplicate

• 'PG.ProteinAccessions' -> ProteinName

• 'EG.ModifiedSequence' -> PeptideSequence

• 'FG.Charge' -> PrecursorCharge; 'F.FrgIon' -> FragmentIon; 'F.Charge' -> ProductCharge

• the intensity column ('F.NormalizedPeakArea' by default; see 'intensity') -> Intensity

• 'F.FrgLossType' -> used by the always-on 'noloss' fragment filter

Required only when the matching filter is enabled:

• 'EG.Identified' -> 'filter_by_identified'

• 'F.ExcludedFromQuantification' -> 'filter_by_excluded'

• 'EG.Qvalue' and 'PG.Qvalue' -> 'filter_by_qvalue'

A missing required column raises an informative error naming the column.

Value

either arrow object or sparklyr table that can be optionally collected into memory by using dplyr::collect function.

Examples

converted_data <- bigSpectronauttoMSstatsFormat(
  system.file("extdata", "spectronaut_input.csv", package = "MSstatsBig"),
  "output_file.csv",
  backend = "arrow")
converted_data <- dplyr::collect(converted_data)
head(converted_data)

# Override Spectronaut's embedded Condition / BioReplicate with
# a custom annotation (e.g. for a paired design):
annot <- data.frame(Run = unique(converted_data[["Run"]]))
annot$BioReplicate <- seq_len(nrow(annot))
annot$Condition <- rep(c("ctrl", "treat"), length.out = nrow(annot))
overridden <- bigSpectronauttoMSstatsFormat(
  system.file("extdata", "spectronaut_input.csv", package = "MSstatsBig"),
  "output_file.csv",
  backend = "arrow",
  annotation = annot)
head(dplyr::collect(overridden))

---

Merge annotation to output of MSstatsPreprocessBig

Description

Merge annotation to output of MSstatsPreprocessBig

Usage

MSstatsAddAnnotationBig(input, annotation)

Arguments

input	output of MSstatsPreprocessBig
annotation	run annotation

Value

table of 'input' and 'annotation' merged by Run column.

Examples

converted_data <- bigFragPipetoMSstatsFormat(
  system.file("extdata", "fgexample.csv", package = "MSstatsBig"),
  "output_file.csv",
  backend = "arrow")
converted_data <- dplyr::collect(converted_data)
head(converted_data)
# Change annotation as an example:
converted_data$Condition <- NULL
converted_data$BioReplicate <- NULL
annot <- data.frame(Run = unique(converted_data[["Run"]]))
annot$BioReplicate <- rep(1:53, times = 2)
annot$Condition <- rep(1:2, each = 53)
head(MSstatsAddAnnotationBig(converted_data, annot))

---

General converter for larger-than-memory csv files in MSstats format 10-column format

Description

General converter for larger-than-memory csv files in MSstats format 10-column format

Usage

MSstatsPreprocessBig(
  input_file,
  output_file_name,
  backend,
  max_feature_count = 100,
  filter_unique_peptides = FALSE,
  aggregate_psms = FALSE,
  filter_few_obs = FALSE,
  remove_annotation = FALSE,
  calculateAnomalyScores = FALSE,
  anomalyModelFeatures = c(),
  connection = NULL
)

Arguments

input_file	name of the input text file in 10-column MSstats format.
output_file_name	name of an output file which will be saved after pre-processing
backend	"arrow" or "sparklyr". Option "sparklyr" requires a spark installation and connection to spark instance provided in the 'connection' parameter.
max_feature_count	maximum number of features per protein. Features will be selected based on highest average intensity.
filter_unique_peptides	If TRUE, shared peptides will be removed. Please refer to the 'Details' section for additional information.
aggregate_psms	If TRUE, multiple measurements per PSM in a Run will be aggregated (by taking maximum value). Please refer to the 'Details' section for additional information.
filter_few_obs	If TRUE, feature with less than 3 observations across runs will be removed. Please refer to the 'Details' section for additional information.
remove_annotation	If TRUE, columns BioReplicate and Condition will be removed to reduce output file size. These will need to be added manually later before using dataProcess function. Only applicable to sparklyr backend.
calculateAnomalyScores	If TRUE, will carry anomaly model features through pipeline
anomalyModelFeatures	Character vector of column names to be carried through the pipeline
connection	Connection to a spark instance created with the 'spark_connect' function from 'sparklyr' package.

Details

Filtering and aggregation may be very time consuming and the ability to perform them in a given R session depends on available memory, settings of external packages, etc. Hence, all value of related parameters ('filter_unique_peptides', 'aggregate_psms', 'filter_few_obs') are set to FALSE by default and only feature selection is performed, which saves both computation time and memory. Appropriately configured spark backend provides the most consistent way to perform these operations.

Value

either arrow object or sparklyr table that can be optionally collected into memory by using dplyr::collect function.

Examples

converted_data <- bigFragPipetoMSstatsFormat(
  system.file("extdata", "fgexample.csv", package = "MSstatsBig"),
  "tencol_format.csv",
  backend="arrow")
procd <- MSstatsPreprocessBig("tencol_format.csv", "proc_out.csv", backend = "arrow")
head(dplyr::collect(procd))

---