Package: DelayedArray 0.31.5
Hervé Pagès
DelayedArray: A unified framework for working transparently with on-disk and in-memory array-like datasets
Wrapping an array-like object (typically an on-disk object) in a DelayedArray object allows one to perform common array operations on it without loading the object in memory. In order to reduce memory usage and optimize performance, operations on the object are either delayed or executed using a block processing mechanism. Note that this also works on in-memory array-like objects like DataFrame objects (typically with Rle columns), Matrix objects, ordinary arrays and, data frames.
Authors:
DelayedArray_0.31.5.tar.gz
DelayedArray_0.31.5.zip(r-4.5)DelayedArray_0.31.5.zip(r-4.4)DelayedArray_0.31.5.zip(r-4.3)
DelayedArray_0.31.5.tgz(r-4.4-arm64)DelayedArray_0.31.5.tgz(r-4.4-x86_64)DelayedArray_0.31.5.tgz(r-4.3-arm64)DelayedArray_0.31.5.tgz(r-4.3-x86_64)
DelayedArray_0.31.5.tar.gz(r-4.5-noble)DelayedArray_0.31.5.tar.gz(r-4.4-noble)
DelayedArray_0.31.5.tgz(r-4.4-emscripten)DelayedArray_0.31.5.tgz(r-4.3-emscripten)
DelayedArray.pdf |DelayedArray.html✨
DelayedArray/json (API)
NEWS
| # Install 'DelayedArray' in R: |
| install.packages('DelayedArray', repos = c('https://bioc.r-universe.dev', 'https://cloud.r-project.org')) |
Bug tracker:https://github.com/bioconductor/delayedarray/issues
On BioConductor:DelayedArray-0.31.4(bioc 3.20)DelayedArray-0.30.1(bioc 3.19)
Last updated 8 days agofrom:654263929d
Exports:acbindapermaperm.DelayedArrayaperm.SparseArraySeedapplyarbindArbitraryArrayGridas.array.SparseArraySeedas.data.frameas.matrixas.matrix.SparseArraySeedAutoRealizationSinkbindROWSBLOCK_write_to_sinkblockApplyblockGridblockReducecbindchunkdimchunkGridclosecoercecolAutoGridcolGridcolMaxscolMeanscolMinscolRangescolsumcolSumscolVarsConstantArrayConstantArraySeedcontentIsPristinecurrentBlockIdcurrentViewportdbinomdefaultAutoGriddefaultMultAutoGridsdefaultSinkAutoGridDelayedArraydense2sparsedimsdlogisdnormdpoisdropDummyArrayGrideffectiveGridendextract_arrayextract_sparse_arrayget_type_sizegetAutoBlockLengthgetAutoBlockShapegetAutoBlockSizegetAutoBPPARAMgetAutoGridMakergetAutoRealizationBackendgetRealizationBackendgreplgridApplygridReducegsubis_noopis_sparseisEmptyisLinearisPristinemakeCappedVolumeBoxmakeNindexFromArrayViewportmakeRegularArrayGridOfCappedLengthViewportsmatrixClassmeanmean.DelayedArraymean.SparseArraySeedmodify_seedsmultGridsncharnetSubsetAndApermnew_DelayedArraynseednzdatanzindexOLD_extract_sparse_arraypathpath<-pbinomplogispmax2pmin2pnormppoisqbinomqlogisqnormqpoisrange.DelayedArrayrange.SparseArraySeedrangesrbindread_blockread_sparse_blockrealizeregisteredRealizationBackendsRegularArrayGridRleArrayrowAutoGridrowGridrowMaxsrowMeansrowMinsrowRangesrowsumrowsum.DelayedMatrixrowSumsrowVarsscalescale.DelayedMatrixseedseed<-seedApplyset_grid_contextsetAutoBlockShapesetAutoBlockSizesetAutoBPPARAMsetAutoGridMakersetAutoRealizationBackendsetRealizationBackendshowshowAsCellshowtreesimplifysinkApplysparse2denseSparseArraySeedsplitsplit.DelayedArraysplitAsListstartsubsummarysummary.DelayedAbindsummary.DelayedApermsummary.DelayedNaryIsoOpsummary.DelayedOpsummary.DelayedSetDimnamessummary.DelayedSubsetsummary.DelayedUnaryIsoOpStacksummary.DelayedUnaryIsoOpWithArgssupportedRealizationBackendssweepttabletolowertouppertypetype<-uniqueunique.DelayedArrayupdateObjectviewportApplyviewportReducewhichwidthwrite_block
Dependencies:abindBiocGenericscrayonIRangeslatticeMatrixMatrixGenericsmatrixStatsS4ArraysS4VectorsSparseArrayXVectorzlibbioc
Working with large arrays in R (slides from July 2017)
Rendered fromA-Working_with_large_arrays.Rnwusingknitr::knitron Jul 05 2024.Last update: 2024-05-06
Started: 2024-05-06
Implementing A DelayedArray Backend
Rendered fromB-Implementing_a_backend.Rmdusingknitr::rmarkdownon Jul 05 2024.Last update: 2024-05-06
Started: 2024-05-06
A DelayedArray / HDF5Array update (slides from April 2021)
Rendered fromC-DelayedArray_HDF5Array_update.Rnwusingknitr::knitron Jul 05 2024.Last update: 2024-05-06
Started: 2024-05-06
Readme and manuals
Help Manual
| Help page | Topics |
|---|---|
| Control the geometry of automatic blocks | AutoBlock-global-settings getAutoBlockLength getAutoBlockShape getAutoBlockSize get_type_size setAutoBlockShape setAutoBlockSize |
| Create automatic grids to use for block processing of array-like objects | AutoGrid blockGrid colAutoGrid colGrid defaultAutoGrid defaultSinkAutoGrid getAutoGridMaker rowAutoGrid rowGrid setAutoGridMaker |
| blockApply() and family | block processing blockApply blockReduce block_processing currentBlockId currentViewport effectiveGrid getAutoBPPARAM gridApply gridReduce setAutoBPPARAM set_grid_context viewportApply viewportReduce |
| A DelayedArray subclass that contains a constant value | class:ConstantArray class:ConstantArraySeed class:ConstantMatrix coerce,ConstantArray,ConstantMatrix-method coerce,ConstantMatrix,ConstantArray-method ConstantArray ConstantArray-class ConstantArraySeed ConstantArraySeed-class ConstantMatrix ConstantMatrix-class DelayedArray,ConstantArraySeed-method extract_array,ConstantArraySeed-method extract_sparse_array,ConstantArraySeed-method is_sparse,ConstantArraySeed-method matrixClass,ConstantArray-method OLD_extract_sparse_array,ConstantArraySeed-method |
| DelayedAbind objects | class:DelayedAbind DelayedAbind DelayedAbind-class dim,DelayedAbind-method dimnames,DelayedAbind-method extract_array,DelayedAbind-method extract_sparse_array,DelayedAbind-method is_noop,DelayedAbind-method is_sparse,DelayedAbind-method OLD_extract_sparse_array,DelayedAbind-method summary,DelayedAbind-method summary.DelayedAbind updateObject,SeedBinder-method |
| DelayedAperm objects | class:DelayedAperm DelayedAperm DelayedAperm-class dim,DelayedAperm-method dimnames,DelayedAperm-method extract_array,DelayedAperm-method extract_sparse_array,DelayedAperm-method is_noop,DelayedAperm-method is_sparse,DelayedAperm-method OLD_extract_sparse_array,DelayedAperm-method summary,DelayedAperm-method summary.DelayedAperm updateObject,SeedDimPicker-method |
| DelayedArray objects | aperm,DelayedArray-method aperm.DelayedArray c,DelayedArray-method class:DelayedArray class:DelayedArray1 class:DelayedMatrix coerce,DelayedArray,COO_SparseArray-method coerce,DelayedArray,DelayedMatrix-method coerce,DelayedArray,SparseArray-method coerce,DelayedArray,SparseArraySeed-method coerce,DelayedMatrix,DelayedArray-method DelayedArray DelayedArray,ANY-method DelayedArray,DelayedArray-method DelayedArray,DelayedOp-method DelayedArray-class DelayedArray1 DelayedArray1-class DelayedMatrix DelayedMatrix-class dim,DelayedArray-method dim<-,DelayedArray-method dimnames,DelayedArray-method dimnames<-,DelayedArray,ANY-method drop,DelayedArray-method extract_array,DelayedArray-method matrixClass matrixClass,DelayedArray-method names,DelayedArray-method names<-,DelayedArray-method new_DelayedArray show,DelayedArray-method split,DelayedArray,ANY-method split.DelayedArray splitAsList,DelayedArray-method type updateObject,DelayedArray-method [,DelayedArray-method [<-,DelayedArray-method [[,DelayedArray-method |
| Statistical functions on DelayedArray objects | dbinom dbinom,DelayedArray-method DelayedArray-stats dlogis dlogis,DelayedArray-method dnorm dnorm,DelayedArray-method dpois dpois,DelayedArray-method pbinom pbinom,DelayedArray-method plogis plogis,DelayedArray-method pnorm pnorm,DelayedArray-method ppois ppois,DelayedArray-method qbinom qbinom,DelayedArray-method qlogis qlogis,DelayedArray-method qnorm qnorm,DelayedArray-method qpois qpois,DelayedArray-method |
| Common operations on DelayedArray objects | !,DelayedArray-method +,DelayedArray,missing-method -,DelayedArray,missing-method acbind,DelayedArray-method anyNA,DelayedArray-method apply apply,DelayedArray-method arbind,DelayedArray-method bindROWS,DelayedArray-method cbind cbind,DelayedArray-method DelayedArray-utils grepl,ANY,DelayedArray-method gsub,ANY,ANY,DelayedArray-method is.finite,DelayedArray-method is.infinite,DelayedArray-method is.na,DelayedArray-method is.nan,DelayedArray-method lengths,DelayedArray-method log,DelayedArray-method mean mean,DelayedArray-method mean.DelayedArray nchar,DelayedArray-method pmax2 pmax2,ANY,ANY-method pmax2,DelayedArray,DelayedArray-method pmax2,DelayedArray,vector-method pmax2,vector,DelayedArray-method pmin2 pmin2,ANY,ANY-method pmin2,DelayedArray,DelayedArray-method pmin2,DelayedArray,vector-method pmin2,vector,DelayedArray-method range range,DelayedArray-method range.DelayedArray rbind rbind,DelayedArray-method round,DelayedArray-method scale scale,DelayedMatrix-method scale.DelayedMatrix signif,DelayedArray-method sub,ANY,ANY,DelayedArray-method sweep sweep,DelayedArray-method table table,DelayedArray-method tolower,DelayedArray-method toupper,DelayedArray-method type<-,DelayedArray-method unique unique,DelayedArray-method unique.DelayedArray which,DelayedArray-method |
| DelayedMatrix multiplication and cross-product | %*% %*%,ANY,DelayedMatrix-method %*%,DelayedMatrix,ANY-method %*%,DelayedMatrix,DelayedMatrix-method crossprod crossprod,ANY,DelayedMatrix-method crossprod,DelayedMatrix,ANY-method crossprod,DelayedMatrix,DelayedMatrix-method crossprod,DelayedMatrix,missing-method DelayedMatrix-mult tcrossprod tcrossprod,ANY,DelayedMatrix-method tcrossprod,DelayedMatrix,ANY-method tcrossprod,DelayedMatrix,DelayedMatrix-method tcrossprod,DelayedMatrix,missing-method |
| rowsum() and colsum() on a DelayedMatrix object | colsum colsum,DelayedMatrix-method DelayedMatrix-rowsum rowsum rowsum,DelayedMatrix-method rowsum.DelayedMatrix |
| DelayedMatrix row/col summarization | colMaxs colMaxs,DelayedMatrix-method colMeans colMeans,DelayedMatrix-method colMins colMins,DelayedMatrix-method colRanges colRanges,DelayedMatrix-method colSums colSums,DelayedMatrix-method colVars colVars,DelayedMatrix-method DelayedMatrix-stats rowMaxs rowMaxs,DelayedMatrix-method rowMeans rowMeans,DelayedMatrix-method rowMins rowMins,DelayedMatrix-method rowRanges rowRanges,DelayedMatrix-method rowSums rowSums,DelayedMatrix-method rowVars rowVars,DelayedMatrix-method |
| DelayedNaryIsoOp objects | class:DelayedNaryIsoOp DelayedNaryIsoOp DelayedNaryIsoOp-class dim,DelayedNaryIsoOp-method dimnames,DelayedNaryIsoOp-method extract_array,DelayedNaryIsoOp-method extract_sparse_array,DelayedNaryIsoOp-method is_sparse,DelayedNaryIsoOp-method OLD_extract_sparse_array,DelayedNaryIsoOp-method summary,DelayedNaryIsoOp-method summary.DelayedNaryIsoOp updateObject,ConformableSeedCombiner-method |
| DelayedOp objects | class:DelayedNaryOp class:DelayedOp class:DelayedUnaryIsoOp class:DelayedUnaryOp DelayedNaryOp DelayedNaryOp-class DelayedOp DelayedOp-class DelayedUnaryIsoOp DelayedUnaryIsoOp-class DelayedUnaryOp DelayedUnaryOp-class dim,DelayedUnaryIsoOp-method dimnames,DelayedUnaryIsoOp-method extract_array,DelayedUnaryIsoOp-method extract_sparse_array,DelayedUnaryIsoOp-method is_noop is_sparse,DelayedUnaryIsoOp-method OLD_extract_sparse_array,DelayedUnaryIsoOp-method summary,DelayedOp-method summary.DelayedOp updateObject,DelayedOp-method |
| DelayedSetDimnames objects | class:DelayedSetDimnames DelayedSetDimnames DelayedSetDimnames-class dimnames,DelayedSetDimnames-method is_noop,DelayedSetDimnames-method summary,DelayedSetDimnames-method summary.DelayedSetDimnames updateObject,DelayedDimnames-method |
| DelayedSubassign objects | class:DelayedSubassign DelayedSubassign DelayedSubassign-class extract_array,DelayedSubassign-method extract_sparse_array,DelayedSubassign-method is_noop,DelayedSubassign-method is_sparse,DelayedSubassign-method OLD_extract_sparse_array,DelayedSubassign-method summary,DelayedSubassign-method summary.DelayedSubassign |
| DelayedSubset objects | class:DelayedSubset DelayedSubset DelayedSubset-class dim,DelayedSubset-method dimnames,DelayedSubset-method extract_array,DelayedSubset-method extract_sparse_array,DelayedSubset-method is_noop,DelayedSubset-method is_sparse,DelayedSubset-method OLD_extract_sparse_array,DelayedSubset-method summary,DelayedSubset-method summary.DelayedSubset |
| DelayedUnaryIsoOpStack objects | class:DelayedUnaryIsoOpStack DelayedUnaryIsoOpStack DelayedUnaryIsoOpStack-class extract_array,DelayedUnaryIsoOpStack-method extract_sparse_array,DelayedUnaryIsoOpStack-method is_sparse,DelayedUnaryIsoOpStack-method OLD_extract_sparse_array,DelayedUnaryIsoOpStack-method summary,DelayedUnaryIsoOpStack-method summary.DelayedUnaryIsoOpStack |
| DelayedUnaryIsoOpWithArgs objects | class:DelayedUnaryIsoOpWithArgs DelayedUnaryIsoOpWithArgs DelayedUnaryIsoOpWithArgs-class extract_array,DelayedUnaryIsoOpWithArgs-method extract_sparse_array,DelayedUnaryIsoOpWithArgs-method is_sparse,DelayedUnaryIsoOpWithArgs-method OLD_extract_sparse_array,DelayedUnaryIsoOpWithArgs-method summary,DelayedUnaryIsoOpWithArgs-method summary.DelayedUnaryIsoOpWithArgs |
| Utilities to make capped volume boxes | capped_volume_boxes isLinear isLinear,ArrayGrid-method isLinear,ArrayViewport-method makeCappedVolumeBox makeRegularArrayGridOfCappedLengthViewports |
| RealizationSink objects | arrayRealizationSink-class AutoRealizationSink class:arrayRealizationSink class:RealizationSink close,RealizationSink-method coerce,arrayRealizationSink,DelayedArray-method dim,arrayRealizationSink-method getAutoRealizationBackend getRealizationBackend RealizationSink RealizationSink-class registeredRealizationBackends setAutoRealizationBackend setRealizationBackend sinkApply supportedRealizationBackends write_block,arrayRealizationSink-method |
| Realize an object in memory or on disk | BLOCK_write_to_sink realize realize,ANY-method |
| RleArray objects | class:RleArray class:RleMatrix coerce,ANY,RleArray-method coerce,ANY,RleMatrix-method coerce,DataFrame,RleArray-method coerce,DelayedArray,RleArray-method coerce,DelayedMatrix,DataFrame-method coerce,DelayedMatrix,RleMatrix-method coerce,RleArray,Rle-method coerce,RleArray,RleMatrix-method coerce,RleList,RleArray-method coerce,RleMatrix,DataFrame-method coerce,RleMatrix,RleArray-method coerce,RleMatrix,RleList-method coerce,RleRealizationSink,DelayedArray-method coerce,RleRealizationSink,RleArray-method DelayedArray,RleArraySeed-method matrixClass,RleArray-method RleArray RleArray-class RleMatrix RleMatrix-class write_block,RleRealizationSink-method |
| RleArraySeed objects | ChunkedRleArraySeed ChunkedRleArraySeed-class class:ChunkedRleArraySeed class:RleArraySeed class:RleRealizationSink class:SolidRleArraySeed coerce,ChunkedRleArraySeed,SolidRleArraySeed-method coerce,RleRealizationSink,ChunkedRleArraySeed-method coerce,RleRealizationSink,Rle-method coerce,RleRealizationSink,RleList-method coerce,SolidRleArraySeed,Rle-method dim,RleArraySeed-method dimnames,RleArraySeed-method extract_array,ChunkedRleArraySeed-method extract_array,SolidRleArraySeed-method RleArraySeed RleArraySeed-class RleRealizationSink RleRealizationSink-class SolidRleArraySeed SolidRleArraySeed-class type,RleRealizationSink-method |
| Visualize and access the leaves of a tree of delayed operations | modify_seeds nseed nseed,ANY-method path path,DelayedOp-method path<-,DelayedOp-method seed seed,DelayedOp-method seed<- seed<-,DelayedOp-method seedApply show,DelayedOp-method showtree |
| Simplify a tree of delayed operations | contentIsPristine isPristine netSubsetAndAperm netSubsetAndAperm,ANY-method netSubsetAndAperm,DelayedArray-method simplify simplify,ANY-method simplify,DelayedAbind-method simplify,DelayedAperm-method simplify,DelayedArray-method simplify,DelayedNaryIsoOp-method simplify,DelayedSetDimnames-method simplify,DelayedSubassign-method simplify,DelayedSubset-method simplify,DelayedUnaryIsoOpStack-method simplify,DelayedUnaryIsoOpWithArgs-method |
| SparseArraySeed objects | aperm,SparseArraySeed-method aperm.SparseArraySeed as.array,SparseArraySeed-method as.array.SparseArraySeed as.matrix,SparseArraySeed-method as.matrix.SparseArraySeed class:SparseArraySeed coerce,ANY,SparseArraySeed-method coerce,dgCMatrix,SparseArraySeed-method coerce,dgRMatrix,SparseArraySeed-method coerce,lgCMatrix,SparseArraySeed-method coerce,lgRMatrix,SparseArraySeed-method coerce,SparseArraySeed,COO_SparseArray-method coerce,SparseArraySeed,CsparseMatrix-method coerce,SparseArraySeed,dgCMatrix-method coerce,SparseArraySeed,dgRMatrix-method coerce,SparseArraySeed,lgCMatrix-method coerce,SparseArraySeed,lgRMatrix-method coerce,SparseArraySeed,RsparseMatrix-method coerce,SparseArraySeed,sparseMatrix-method dense2sparse dimnames,SparseArraySeed-method dimnames<-,SparseArraySeed,ANY-method extract_array,SparseArraySeed-method is_sparse,SparseArraySeed-method nzdata nzdata,SparseArraySeed-method nzindex nzindex,SparseArraySeed-method OLD_extract_sparse_array OLD_extract_sparse_array,dgCMatrix-method OLD_extract_sparse_array,dgRMatrix-method OLD_extract_sparse_array,lgCMatrix-method OLD_extract_sparse_array,lgRMatrix-method OLD_extract_sparse_array,SparseArraySeed-method sparse2dense SparseArraySeed SparseArraySeed-class |
| Operate natively on SparseArraySeed objects | all,SparseArraySeed-method any,SparseArraySeed-method anyNA,SparseArraySeed-method cbind,SparseArraySeed-method is.infinite,SparseArraySeed-method is.na,SparseArraySeed-method is.nan,SparseArraySeed-method max,SparseArraySeed-method mean,SparseArraySeed-method mean.SparseArraySeed min,SparseArraySeed-method nchar,SparseArraySeed-method prod,SparseArraySeed-method range,SparseArraySeed-method range.SparseArraySeed rbind,SparseArraySeed-method SparseArraySeed-utils sum,SparseArraySeed-method tolower,SparseArraySeed-method toupper,SparseArraySeed-method which,SparseArraySeed-method |
