NEWS

---

matter 2.9.1

BUG FIXES

• Change C-level 'Free()' calls to 'R_Free()'

matter 2.7.10 (2024-10-16)

NEW FEATURES

• Add 'fetch' and 'flash()' for base R types

SIGNIFICANT USER-VISIBLE CHANGES

• Update 'fetch()' and 'flash()' to pass more arguments to underlying 'matter' constructors

matter 2.7.9

NEW FEATURES

• Add 'pivots' argument to 'fastmap()' to support more than 2 pivot candidates per iteration

SIGNIFICANT USER-VISIBLE CHANGES

• Update 'fetch()' and 'flash()' to preserve atoms types
• Update 'fetch()' and 'flash()' to fall back to serial execution if BPPARAM is a distributed cluster

BUG FIXES

• Add 'chunkopts' to 'nscentroids()' and 'sgmix()' signatures to fix downstream '...' bugs
• Fix graphical parameters not applying to vizi plots

matter 2.7.8

BUG FIXES

• Fix shared memory segmentation faults on Linux
• Fix voxel plotting when length(unique(z)) == 1L

matter 2.7.7

NEW FEATURES

• New 'fetch()' and 'flash()' generics for moving data between file storage and shared memory

SIGNIFICANT USER-VISIBLE CHANGES

• Support both number of items OR size in bytes when setting getOption("matter.default.chunksize")
• Now using '@' prefix as shared memory identifier for file system compatibility (e.g., Windows)
• Deprecated 'sgmixn()'; updated 'sgmix()' to support multichannel images directly
• Improved RNG behavior for non-L'Ecuyer seeds

BUG FIXES

• Fix potential infinite loop in 'mi_learn()'
• Add warning in chunk-apply functions if 'RNG=TRUE' but 'RNGkind()' is _not_ "L'Ecuyer-CMRG"

matter 2.7.6

SIGNIFICANT USER-VISIBLE CHANGES

• Add 'cpals()' and 'dpals()' to list available palettes
• Moving 'simple_logger' now appends 'sessionInfo()'
• Statistical methods 'fastmap()' and 'nscentroids()' now use 'rowDists()' and 'colDists()' directly
• Optional argument 'distfun' has new requirements in methods 'fastmap()' and 'nscentroids()'
• Class 'matter_str' now inherits from 'matter_list' for consistency between R and C++ interfaces
• Export '.rowDists()' and '.colDists()' functions

BUG FIXES

• Fix 'simple_logger' finalizer on R session exit

matter 2.7.5

NEW FEATURES

• Shared memory support comes to 'matter' via boost!
• Add 'as.shared()' for coercion to shared memory 'matter' objects
• Use 'path=":memory:"' to construct 'matter' object in shared memory
• Use 'mem()' to monitor shared memory usage

SIGNIFICANT USER-VISIBLE CHANGES

• Add shared memory support to 'mem()' output
• Add cluster memory support to 'memtime()' output
• Add cluster memory monitoring with 'memcl()' function
• New documentation for several previously-undocumented utility functions that may be useful for others
• New 'permute' argument in chunk-apply functions

BUG FIXES

• Improved behavior for the general 'matter()' constructor
• Use private RNG for 'uuid()' to avoid collisions from 'set.seed()' and to avoid changing '.Random.seed'
• Fix 'is_gridded()' and 'to_raster()' behavior for non-integer coordinates and character rasters

matter 2.7.4

NEW FEATURES

• Add 'SnowfastParam' class to support PSOCK clusters for faster startups and faster serialization
• Add getOption("matter.default.chunksize") for setting the approximate size of chunks in bytes
• Add getOption("matter.default.serialize") for managing serialization behavior in chunk-apply functions
• New 'simple_logger' class for writing log files
• New 'options(matter.temp.dir=tempdir())' replaces the previous 'options(matter.dump.dir=tempdir())'
• New 'options(matter.temp.gc=TRUE)' controls whether temporary matter files are automatically removed when all R objects referencing them are garbage collected
• Temporary matter files are now garbage collected!

SIGNIFICANT USER-VISIBLE CHANGES

• New 'chunkopts' argument replaces 'nchunks' and includes the new chunk options ("chunksize" and "serialize")

BUG FIXES

• Fix leaky closures in internal chunk loop functions causing slow parallel performance on large datasets
• Fix internal bug in 'eval_exprs()' where 'recursive' attribute was always set to 'TRUE'

matter 2.7.3

NEW FEATURES

• Add new functions 'knnmax()' and 'knnmin()'
• Add new functions 'filtn_ma()', 'filtn_conv()', etc.
• Add support for N-dimensional signals to 'estnoise_*()' functions
• Add 'findpeaks_knn()' for KNN-based peak detection

SIGNIFICANT USER-VISIBLE CHANGES

• Add signal processing vignette
• Faster 'knnsearch()' when the data and query are the same
• Ties in 'knnsearch()' are now broken based on data order
• Updated 'estnoise_sd()', 'estnoise_mad()', and 'estnoise_quant()'

BUG FIXES

• Improvements to 'filt1_adapt()' and 'filt2_adapt()'

matter 2.7.2

NEW FEATURES

• Support 2D signals in 'plot_signal()' (e.g., for ion mobility)
• New argument 'alphapow' in 'plot_signal()' and 'plot_image()'

SIGNIFICANT USER-VISIBLE CHANGES

• Plot method for 'vizi_bars' now sums duplicate bar counts

BUG FIXES

• Fix chunk-apply functions for BPPARAM=NULL
• Fix 'chunkMapply()' named list errors
• Fix 'is_gridded()' so infinite 'estres()' result gives FALSE
• Give more useful error messages for 'dpal()' and 'cpal()'
• Fix alpha channel in 'plot_image()'

matter 2.7.1

NEW FEATURES

• Support subsetting of deferred operations
• Add 'chunked' classes for chunked objects
• Add 's_stat()' for grouped streaming statistics
• For 'plot_image()', set 'useRaster' to plot 3d surface/volume vs voxels
• Add 'enhance_adj()' function for adjusting extreme values

SIGNIFICANT USER-VISIBLE CHANGES

• Performance improvements for chunk-apply functions
• Support chunk-apply functions on remote cluster workers
• Reduced memory usage for chunk-apply functions on SNOW clusters
• Image smoothing 'filt2_*()' functions support multiple channels
• Contrast enhancement 'enhance_*()' functions support multiple channels
• Remove 'biglm' from imports

BUG FIXES

• Fix subplot axis scaling and aspect ratio for 'plotly' graphics
• Fix passing colors to line charts for 'plotly' graphics

matter 2.6.3

BUG FIXES

• Fix 'filt2_gauss()' behavior with NAs

matter 2.6.2

BUG FIXES

• Fix 'sgmixn()' on 'matter_mat' and 'sparse_mat' input

matter 2.6.1

BUG FIXES

• Fix 'matter_list' error on zero-length elements

matter 2.5.22

BUG FIXES

• Fix y-flipped raster images on non-macOS platforms

matter 2.5.21

NEW FEATURES

• Add new vizi mark 'image' for plotting pre-rastered images
• Add arguments 'rasterImages' and 'rasterParams' to 'plot_image()'

matter 2.5.20

SIGNIFICANT USER-VISIBLE CHANGES

• Update 'trans2d()' and 'warp2_trans()' to support arrays

matter 2.5.19

NEW FEATURES

• Add new vizi mark 'rules' for reference lines

matter 2.5.18

NEW FEATURES

• Add support for optional 'plotly' graphics output
• Add 'vizi_engine()' for setting the plotting engine
• Add support for 3D images to 'plot_image()'

SIGNIFICANT USER-VISIBLE CHANGES

• Export 'parse_formula()' for developer use

BUG FIXES

• Fix 'plotly' error from non-gridded voxels

matter 2.5.17

SIGNIFICANT USER-VISIBLE CHANGES

• Allow passing subplots to 'as_facets()' via '...'

BUG FIXES

• Fix 'rowsweep()'/'colsweep()' behavior with NA groups
• Fix 'cv_do()' and 'mi_learn()' behavior with NA labels

matter 2.5.16

SIGNIFICANT USER-VISIBLE CHANGES

• Add PLS 1-component regressions to 'opls' output
• Add 'fitted()' method for 'opls'
• Update 'predict()' method for 'opls'
• Add multiple instance learning support to 'cv_do()'

BUG FIXES

• Fix 'opls_nipals()' usage with 'mi_learn()'
• Fix error setting bags to negative class in 'mi_learn()'

matter 2.5.15

BUG FIXES

• Use names "MacroRecall"/"MacroPrecision" in 'cv_do()'

matter 2.5.14

NEW FEATURES

• Add function 'rocscore()' for calculating ROC AUC

BUG FIXES

• Fix bug in 'mergepeaks()' merging too aggressively
• Fix NAs in probability in 'nscentroids()'
• Fix formatting for 'size_bytes()' when vmem is 0
• Fix bug in 'cv_do()' not processing test set

matter 2.5.13

BUG FIXES

• Fix error in 'estnoise_filt()' when all peaks are noise

matter 2.5.12

SIGNIFICANT USER-VISIBLE CHANGES

• Add 'free' argument to 'plot_signal()' and 'plot_image()'
• Add 'style' argument to 'set_par()'
• Return 'fitted.values' from each fold in 'cv_do()'
• Add 'pos' argument to 'mi_learn()' to specify positive class
• Export 'chunkify()' and 'chunk_writer()' utilities
• Export 'size_bytes()' constructor

BUG FIXES

• Improve error messages for invalid plotting options
• Rescale 'alpha' channel for images when 'enhance=TRUE'
• Return 'probability' component for 'sgmixn()'
• Prevent mark 'boxplot' from plotting extra axes
• Fix 'mi_learn()' failing for missing values in response
• Fix 'plot_image()' failing for constant opacity if 'scale=TRUE'

matter 2.5.11

NEW FEATURES

• Add RNG utility functions 'RNGStreams()', 'getRNGStream()', and 'setRNGStream()'
• Add 'seeds' argument for parallel-safe RNG for 'chunk_lapply()', etc.
• Add 'type' argument to 'drle' constructor to allow pure-RLE or sequential encoding
• Add function 'sgmixn()' for fitting multiple spatial Gaussian mixture models in parallel
• Add vizi marks 'intervals' and 'boxplot'
• Add 'predscore()' for scoring predictions
• Add 'cv_do()' for performing cross-validation

SIGNIFICANT USER-VISIBLE CHANGES

• Change 'fitted()' argument 'type' to use "response" (instead of "probability") for 'nscentroids'
• Print methods for models (e.g., 'pls', etc.) now truncate output to 'getOption("matter.show.head.n")'
• Add 'jitter' transformation for vizi mark 'points', etc.
• Export 'avg()' utility function

BUG FIXES

• Fix bug in 'nscentroids()' predictions
• Fix error in 'nscentroids()' for one-class models
• Fix error in 'sgmix()' caused by singleton classes
• Pass 'weights' argument to 'distfun()' in 'fastmap()' and 'nscentroids()'
• Fix error in 'plot_image()' for NA-only images

matter 2.5.10

NEW FEATURES

• Add function 'peakheights()'
• Add nearest shrunken centroids ('nscentroids()')
• Add spatial Gaussian mixture model ('sgmix()')
• Add colocalization coefficients ('coscore()')
• Add multiple instance learning ('mi_learn()')

SIGNIFICANT USER-VISIBLE CHANGES

• Rename 'estres()' parameter 'tol.ref' to 'ref'
• Export 'array_ind()' and 'linear_ind()'
• Add 'nchunks' argument to 'prcomp()' and 'pls()'
• Vectorize 'predict.pls()' over 'k' argument
• Set corresponding attributes to NULL when center/scale are FALSE in 'rowscale()' and 'colscale()'

BUG FIXES

• Fix 'tolerance()' returning invalid value for 'tolerance()<-'
• Fix error in 'peakareas()' when peak boundaries are NULL
• Remove duplicate peaks in 'findpeaks_cwt()'
• Fix potential infinite loop in 'binpeaks()'
• Fix potential incorrect vector lengths in 'simspec()'

matter 2.5.9

BUG FIXES

• When writing to a file in chunk apply functions, preserve order in file even if chunks are processed out-of-order

matter 2.5.8

NEW FEATURES

• Added scaling functions 'rescale_rms()', 'rescale_sum()', 'rescale_ref()', 'rescale_range()', and 'rescale_iqr()'

matter 2.5.7

SIGNIFICANT USER-VISIBLE CHANGES

• Moved 'keys' and 'keys<-' generics to Cardinal
• Removed 'chunksize' and 'chunksize<-' generics

BUG FIXES

• Fix error with NAs in 'add_alpha()'
• Fix flipped x/y coordinates in 'inpoly()'

matter 2.5.6

BUG FIXES

• Fix error in apply functions when 'BPPARAM=NULL'
• Remove zero-height peaks in 'plot_signal()'

matter 2.5.5

NEW FEATURES

• Added 'plot_signal()' and 'plot_image()'
• Added new mark 'vizi_text'

SIGNIFICANT USER-VISIBLE CHANGES

• Add 'vizi' style 'classic' for transparent background
• Implement 'vm_used()' for 'sparse_arr' objects
• Implement 'combine()' for 'vizi_plot' objects
• Rename 'plot_facets()' to 'as_facets()'

BUG FIXES

• Fixed RStudio plotting bugs due to 'par(bg="transparent")'
• Fixed 'matter_list' subsetting not subsetting type
• Fixed 'vizi_pixels' issues with y-axis
• Fixed plotting functions not passing along parameters
• Handle mixed int/double types in 'ltob()' and 'lttb()'
• Implement 'vm_used()' for 'sparse_arr'

matter 2.5.4

BUG FIXES

• Improved 'mergepeaks()' efficiency

matter 2.5.3

BUG FIXES

• Fixed stack overflow in 'sparse_mat' subsetting
• Fixed 'mergepeaks()' failing for missing peaks

matter 2.5.2

SIGNIFICANT USER-VISIBLE CHANGES

• For 'estres()', allow 'tol=NA' or 'tol=Inf'

matter 2.5.1

BUG FIXES

• Fixed bug in 'simspec()' not respecting 'units'
• Enabled calling 'vizi()' with no arguments

matter 2.3.18

BUG FIXES

• Fixed bug in 'kdtree()' stack size allocation
• Cleaned up a few C++ compiler warnings

matter 2.3.17

SIGNIFICANT USER-VISIBLE CHANGES

• Parameter 'filename' in 'struct()' is now 'path'

matter 2.3.16

SIGNIFICANT USER-VISIBLE CHANGES

• Added 'checksum' method for character vectors (i.e., files)

matter 2.3.15

BUG FIXES

• Fixed erroneous warning in 'matter_list' or 'matter_arr' constructors with existing files but no 'data' argument

matter 2.3.14

SIGNIFICANT USER-VISIBLE CHANGES

• Added 'append' argument to matter constructors
• Simplified 'sparse_arr' implementation by removing shared index representation (which was not really used)
• When writing to an output file, 'chunkApply()' and friends now write entire chunks instead of each element
• Added coercions from 'matter_list' to 'matter_arr', etc.

BUG FIXES

• No longer truncate existing files when creating a matter object if nonexistent files are also included in the path

matter 2.3.13

BUG FIXES

• Fixed translation from aliased C types to R types

matter 2.3.12

NEW FEATURES

• Added 'image' method for formulas

matter 2.3.11

SIGNIFICANT USER-VISIBLE CHANGES

• Changed specification of 'atoms' data types
• Use 'int16', int32', 'uint32', etc. for integer types
• Use 'float32' and 'float64' for floating point typess
• Added aliases so 'short', 'int', 'double', etc., still work

matter 2.3.10

NEW FEATURES

• Added 'rowMaj' S4 generic and methods

SIGNIFICANT USER-VISIBLE CHANGES

• Remove defunct S4 generics

BUG FIXES

• Fixed bug in 'approx1' for "max" and "min" interpolation

matter 2.3.9

NEW FEATURES

• Added 'vizi_pixels' for image plotting
• Added 'vizi_voxels' for 3D image plotting

SIGNIFICANT USER-VISIBLE CHANGES

• In estnoise_*() functions, renamed argument 'width' to 'n'

matter 2.3.8

NEW FEATURES

• Added PLS (partial least squares)
• Added 'pls_nipals' (NIPALS)
• Added 'pls_simpls' (SIMPLS)
• Added 'pls_kernel' (kernel algorithms #1 and #2)
• Added OPLS (orthogonal PLS)
• Added 'opls_nipals' (NIPALS)
• Added FastMap projection with 'fastmap()'
• Added 'rowDists()' and 'colDists()'
• Added 'convolve_at()'

SIGNIFICANT USER-VISIBLE CHANGES

• Added regularization to NMF functions for stability

matter 2.3.7

NEW FEATURES

• Added NMF (nonnegative matrix factorization)
• Added 'nnmf_mult()' (multiplicative updates)
• Added 'nnmf_als()' (alternating least squares)
• Exported 'prcomp_lanczos()'

SIGNIFICANT USER-VISIBLE CHANGES

• Accessing matter objects now checks for user interrupts

matter 2.3.6

NEW FEATURES

• Added 'rowdist()' and 'coldist()'
• Added 'rowdist_at()' and 'coldist_at()'
• Added parallelization support for matrix multiplication

matter 2.3.5

NEW FEATURES

• Added 'enhance_hist()' for histogram equalization
• Added 'enhance_adapt()' for CLAHE

BUG FIXES

• Fixed handling of missing values in smoothing filters

matter 2.3.4

NEW FEATURES

• Added CWT based peak detection with 'findpeaks_cwt()', 'findridges()' and 'cwt()'
• Added 'estnoise_quant()'
• Added 'filt1_conv()'
• Added 'filt2_conv()'

matter 2.3.3

NEW FEATURES

• Added 'trans2d()' for affine transformations
• Added 'warp2_trans()' for transformation-based image registration using 'optim()'

matter 2.3.2

NEW FEATURES

• More new signal processing features!
• Added 'approx2()' for 2D signal resampling and interpolation of scattered data
• Added 2D filtering including: 'filt2_ma()', 'filt2_gauss()', 'filt2_bi()', 'filt2_adapt()', 'filt2_guide'
• Added nonlinear diffusion 'filt1_diff()' and 'filt2_diff()'
• Added the traditional Savitzky-Golay filter 'filt1_sg()'
• Added 'kdsearch()' and 'kdtree()' for K-dimensional searches

SIGNIFICANT USER-VISIBLE CHANGES

• Updated package DESCRIPTION

matter 2.3.1

NEW FEATURES

• Lots of new signal processing features!
• Added 'approx1()' for signal resampling and interpolation
• Added 1D filtering including: 'filt1_ma()', 'filt1_gauss()', 'filt1_bi()', 'filt1_adapt()', 'filt1_guide()', 'filt1_pag()'
• Added 1D warping and alignment including: 'warp1_loc()', 'warp1_dtc()', 'warp1_cow()'
• Added continuum estimation including: 'estbase_loc()', 'estbase_hull()', 'estbase_snip()', 'estbase_med()'
• Added local noise estimation including: 'estnoise_sd()', 'estnoise_mad()', 'estnoise_diff()', 'estnoise_filt()'
• Added peak processing including: 'findpeaks()', 'peakwidths()', 'peakareas()', 'binpeaks()', 'mergepeaks()'
• Added 'downsample()' for signal and time series visualization
• Added 'vizi_plot' visualization methods

BUG FIXES

• Fix missing cstdint include for gcc 13.1.1 compatibility

matter 2.1.1

BUG FIXES

• Fixed NAMESPACE issue by importing 'Matrix::rowSums()' etc.

matter 2.0.1 (2022-11-16)

SIGNIFICANT USER-VISIBLE CHANGES

• Updated "User guide" vignette

BUG FIXES

• Fixed combining for matter vectors and lists
• Fixed slow 'matter_mat()' constructor for large # of atoms
• Fixed bug with BPPARAM not passed in 'rowStats()'/'colStats()'

matter 1.99.2 (2022-10-31)

BUG FIXES

• Fix 'chunkApply()' behavior for when nchunks == 1
• Fix 'atoms' failure when 'source' is a compressed 'drle'
• Properly ignore missing groups in 'rowsweep()' and 'colsweep()'

matter 1.99.1 (2022-10-30)

BUG FIXES

• Minor bug fixes to 'matter_list' and 'struct()' behavior
• Minor bug fix to endomorphic subsetting of sparse matrices
• Minor bug fix to 'binvec()' behavior at vector endpoints

matter 1.99.0 (2022-10-23)

NEW FEATURES

• Complete re-implementation of sparse array code in C++
• Complete re-implementation of matter object C++ backends
• New 'sparse_arr' class with 'sparse_mat' and 'sparse_vec'
• New 'matter_arr' class with 'matter_mat' and 'matter_vec'
• New 'asearch()' function for approximate search with interpolation
• New 'chunkApply()' function replacing 'chunk_apply()', etc.
• New 'colsweep()' and 'rowsweep()' functions w/ group parameter
• New 'colscale()' and 'rowscale()' functions w/ group parameter
• New 'drle_fct' class for delta-run-length encoded factors
• New interpolation options for 'sparse_arr' objects
• Simplified deferred arithmetic operations interface

SIGNIFICANT USER-VISIBLE CHANGES

• New simplified constructors for most matter objects
• Updated S4 internals for most matter objects
• Updated S4 internals for sparse array objects
• Deprecated some rarely-used functions and classes
• Deprecated some S4 generic conflicts with core BioC packages

BUG FIXES

• Fix to 'stream_var' and 'stream_sd' behavior when n=0 or n=1

matter 1.19.1 (2021-10-20)

BUG FIXES

• Fix error in 'cbind()' on two 'matter_vec' objects

matter 1.17.1 (2020-11-27)

BUG FIXES

• Fix 'apply()' signatures for R 4.1

matter 1.15.1

BUG FIXES

• Remove '...' from 'which()'

matter 1.13.10

NEW FEATURES

• Added 's_nnzero()' stream-stat for non-zero entries
• Added 'push()' and 'pull()' generics for future use

matter 1.13.9

BUG FIXES

• Minor updates to 'chunk_apply()' internals

matter 1.13.8

BUG FIXES

• Fixed bugs due to logical condition length > 1

matter 1.13.7

BUG FIXES

• Fixed to internal function 'linearInd()'

matter 1.13.6

BUG FIXES

• Fixed bug caused by change in R 4.0 so that class of matrix is now c("matrix", "array")

matter 1.13.5

NEW FEATURES

• Support for 'view="chunk"' when specifying 'pattern'

matter 1.13.4

NEW FEATURES

• Added 'pattern' argument to 'chunk_mapply()'

matter 1.13.3

NEW FEATURES

• Implemented 'outfile' argument for writing results to a file for 'chunk_apply()' and 'chunk_mapply()'

matter 1.13.2

NEW FEATURES

• Added 'pattern' argument to 'chunk_apply()'

matter 1.13.1 (2019-10-30)

NEW FEATURES

• Added 'chunk_mapply()'

matter 1.11.7 (2019-10-25)

BUG FIXES

• Fixed errors in 'rowStats()' and 'colStats()' when calculating group statistics over minor dimensions

matter 1.11.6 (2019-10-23)

NEW FEATURES

• Added 'locmax()' function for finding local maxima
• Added 'binvec()' function for binning vectors

matter 1.11.5 (2019-10-13)

NEW FEATURES

• Exposed 'chunk_apply()' for applying functions over parallelized chunks of vectors and matrices

matter 1.11.4 (2019-10-13)

NEW FEATURES

• Added 'rowStats()' and 'colStats()' methods for chunk-applying grouped summary statistics over the rows and columns of matrices

SIGNIFICANT USER-VISIBLE CHANGES

• Family of 'apply()' and 'lapply()' methods now attempt to respect the object's 'chunksize()'
• Added getOption('matter.default.chunksize') for setting default chunksize for matter objects

matter 1.11.3

SIGNIFICANT USER-VISIBLE CHANGES

• Faster subsetting of matter-backed ALTREP objects
• Updated 'bsearch()' to return the closest match when tol > 0
• Added getOption('matter.dump.dir') to control where temporary files are stored for matter objects

matter 1.11.2

NEW FEATURES

• Coercing to native R types now returns an ALTREP representation for most 'matter' objects
• Use 'as.altrep()' method to coerce an existing 'matter' object to ALTREP or to coerce native R types to 'matter'-backed out-of-memory ALTREP objects
• ALTREP coercion can be controlled by new options: getOption('matter.coerce.altrep') getOption('matter.coerce.altrep.list') getOption('matter.wrap.altrep')
• See ?'matter-options' for details

SIGNIFICANT USER-VISIBLE CHANGES

• The 'show()' method for 'matter' objects now prints a preview of the data head
• Printing of data can be controlled with new options getOption('matter.show.head') and getOption('matter.show.head.n')

matter 1.11.1

NEW FEATURES

• Added 'stream_stat' class for streaming statistics with functions s_mean(), s_var(), s_sd(), etc.

matter 1.9.11

BUG FIXES

• Fixed documentation link to 'base::lapply'

matter 1.9.10 (2018-12-22)

SIGNIFICANT USER-VISIBLE CHANGES

• Vectors can be used as 'virtual_mat' rows/columns now

matter 1.9.9 (2018-12-22)

SIGNIFICANT USER-VISIBLE CHANGES

• Constructor for 'drle' now allows forcing delta = 0
• Updated display descriptions in 'show' methods
• Check for NULL dimensions when subsetting 'virtual_mat' and 'sparse_mat' classes

matter 1.9.8 (2018-12-18)

BUG FIXES

• Check for NULL dimensions in 'matter_arr' subsetting

matter 1.9.7 (2018-12-18)

BUG FIXES

• Fixed bug in atoms() where 'index_offset' and 'index_extent' would overflow when creating long 'matter_vec' objects

matter 1.9.6 (2018-12-14)

SIGNIFICANT USER-VISIBLE CHANGES

• Subsetting matter matrices, arrays, and data frames now checks for the correct number of dimensions
• Linear indexing support for 'matter_mat' and 'matter_arr'

BUG FIXES

• Class checks now use R_check_slot_etc() in C so subclasses of 'matter' objects should inherit correct behavior
• Fixed other indexing issues in subsetting + assignment

matter 1.9.5 (2018-12-14)

SIGNIFICANT USER-VISIBLE CHANGES

• Updated documentation with 'matter-datatypes', size of data types are now explicitly defined
• Changed 'filemode' slot to be a factor with levels c('r', 'w', 'rw') instead of a string
• Data access now uses C++ fstream instead of C

BUG FIXES

• On Windows, accessing data elements in large files with byte offets larger than MAX_INT now works
• On Windows, data modes of 'long' and 'ulong' are now correctly 64-bit integer types
• Assignment of negative values now checked for 'ulong'

matter 1.9.4

NEW FEATURES

• Added %% and %/% delayed operations from Arith group
• Added & and | delayed operations from Logic group
• All Ops group generics now supported as delayed operations
• Added 'lapply' and 'sapply' functions for 'matter_list'
• Support for BiocParallel in 'apply', 'lapply', and 'sapply'

BUG FIXES

• Fix floating point exception error when dividing by integer 0L in delayed division operations

matter 1.9.3

NEW FEATURES

• Added matrix multiplication for 'sparse_mat' matrices
• Added matrix multiplication for 'virtual_mat' matrices

matter 1.9.2 (2018-11-12)

BUG FIXES

• Renamed internal class 'bytes' to 'num_bytes' due to Rcpp

matter 1.9.1 (2018-11-12)

BUG FIXES

• Removed BiocGenerics 'lengths' from NAMESPACE

matter 1.8.2 (2018-12-13)

BUG FIXES

• Fixed large file offset bug on Windows (again)

matter 1.8.1 (2018-12-11)

BUG FIXES

• Fixed bug causing large files to read incorrectly on Windows

matter 1.7.6

NEW FEATURES

• Added 'apply' methods for 'sparse_mat' and 'virtual_mat'
• Added 'vm_used' function to exported internal utilities
• Can infer length of 'matter_vec' from paths when missing
• Can coerce 'matter_list' to 'matter_matc' or 'matter_matr' if all elements of the list are the same length

SIGNIFICANT USER-VISIBLE CHANGES

• Initializing a matter object with data to an existing file no longer results in a warning if 'filemode' is supplied

BUG FIXES

• Fixed bug where subsetting 'sparse_mat' objects would pull 'matter_list' key-value pairs into memory
• Subsetting 'sparse_mat' objects with out-of-bounds subscripts when 'drop=NULL' is now an error

matter 1.7.5

NEW FEATURES

• Added 'as.matrix' methods for 'sparse_mat' and 'virtual_mat'

matter 1.7.4

NEW FEATURES

• Added coercion from 'matter_mat' to 'matter_list'

matter 1.7.3

NEW FEATURES

• Added 'rep_vt' class for virtual replicated vectors

matter 1.7.2

NEW FEATURES

• Updated installation instructions for BiocManager
• Setting 'sparse_mat' keys also updated nrows/ncols

BUG FIXES

• Fixed mem() to reflect R >= 3.5 gc() function

matter 1.7.1

BUG FIXES

• Fixed file.exists() bug when length(paths) > 1

matter 1.5.4

NEW FEATURES

• Added 'struct' convenience function for on-disk C-style structs (wrapper for 'matter_list')

matter 1.5.3

NEW FEATURES

• Added remainder of Summary group, including 'range', 'min', 'max', 'prod', 'any', and 'all' methods
• Added options(matter.cast.warning=FALSE) for turning off C type coercion warnings
• Exported low-level utilities 'sizeof', 'make_datamode', 'convert_datamode', and 'widest_datamode'
• Added 'combiner' generic function and method for setting/getting the 'combiner' for 'sparse_mat'
• Added 'min' and 'max' combiner functions for 'sparse_mat' matrices with tolerance > 0
• Added 'biglm' method for 'matter_df' data frames

BUG FIXES

• Fixed bug when coercing 'matter_list' to 'matter_vec'

matter 1.5.2 (2017-11-12)

NEW FEATURES

• All 'matter' subclasses now support endomorphic subsetting via 'drop=NULL' wherever appropriate
• Setting the 'dim' slot via 'dim<-' now switches the class between 'matter_vec' and 'matter_arr'
• Added 'virtual_mat' class for virtual matrices

SIGNIFICANT USER-VISIBLE CHANGES

• Use 'drop=NULL' from now on instead of 'drop=NA' to do endomorphic subsetting of matter matrices
• Added 'matter_vt' virtual class for matter objects which may exist both on-disk and in-memory
• Added 'matter_tbl' virtual class for data tables

BUG FIXES

• Fixed read/write bug when subsetting across atoms

matter 1.5.1

NEW FEATURES

• Added 'sparse_mat' class for sparse matrics (potentially on-disk) with subclasses 'sparse_matc' for CSC matrices and 'sparse_matr' for CSR matrices
• Added 'bsearch' function for fast binary searches
• Added 'uuid' function for generating UUIDs as both 'raw' and and 'character' vectors
• Added a 'checksum' method for doing sha1 and md5 checksums of all files associated with a 'matter' object

matter 1.3.8 (2017-10-26)

NEW FEATURES

• Added 'matter_fc' class for on-disk factors

SIGNIFICANT USER-VISIBLE CHANGES

• Added character encodings to 'matter_str' on-disk strings

BUG FIXES

• Fixed bug in 'matter_df' when subsetting by columns
• Fixed bug when coercing 'data.frame's to 'matter_df'

matter 1.3.7 (2017-10-25)

BUG FIXES

• Fixed bug where default 'matter_df' chunksize was tiny
• Changed 'datamode' for 'matter_df' from 'list' to 'virtual'

matter 1.3.6 (2017-10-20)

SIGNIFICANT USER-VISIBLE CHANGES

• Lists (ragged arrays) may now be heterogenous (elements must still be vectors)
• More memory-efficient initialization of data on disk when length(data) is 1
• Files are now created by default in constructors if they do not already exist
• When coercing with 'as.matter', names and/or dimnames are now retained

matter 1.3.5 (2017-10-18)

SIGNIFICANT USER-VISIBLE CHANGES

• Improved length/dim defaults when data is given to constructors

BUG FIXES

• Trying to write to a read-only dataset now fails gracefully
• Fixed integer overflow when subsetting via subsetMatterMatrix

matter 1.3.4

BUG FIXES

• Fixed bug where NAs did not propogate correctly for integers
• It is now an error when 'matter()' cannot infer data structure

matter 1.3.3

SIGNIFICANT USER-VISIBLE CHANGES

• Added new vignettes with benchmarks and examples on real data

BUG FIXES

• Fixed bug in delayed arithmetic operations with vectors

matter 1.3.2

NEW FEATURES

• Added class 'matter_list' for homogenous lists (ragged arrays)
• Added class 'matter_str' for on-disk strings
• Added class 'matter_df' for on-disk data frames
• Added 'as.matter' for function coercing to matter objects

matter 1.3.1

NEW FEATURES

• Updated base matter classes (matter_vec, matter_mat, matter_arr) to support 'Arith' and 'Compare' group generics

matter 1.1.3

NEW FEATURES

• Added class 'drle' for delta-run-length encoding vectors
• Added '+', '-', '*', '/', '^', 'exp', 'log', 'log2', and 'log10' as possible delayed operations to on-disk atoms

SIGNIFICANT USER-VISIBLE CHANGES

• Slots of 'atoms' class now use delta-run-length encoding
• Reduced metadata size by changing 'atoms' class to use groups rather then relying on a 'list' of 'atoms'
• The 'scale' method for 'matter_mat' now matches 'scale.default' more correctly when 'center = FALSE' and 'scale = TRUE'

matter 1.1.2

NEW FEATURES

• Added support for char, uchar, ushort, uint, and ulong datamodes
• Added support for raw (Rbyte) matter objects

SIGNIFICANT USER-VISIBLE CHANGES

• S4 methods for matrix-specific summary statistics are now only defined on matter_mat and its subclasses

BUG FIXES

• Dramatically improved speed of matrix multiplication

matter 1.1.1 (2016-11-29)

NEW FEATURES

• Added 'crossprod' (t(x) %*% y) and 'tcrossprod' (x %*% t(y)) methods
• Added 'atomdata' accessor method, for which 'adata' is now an alias

BUG FIXES

• Added S3 versions of some S4 methods to fix scoping issues
• Removed Cardinal package from Suggests to avoid circular dependency
• Reduced memory consumption in bigglm-matter method

matter 0.99.11 (2016-10-11)

BUG FIXES

• Updated documentation details.

matter 0.99.10 (2016-10-11)

SIGNIFICANT USER-VISIBLE CHANGES

• Updated vignettes with installation instructions, faster build time
• Added 'adata' method for accessing 'matter' class @data slot

matter 0.99.9

NEW FEATURES

• Added delayed scaling and centering via 'scale' method
• Added 'prcomp' method for principal components analysis

SIGNIFICANT USER-VISIBLE CHANGES

• Renamed 'colSd' -> 'colSds', 'colVar' -> 'colVars', etc.
• Renamed 'filepaths' -> 'paths' and 'file_id' -> 'source_id'
• Moved 'irlba' from Suggests to Imports to support new 'prcomp' method
• Updated vignette to use new 'prcomp' method in the PCA example

BUG FIXES

• Fixed bug when combining 'matter' objects with multiple data sources

matter 0.99.8

SIGNIFICANT USER-VISIBLE CHANGES

• In S4 class 'atoms', slot 'file_id' is now type 'integer' to save space
• In S4 class 'atoms', slot 'datamode' is now type 'integer' to save space
• More comprehensive error messages in constructors for S4 classes

matter 0.99.7

BUG FIXES

• Try to fix namespace std::isnan scoping issues on Windows

matter 0.99.6

BUG FIXES

• Fixed handling of NA, NaN, Inf, and -Inf during C type coercion
• Improved handling of NA, NaN, Inf, and -Inf in summary stats
• Fixed handling of NAs in matrix multiplication for integers

matter 0.99.5

BUG FIXES

• Fixed .Call native routine registration for C++ methods
• Added "C_" prefix for C++ methods called through .Call

matter 0.99.4

SIGNIFICANT USER-VISIBLE CHANGES

• Added .Call native routine registration for C++ methods

matter 0.99.3

BUG FIXES

• Import generics from S4Vectors for 'colMeans', 'colSums', 'rowMeans', and 'rowSums'
• Cleaned up method signatures and class unions

matter 0.99.2

BUG FIXES

• Cleaned up double assignments in C++ code

matter 0.99.1

BUG FIXES

• Version bump for Bioconductor build system

matter 0.99.0

BUG FIXES

• Updated PCA example in vignette (irlba now requires 'mult' argument to be non-missing for non-C execution)
• Added irlba unit test

matter 0.6

SIGNIFICANT USER-VISIBLE CHANGES

• Updated maintainer and author email address

BUG FIXES

• Fixed bug in rowVar for matter_matc

matter 0.5

SIGNIFICANT USER-VISIBLE CHANGES

• Updated documentation with examples and added unit tests

BUG FIXES

• Fixed matrix multiplication on mixed data types (int x double)

matter 0.4

SIGNIFICANT USER-VISIBLE CHANGES

• Added support for class-preserving subsetting of 'matter' matrices with 'drop=NA' argument for Cardinal compatibility.
• Added new 'show' method for 'matter' vectors and matrices showing their size in in-memory and size on disk.

matter 0.3

NEW FEATURES

• Added C++ class 'MatterAccessor' for iterating through. a buffered version of a 'matter' vector or matrix
• Added summary statistics including 'sum', 'mean', 'var', 'sd', 'colSums', 'colMeans', 'colVar', 'colSd', 'rowSums', 'rowMeans', 'rowVar', and 'rowSd'.
• Added support for 'apply' method for 'matter' matrices
• Added support for 'bigglm' linear regression.
• Added basic matrix multiplication for 'matter' matrices with an in-memory R matrix or vector.

matter 0.2

SIGNIFICANT USER-VISIBLE CHANGES

• Overhauled backend to use C++ classes 'Matter' and 'Atoms', to maximizes use of sequential reads versus random reads.

matter 0.1

SIGNIFICANT USER-VISIBLE CHANGES

• First rough implementation of matter including the classes 'atoms' and 'matter', and subclasses 'matter_vec', 'matter_matc', 'matter_matr', with a C backend.