NEWS

---

gdsfmt 1.40.2

UTILITIES

• add "#define STRICT_R_HEADERS 1" to the C++ header file according to R_r86984
• update the C codes according to '_R_USE_STRICT_R_HEADERS_=true' & '_R_CXX_USE_NO_REMAP_=true'

gdsfmt 1.38.1

UTILITIES

• fix the compiler warning: -Wformat-security

gdsfmt 1.36.1

UTILITIES

• 'gdsfmt:::.reopen()' allows forking

gdsfmt 1.36.0

UTILITIES

• fix the compiler warning: sprintf is deprecated
• LZ4 updated to v1.9.4
• XZ updated to v5.2.9
• update zlib to v1.2.13

NEW FEATURES

• 'system.gds()$compiler.flag[1]' is either "64-bit" or "32-bit" indicating the number of bits of internal data pointer
• new argument 'use.abspath=TRUE' in ‘openfn.gds()' and 'createfn.gds()': the behavior before v1.35.4 is the same as ’use.abspath=TRUE'

gdsfmt 1.34.1

UTILITIES

• avoid using 'crayon::blurred()' in the display (RStudio blurs the screen output)

gdsfmt 1.34.0

UTILITIES

• update the web links
• update zlib to v1.2.12

gdsfmt 1.32.0

UTILITIES

• optimize using the utilities of the Matrix package for sparse matrices

gdsfmt 1.28.1

UTILITIES

• update according to gcc-11

gdsfmt 1.28.0

NEW FEATURES

• new function 'exist.gdsn()'
• new function 'is.sparse.gdsn()'

UTILITIES

• LZ4 updated to v1.9.3 from v1.9.2
• XZ is updated to v5.2.5 from v5.2.4
• 'apply.gdsn()': work around with factor variables if less-than-32-bit integers are stored
• a new component 'is.sparse' in 'objdesp.gdsn()'
• 'options(gds.verbose=TRUE)' to show additional information

gdsfmt 1.26.1

UTILITIES

• comply with the R devel (> v4.0.3) to work with factor variables in 'apply.gdsn()'

gdsfmt 1.24.1

UTILITIES

• 'show' option in 'print.gds.class()' for array preview

gdsfmt 1.24.0

NEW FEATURES

• new option 'allow.error' in 'openfn.gds()' for data recovery
• new option 'log.only' in 'diagnosis.gds()'
• new C functions 'GDS_Node_Unload()' and 'GDS_Node_Load()' in R_GDS.h
• new sparse array data types in GDS (SparseReal32, SparseReal64, SparseInt8, SparseInt16, SparseInt32, SparseInt64, SparseUInt8, SparseUInt16, SparseUInt32, SparseUInt64)
• the opened gds file will be closed when the object is garbage collected

UTILITIES

• zlib updated to v1.2.11 from v1.2.8
• xz updated to v5.2.4 from v5.2.3
• LZ4 updated to v1.9.2 from v1.7.5
• ‘showfile.gds()' does not return the object of ’gds.class'
• new 'nmax' and 'depth' in 'print.gdsn.class()' and 'print.gds.class()'

gdsfmt 1.22.0

NEW FEATURES

• a new function 'unload.gdsn()' to unload a GDS node from memory

UTILITIES

• add '#pragma GCC optimize("O3")' to some of C++ files when GCC is used
• add the compiler information in 'system.gds()'
• change the file name "vignettes/gdsfmt_vignette.Rmd" to "vignettes/gdsfmt.Rmd", so 'vignette("gdsfmt")' can work directly

BUG FIXES

• avoid the segfault if the data type is not registered internally
• use O_CLOEXEC (the close-on-exec flag) when open and create files to avoid potentially leaking file descriptors in forked processes

gdsfmt 1.20.0

UTILITIES

• optimize the C implementation of 'packedreal8' using a look-up table

NEW FEATURES

• new data types 'packedreal8u', 'packedreal16u', 'packedreal24u' and 'packedreal32u'

BUG FIXES

• the compression method 'LZ4_RA.max' does not compress data
• 'add.gdsn()' fails if a factor variable has no level
• ‘add.gdsn(, storage=index.gdsn())' accepts the additional parameters from 'index.gdsn()', e.g., ’offset' and 'scale' for packedreal8

gdsfmt 1.18.1

BUG FIXES

• the node name should not contain a slash '/'

gdsfmt 1.18.0

NEW FEATURES

• new options 'recursive' and 'include.dirs' in 'ls.gdsn()': the listing recurses into child nodes

UTILITIES

• replace BiocInstaller biocLite mentions with BiocManager
• 'digest.gdsn()' fails if the digest package is not installed
• SIMD optimization in 2-bit array decoding with a logical vector of selection (3x speedup when there are lots of zeros)

BUG FIXES

• bug fixed: 'put.attr.gdsn()' fails to update the existing attribute

gdsfmt 1.16.0

UTILITIES

• a new storage name 'single' in 'add.gdsn()' for single-precision floating numbers
• improve the efficiency of bit2-unpacking when there are lots of zero
• ‘system.gds()' outputs ’POPCNT' flag if available
• enable the compression modes "LZMA.ultra", "LZMA.ultra_max", "LZMA_RA.ultra" and "LZMA_RA.ultra_max"
• show more compression information in 'system.gds()'

BUG FIXES

• avoid the integer overflow when the compression rate is too small using LZMA_RA (e.g., < 0.01%)

gdsfmt 1.14.0-1.14.1

UTILITIES

• tweak error messages in 'apply.gdsn()'
• ‘cleanup.gds()' allows a file name with a prefix ’~' which will be automatically replaced by the home directory

gdsfmt 1.12.0

UTILITIES

• update liblzma to v5.2.3
• update lz4 to v1.7.5
• a new citation

gdsfmt 1.10.1

NEW FEATURES

• new data types (variable-length encoding of signed and unsigned integers)

gdsfmt 1.10.0

• the version number was bumped for the Bioconductor release version 3.4

gdsfmt 1.8.0-1.8.4

• the version number was bumped for the Bioconductor release version 3.3 UTILITIES
• define C MACRO 'COREARRAY_ATTR_PACKED' and 'COREARRAY_SIMD_ATTR_ALIGN' in CoreDEF.h
• SIMD optimization in 1-bit and 2-bit array encoding/decoding (e.g., decode, RAW output: +20% for 2-bit, +50% for 1-bit)

gdsfmt 1.7.18-1.7.22

NEW FEATURES

• LZMA compression algorithm is available in the GDS system (LZMA, LZMA_RA)
• faster implementation of variable-length string: the default string becomes string with the length stored in the file instead of null-terminated string (new GDS data types: dStr8, dStr16 and dStr32)

UTILITIES

• improve the read speed of characters (+18%)
• significantly improve random access of characters
• correctly interpret factor variable in 'digest.gdsn()' when 'digest.gdsn(..., action="Robject")', since factors are not integers

gdsfmt 1.7.0-1.7.17

NEW FEATURES

• 'digest.gdsn()' to create hash function digests (e.g., md5, sha1, sha256, sha384, sha512), requiring the package digest
• new function 'summarize.gdsn()'
• 'show()' displays the content preview
• define C MACRO 'COREARRAY_REGISTER_BIT32' and 'COREARRAY_REGISTER_BIT64' in CoreDEF.h
• new C functions 'GDS_R_Append()' and 'GDS_R_AppendEx()' in R_GDS.h
• allows efficiently concatenating compressed blocks (i.e., ZIP_RA and LZ4_RA)
• v1.7.12: add a new data type: packedreal24
• define C MACRO 'COREARRAY_SIMD_SSSE3' in CoreDEF.h
• v1.7.13: 'GDS_Array_ReadData()', 'GDS_Array_ReadDataEx()', 'GDS_Array_WriteData()' and 'GDS_Array_AppendData()' return 'void*' instead of 'void' in R_GDS.h
• v1.7.15: 'GDS_Array_ReadData()' and 'GDS_Array_ReadDataEx()' allow Start=NULL or Length=NULL
• v1.7.16: new C function 'GDS_Array_AppendStrLen()' in R_GDS.h

UTILITIES

• 'paste(..., sep="")' is replaced by 'paste0(...)' (requiring >=R_v2.15.0)
• improve random access for ZIP_RA and LZ4_RA, e.g., in the example of vignette, +12% for ZIP_RA and 1.7-fold speedup in LZ4_RA

DEPRECATED AND DEFUNCT

• bit17, ..., bit23, bit25, ..., bit31, sbit17, ..., sbit23, sbit25, ..., sbit31 are deprecated, and instead it is suggested to use data compression

BUG FIXES

• v1.7.7: fix a potential issue of uninitialized value in the first parameter passed to 'LZ4_decompress_safe_continue' (detected by valgrind)
• v1.7.14: fix an issue of 'seldim' in ‘assign.gdsn()': ’seldim' should allow NULL in a vector

gdsfmt 1.6.0-1.6.2

• the version number was bumped for the Bioconductor release version 3.2
• 'attribute.trim=FALSE' in 'print.gdsn.class()' by default NEW FEATURES
• 'diagnosis.gds()' returns detailed data block information BUG FIXES
• v1.6.2: it might be a rare bug (i.e., stop the program when getting Z_BUF_ERROR), now the GDS kernel ignores Z_BUF_ERROR in deflate() and inflate(); see http://zlib.net/zlib_how.html for further explanation

gdsfmt 1.5.7-1.5.16

NEW FEATURES

• a new argument 'seldim' in 'assign.gdsn()'
• 'append.gdsn()' allows appending data from a GDS node
• ‘add.gdsn()' allows ’storage' to be a 'gdsn.class' object
• ‘put.attr.gdsn()' allows ’val' to be a 'gdsn.class' object
• 'readex.gdsn()' allows using numeric vectors to select the subset
• the values returned from ‘read.gdsn()' and 'readex.gdsn()' can be substituted by the values defined by users: ’.value' and '.substitute' are added to 'read.gdsn()' and 'readex.gdsn()'
• new functions 'copyto.gdsn()', 'getfolder.gdsn()' and 'permdim.gdsn()'
• a new option 'replace+rename' in 'moveto.gdsn()'
• the values passed to the user-defined function in ‘apply.gdsn()' and 'clusterApply.gdsn()' can be substituted by the values defined by users: ’.value' and '.substitute' are added to these functions
• new C functions 'GDS_R_Obj_SEXP2SEXP()', 'GDS_Iter_RDataEx()', 'GDS_Iter_Position()', 'GDS_Parallel_TryLockMutex()', 'GDS_Parallel_InitCondition()', 'GDS_Parallel_DoneCondition()', 'GDS_Parallel_SignalCondition()', 'GDS_Parallel_BroadcastCondition()' and 'GDS_Parallel_WaitCondition()' in "include/R_GDS.h"
• new arguments 'attribute' and 'attribute.trim' in 'print.gds.class()' and 'print.gdsn.class()'
• 'delete.attr.gdsn()' allows multiple variable names
• gdsfmt_1.5.16: hidden flag is an internal flag along with 'R.invisible', 'objdesp.gdsn()' returns the hidden flag

DEPRECATED AND DEFUNCT

• discontinue the support of SNPRelate (<= v0.9.*)
• 'GDS_R_NodeValid()' and 'GDS_R_NodeValid_SEXP()' are removed from "include/R_GDS.h"
• 'GDS_R_SEXP2Obj()' in "include/R_GDS.h" has two arguments now

BUG FIXES

• fix an issue in 'print.gdsn.class()' if hidden nodes are included
• fix the issues #3 (https://github.com/zhengxwen/gdsfmt/issues/3) and #11 (https://github.com/zhengxwen/gdsfmt/issues/11)
• fix a portable issue on Windows when calling a condition object (dPlatform.h/dPlatform.cpp)
• fix a compiling issue on Solaris with suncc
• fix an issue of additional arguments in 'add.gdsn()', https://github.com/zhengxwen/gdsfmt/issues/12
• fix an issue of uncompressing variable-length string, https://github.com/zhengxwen/gdsfmt/issues/13

gdsfmt 1.5.0-1.5.6

NEW FEATURES

• LZ4 library is updated to r131
• a new argument 'include.hidden' is added to the functions 'cnt.gdsn()' and 'ls.gdsn()'
• define 'GDS_MAX_NUM_DIMENSION' and 'GDS_R_READ_DEFAULT_MODE' in "include/R_GDS.h"
• a new C function 'GDS_R_SEXP2FileRoot()' in "include/R_GDS.h"
• improve the terminal output via the package crayon

UTILITIES

• adjust for Xcode 6.3.1
• increase test coverage

BUG FIXES

• fix an issue in 'append.gdsn()', https://github.com/zhengxwen/gdsfmt/issues/9
• fix an issue in 'assign.gdsn()', https://github.com/zhengxwen/gdsfmt/issues/10

gdsfmt 1.4.0

• the version number was bumped for the Bioconductor release version 3.1

gdsfmt 1.3.0-1.3.11

NEW FEATURES

• a new argument 'visible' is added to the functions 'add.gdsn()', 'addfile.gdsn()' and 'addfolder.gdsn()'
• ‘objdesp.gdsn()' returns ’encoder' to indicate the compression algorithm
• add a new function 'system.gds()' showing system configuration
• support efficient random access of zlib compressed data, which are composed of independent compressed blocks (ZIP_RA)
• support LZ4 compression format (http://code.google.com/p/lz4/), based on "lz4frame API" of r128
• allow R RAW data (interpreted as 8-bit signed integer) to replace 32-bit integer with 'read.gdsn()', 'readex.gdsn()', 'apply.gdsn()', 'clusterApply.gdsn()', 'write.gdsn()', 'append.gdsn()'
• a new argument 'target.node' is added to 'apply.gdsn()', which allows appending data to a target GDS variable
• ‘apply.gdsn()', 'clusterApply.gdsn()': the argument ’as.is' allows "logical" and "raw"
• more argument checking in 'write.gdsn()'
• new components 'trait' and 'param' in the return value of 'objdesp.gdsn()'
• add new data types: packedreal8, packedreal16, packedreal32
• a new argument 'permute' in the function 'setdim.gdsn()'

SIGNIFICANT USER-VISIBLE CHANGES

• add a R Markdown vignette

BUG FIXES

• minor fixes
• fix an issue of variable-length string, https://github.com/zhengxwen/gdsfmt/issues/7
• minor fixes, 'sync.gds()' synchronizes the GDS file
• fix a stream read error bug, https://github.com/zhengxwen/gdsfmt/issues/6
• fix an issue in 'add.gdsn()', https://github.com/zhengxwen/gdsfmt/issues/8
• fix the problem of 'setdim.gdsn()' with variable-length string

gdsfmt 1.1.0 (2014-09-11)

NEW FEATURES

• add a new function 'is.element.gdsn()'
• allow closing GDS files in 'showfile.gds()'

BUG FIXES

• fully support big-endian systems
• fix memory leaks in 'cleanup.gds()'

gdsfmt 1.0.4 (2014-04-16)

NEW FEATURES

• 'apply.gdsn()' and 'clusterApply.gdsn()' support characters
• add a new function 'moveto.gds()' to relocate GDS variables
• add a new function 'diagnosis.gds()' to diagnose the GDS file

SIGNIFICANT USER-VISIBLE CHANGES

• ‘apply.gdsn()', 'clusterApply.gdsn()': make the returned value invisible if ’as.is="none"'
• more options in 'read.gdsn()' and 'readex.gdsn()'
• more Unit tests

BUG FIXES

• fix a bug in 'delete.gdsn()': allocated resource is not released in the GDS file

gdsfmt 1.0.3 (2014-03-19)

NEW FEATURES

• add two new arguments 'allow.duplicate' and 'allow.fork' to the function 'openfn.gds()'
• add a new function 'showfile.gds'
• allow reading a GDS file simultaneously between multiple forked processes (applied in 'mclapply()' etc)
• support the LinkingTo mechanism, via 'R_RegisterCCallable' and 'R_GetCCallable'

gdsfmt 1.0.2 (2014-01-24)

NEW FEATURES

• add 'size' and 'good' to the returned list from 'objdesp.gdsn()' indicating the state of GDS node
• add a new function 'cache.gdsn'

BUG FIXES

• fix the memory issues reported by valgrind

gdsfmt 1.0.1 (2014-01-07)

NEW FEATURES

• add a new argument 'replace' to the function 'addfile.gdsn()', which allows replacing the existing variable by a new one
• add a new function 'addfolder.gdsn()' allowing a virtual folder linking to other GDS files
• add 'message' to the returned list from 'objdesp.gdsn()', which allows tracking error messages or log information

SIGNIFICANT USER-VISIBLE CHANGES

• remove the argument 'deep' from the function 'cleanup.gds()' to simplify calling
• reduce the package size

BUG FIXES

• backward compatible with unknown or undefined classes in GDS system

gdsfmt 1.0.0 (2013-09-21)

NEW FEATURES

• support long vectors (>= R v3.0), allowing >4G (memory size) vectors in R
• ~20x speedup in reading characters from a GDS file, compared to the previous version
• add a new argument 'replace' to the function 'add.gdsn()', which allows replacing the existing variable by a new one
• add a new argument 'simplify' to the functions 'read.gdsn()' and 'readex.gdsn()'

SIGNIFICANT USER-VISIBLE CHANGES

• speed improvement for other primitive data types
• a warning is given, when a variable with missing characters is written to a GDS variable
• replace all '.C()' by '.Call()' internally
• reduce the package size

BUG FIXES

• improve the function 'objdesp.gdsn()'
• fix a bug in 'delete.gdsn()'

gdsfmt 0.9.13-0.9.15

BUG FIXES

• fix an issue of memory leak when a compressor or decompressor is loaded
• fix an error in the CITATION file
• compiler issue fix: Solaris 10
• use 'inherits' to check the inheritance of object install 'class() =='

gdsfmt 0.9.12 (2013-02-19)

NEW FEATURES

• support variable-length string (e.g., VStr8)

SIGNIFICANT USER-VISIBLE CHANGES

• add an argument 'path' to the function ‘index.gdsn()', which uses ’/' as a separator
• support a faster defragmentation algorithm in 'cleanup.gds()'
• 'character' in the function 'add.gdsn()' refers to variable-length string by default
• fixed-length strings are "fstring", "fstring16" and "fstring32" in the function 'add.gdsn()'
• variable-length string are 'string', 'string16' and 'string32' in the function 'add.gdsn()'
• support the 'R.invisible' attribute to hide a GDS node, until adding 'all=TRUE' to 'print.gds.class()' or 'print.gdsn.class()'
• improve the display of hierarchical structure
• the argument "storage" in the function 'add.gdsn()' is not case-sensitive now

BUG FIXES

• minor bug fix in 'readex.gdsn()' when

gdsfmt 0.9.11 (2013-01-26)

NEW FEATURES

• 'put.attr.gdsn()' allows a vector with more than one elements

SIGNIFICANT USER-VISIBLE CHANGES

• it is more efficient to store a factor variable
• 'apply.gdsn()' is re-written in C/C++
• the function 'applylt.gdsn()' is merged into 'apply.gdsn()'
• the function 'clusterApplylt.gdsn() is merged into 'clusterApply.gdsn()'
• improve 'clusterApply.gdsn()'
• S3 class name 'gdsclass' is replaced by 'gds.class'
• S3 class name 'gdsn' is replaced by 'gdsn.class'

DEPRECATED AND DEFUNCT

• deprecate 'applylt.gdsn()' and 'clusterApplylt.gdsn()'

BUG FIXES

• bug fix: add a folder using 'add.gdsn()'

gdsfmt 0.9.1-0.9.10

NEW FEATURES

• add two functions with the support of the parallel package (R 2.14.0): 'clusterApply.gdsn()', 'clusterApplylt.gdsn()'

SIGNIFICANT USER-VISIBLE CHANGES

• change 'wstring' to 'string16' in 'add.gdsn()'
• change 'dwstring' to 'string32' in 'add.gdsn()'
• add RUnit tests
• support GCC4.7 compiler

BUG FIXES

• fix warnings
• fix a bug: correct the dimension size of array data with more than two dimensions
• fix bugs: 'append.gdsn' appends data of bit9, bit10, etc, correctly
• fix a minor bug of compression stream

gdsfmt 0.9.0

• first release of gdsfmt