Exploring the data

We can get an summary of the array properties, such as its shape and datatype, or group properties, using the function zarr_overview()².

zarr_overview(zarr_example)

## Type: Array
## Path: /tmp/RtmpK7i2qY/Rinstca713b08f3d/Rarr/extdata/zarr_examples/column-first/int32.zarr
## Shape: 30 x 20 x 10
## Chunk Shape: 10 x 10 x 5
## No. of Chunks: 12 (3 x 2 x 2)
## Data Type: int32
## Endianness: little
## Compressor: blosc
## Attributes: no

You can use this to check that the location is a valid Zarr array, and that the shape and datatype of the array content are what you are expecting. For example, we can see in the output above that the data type (int32) corresponds to what we expect.

Reading the Zarr array

The summary information retrieved above is required, as to read the array with Rarr you need to know the shape and size of the array (unless you want to read the entire array). From the previous output we can see our example array has three dimensions of size 30 x 20 x 10. We can select the subset we want to extract using a list. The list must have the same length as the number of dimensions in our array, with each element of the list corresponding to the indices you want to extract in that dimension.

index <- list(1:4, 1:2, 1)

We then extract the subset using read_zarr_array():

read_zarr_array(zarr_example, index = index)

## , , 1
## 
##      [,1] [,2]
## [1,]    1    2
## [2,]    1    0
## [3,]    1    0
## [4,]    1    0

Creating an “empty” array

Despite the name, you can actually think of create_empty_zarr_array() as creating an array where all the values are the same. The Zarr specification allows for “uninitialized” chunks, which are not actually present on disk. In this case, any reading application assumes the entirety of the chunk is filled with a single value, which is found in the array metadata. This allows for very efficient creation of the new array, since only a small metadata file is actually written. However it is necessary to provide some additional details, such as the shape of the array, since there’s no R array to infer these from. Let’s look at an example:

path <- tempfile()
create_empty_zarr_array(
  zarr_array_path = path,
  dim = c(50, 20),
  chunk_dim = c(5, 10),
  data_type = "integer",
  fill_value = 7L
)

First we have to provide a location for the array to be created using the zarr_array_path argument. Then we provide the dimensions of the new array, and the shape of the chunks it should be split into. These two arguments must be compatible with one another i.e. have the same number of dimensions and no value in chunk_dim should exceed the corresponding value in dim. The data_type argument defines what type of values will be stored in the array. Finally we use the fill_value argument to provide our default value for the uninitialized chunks. The next few lines check what’s actually been created on our file system. First, we use list.files() to confirm that that only file that’s been created is the zarr.json metadata; there are no chunk files. Then we use table() to check the contents of the array, and confirm that when it’s read the resulting array in R is full of 7s, our fill value.

list.files(path, all.files = TRUE, no.. = TRUE)

## [1] "zarr.json"

table(read_zarr_array(path))

## 
##    7 
## 1000

Updating a subset of an existing array

Lets assume we want to update the first row of our array to contain the sequence of integers from 1 to 20. In the code below we create an example vector containing the new data. We then use update_zarr_array(), passing the location of the Zarr and the new values to be inserted. Finally, we provide the index argument which defines which elements in the Zarr array should be updated. It’s important that the shape and number of values in x corresponds to the total count of points in the Zarr array we want to update e.g. in this case we’re updating a single row of 20 values.

x <- 1:20
update_zarr_array(
  zarr_array_path = path,
  x = x,
  index = list(1, 1:20)
)

As before, we can take a look at what’s happened on disk and confirm the values are present in the array if we read it into R.

list.files(path, all.files = TRUE, no.. = TRUE)

## [1] "c"         "zarr.json"

read_zarr_array(path, index = list(1:2, 1:5))

##      [,1] [,2] [,3] [,4] [,5]
## [1,]    1    2    3    4    5
## [2,]    7    7    7    7    7

table(read_zarr_array(path))

## 
##   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20 
##   1   1   1   1   1   1 981   1   1   1   1   1   1   1   1   1   1   1   1   1

Here list.files() confirms that there’s now two chunk files that have been created. When we first created the Zarr we specified that the chunks should be 10 columns wide, so by modifying 20 columns we’d expect at least two chunks to be realized on disk. We use read_zarr_array() to confirm visually that the first row contains our sequence of values, whilst the second row is still all 7. We use table() to confirm that the total contents is as expected.