Extending the SummarizedExperiment class

Motivation

A large number of Bioconductor packages contain extensions of the standard SummarizedExperiment class from the SummarizedExperiment package. This allows developers to take advantage of the power of the SummarizedExperiment representation for synchronising data and metadata, while still accommodating specialized data structures for particular scientific applications. This document is intended to provide a developer-level “best practices” reference for the creation of these derived classes.

Deriving a simple class

Overview

To introduce various concepts, we will start off with a simple derived class that does not add any new slots. This is occasionally useful when additional constraints need to be placed on the derived class. In this example, we will assume that we want our class to minimally hold a "counts" assay that contains non-negative values1.

Defining the class and its constructor

We name our new class CountSE and define it using the setClass function from the methods package, as is conventionally done for all S4 classes. We use Roxygen’s #' tags to trigger the generation of import/export statements in the NAMESPACE of our package.

#' @export
#' @import methods
#' @importClassesFrom SummarizedExperiment SummarizedExperiment
.CountSE <- setClass("CountSE", contains="SummarizedExperiment")

We define a constructor that accepts a count matrix to create a CountSE object. We use ... to pass further arguments to the SummarizedExperiment constructor, which allows us to avoid re-specifying all its arguments.

#' @export
#' @importFrom SummarizedExperiment SummarizedExperiment
CountSE <- function(counts, ...) {
    se <- SummarizedExperiment(list(counts=counts), ...)
    .CountSE(se)
}

Defining a validity method

We define a validity method that enforces the constraints that we described earlier. This is done by defining a validity function using setValidity2 from the S4Vectors package2. Returning a string indicates that there is a problem and triggers an error in the R session.

setValidity2("CountSE", function(object) {
    msg <- NULL

    if (assayNames(object)[1] != "counts") {
        msg <- c(msg, "'counts' must be first assay")
    }

    if (min(assay(object)) < 0) {
        msg <- c(msg, "negative values in 'counts'")
    }

    if (is.null(msg)) {
        TRUE
    } else msg
})
## Class "CountSE" [in ".GlobalEnv"]
## 
## Slots:
##                                                                               
## Name:            colData            assays             NAMES   elementMetadata
## Class:         DataFrame    Assays_OR_NULL character_OR_NULL         DataFrame
##                         
## Name:           metadata
## Class:              list
## 
## Extends: 
## Class "SummarizedExperiment", directly
## Class "RectangularData", by class "SummarizedExperiment", distance 2
## Class "Vector", by class "SummarizedExperiment", distance 2
## Class "Annotated", by class "SummarizedExperiment", distance 3
## Class "vector_OR_Vector", by class "SummarizedExperiment", distance 3

The constructor yields the expected output when counts are provided:

CountSE(matrix(rpois(100, lambda=1), ncol=5))
## class: CountSE 
## dim: 20 5 
## metadata(0):
## assays(1): counts
## rownames: NULL
## rowData names(0):
## colnames: NULL
## colData names(0):

… and an (expected) error otherwise:

CountSE(matrix(rnorm(100), ncol=5))
## Error in validObject(.Object): invalid class "CountSE" object: 
##     negative values in 'counts'

Defining a getter method

A generic is a group of functions with the same name that operate on different classes. Upon calling the generic on an object, the S4 dispatch system will choose the most appropriate function to use based on the object class. This allows users and developers to write code that is agnostic to the type of input class.

Let’s say that it is of particular scientific interest to obtain the counts with a flipped sign. We observe that there are no existing generics that do this task, e.g., in BiocGenerics or S4Vectors3. Instead, we define a new generic negcounts:

#' @export
setGeneric("negcounts", function(x, ...) standardGeneric("negcounts"))
## [1] "negcounts"

We then define a specific method for our CountSE class4:

#' @export
#' @importFrom SummarizedExperiment assay
setMethod("negcounts", "CountSE", function(x, withDimnames=TRUE) {
    -assay(x, withDimnames=withDimnames)
})

If any other developers need to compute negative counts for their own classes, they can simply use the negcounts generic defined in our package.

Some comments on package organization

It is convention to put all class definitions (i.e., the setClass statement) in a file named AllClasses.R, all new generic definitions in a file named AllGenerics.R, and all method definitions in files that are alphanumerically ordered below the first two. This is because R collates files by alphanumeric order when building a package. It is critical that the collation (and definition) of the classes and generics occurs before that of the corresponding methods, otherwise errors will occur. If alphanumeric ordering is inappropriate, developers can manually specify the collation order using Collate: in the DESCRIPTION file - see Writing R Extensions for more details.

Deriving a class with custom slots

Class definition

In practice, most derived classes will need to store application-specific data structures. For the rest of this document, we will be considering the derivation of a class with custom slots to hold such structures. First, we consider 1D data structures:

  • rowVec: 1:1 mapping from each value to a row of the SummarizedExperiment.
  • colVec: 1:1 mapping from each value to a column of the SummarizedExperiment.

Any 1D structure can be used if it supports length, c, [ and [<-. For simplicity, we will use integer vectors for the *.vec slots.

We also consider some 2D data structures:

  • rowToRowMat: 1:1 mapping from each row to a row of the SummarizedExperiment.
  • colToColMat: 1:1 mapping from each column to a column of the SummarizedExperiment.
  • rowToColMat: 1:1 mapping from each row to a column of the SummarizedExperiment.
  • colToRowMat: 1:1 mapping from each column to a row of the SummarizedExperiment.

Any 2D structure can be used if it supports nrow, ncol, cbind, rbind, [ and [<-. For simplicity, we will use (numeric) matrices for the *.mat slots.

Definition of the class is achieved using setClass, using the slots= argument to specify the new custom slots5.

#' @export
#' @import methods
#' @importClassesFrom SummarizedExperiment SummarizedExperiment
.ExampleClass <- setClass("ExampleClass",
    slots= representation(
        rowVec="integer",
        colVec="integer",
        rowToRowMat="matrix",
        colToColMat="matrix",
        rowToColMat="matrix",
        colToRowMat="matrix"
    ),
    contains="SummarizedExperiment"
)

Defining the constructor

The constructor should provide some arguments for setting the new slots in the derived class definition. The default values should be set such that calling the constructor without any arguments returns a valid ExampleClass object.

#' @export
#' @importFrom SummarizedExperiment SummarizedExperiment
ExampleClass <- function(
    rowVec=integer(0), 
    colVec=integer(0),
    rowToRowMat=matrix(0,0,0),
    colToColMat=matrix(0,0,0),
    rowToColMat=matrix(0,0,0),
    colToRowMat=matrix(0,0,0),
    ...)
{
    se <- SummarizedExperiment(...)
    .ExampleClass(se, rowVec=rowVec, colVec=colVec,
        rowToRowMat=rowToRowMat, colToColMat=colToColMat, 
        rowToColMat=rowToColMat, colToRowMat=colToRowMat)
}

Creating getter methods

For 1D data structures

We define some getter generics for the custom slots containing the 1D structures.

#' @export
setGeneric("rowVec", function(x, ...) standardGeneric("rowVec"))
## [1] "rowVec"
#' @export
setGeneric("colVec", function(x, ...) standardGeneric("colVec"))
## [1] "colVec"

We then define the class-specific methods for these generics. Note the withDimnames=TRUE argument, which enforces consistency between the names of the extracted object and the original SummarizedExperiment. It is possible to turn this off for greater efficiency, e.g., for internal usage where names are not necessary.

#' @export
setMethod("rowVec", "ExampleClass", function(x, withDimnames=TRUE) {
    out <- x@rowVec
    if (withDimnames) 
        names(out) <- rownames(x)
    out
})

#' @export
setMethod("colVec", "ExampleClass", function(x, withDimnames=TRUE) {
    out <- x@colVec
    if (withDimnames) 
        names(out) <- colnames(x)
    out
})

For 2D data structures

We repeat this process for the 2D structures.

#' @export
setGeneric("rowToRowMat", function(x, ...) standardGeneric("rowToRowMat"))
## [1] "rowToRowMat"
#' @export
setGeneric("colToColMat", function(x, ...) standardGeneric("colToColMat"))
## [1] "colToColMat"
#' @export
setGeneric("rowToColMat", function(x, ...) standardGeneric("rowToColMat"))
## [1] "rowToColMat"
#' @export
setGeneric("colToRowMat", function(x, ...) standardGeneric("colToRowMat"))
## [1] "colToRowMat"

Again, we define class-specific methods for these generics.

#' @export
setMethod("rowToRowMat", "ExampleClass", function(x, withDimnames=TRUE) {
    out <- x@rowToRowMat
    if (withDimnames) 
        rownames(out) <- rownames(x)
    out
})

#' @export
setMethod("colToColMat", "ExampleClass", function(x, withDimnames=TRUE) {
    out <- x@colToColMat
    if (withDimnames) 
        colnames(out) <- colnames(x)
    out
})

#' @export
setMethod("rowToColMat", "ExampleClass", function(x, withDimnames=TRUE) {
    out <- x@rowToColMat
    if (withDimnames) 
        rownames(out) <- colnames(x)
    out
})

#' @export
setMethod("colToRowMat", "ExampleClass", function(x, withDimnames=TRUE) {
    out <- x@colToRowMat
    if (withDimnames) 
        colnames(out) <- rownames(x)
    out
})

For SummarizedExperiment slots

The getter methods defined in SummarizedExperiment can be directly used to retrieve data from slots in the base class. These should generally not require any re-defining for a derived class. However, if it is necessary, the methods should use callNextMethod internally. This will call the method for the base SummarizedExperiment class, the output of which can be modified as required.

#' @export
#' @importMethodsFrom SummarizedExperiment rowData
setMethod("rowData", "ExampleClass", function(x, ...) {
    out <- callNextMethod()
    
    # Do something extra here.
    out$extra <- runif(nrow(out))

    # Returning the rowData object.
    out
})

Defining the validity method

We use setValidity2 to define a validity function for ExampleClass. We use the previously defined getter functions to retrieve the slot values rather than using @. This is generally a good idea to keep the interface separate from the implementation6. We also set withDimnames=FALSE in our getter calls, as consistent naming is not necessary for internal functions.

#' @importFrom BiocGenerics NCOL NROW
setValidity2("ExampleClass", function(object) {
    NR <- NROW(object)
    NC <- NCOL(object)
    msg <- NULL

    # 1D
    if (length(rowVec(object, withDimnames=FALSE)) != NR) {
        msg <- c(msg, "'rowVec' should have length equal to the number of rows")
    }
    if (length(colVec(object, withDimnames=FALSE)) != NC) {
        msg <- c(
            msg, "'colVec' should have length equal to the number of columns"
        )
    }

    # 2D
    if (NROW(rowToRowMat(object, withDimnames=FALSE)) != NR) {
        msg <- c(
            msg, "'nrow(rowToRowMat)' should be equal to the number of rows"
        )
    }
    if (NCOL(colToColMat(object, withDimnames=FALSE)) != NC) {
        msg <- c(
            msg, "'ncol(colToColMat)' should be equal to the number of columns"
        )
    }
    if (NROW(rowToColMat(object, withDimnames=FALSE)) != NC) {
        msg <- c(
            msg, "'nrow(rowToColMat)' should be equal to the number of columns"
        )
    }
    if (NCOL(colToRowMat(object, withDimnames=FALSE)) != NR) {
        msg <- c(
            msg, "'ncol(colToRowMat)' should be equal to the number of rows"
        )
    }

    if (length(msg)) {
        msg
    } else TRUE
})
## Class "ExampleClass" [in ".GlobalEnv"]
## 
## Slots:
##                                                                               
## Name:             rowVec            colVec       rowToRowMat       colToColMat
## Class:           integer           integer            matrix            matrix
##                                                                               
## Name:        rowToColMat       colToRowMat           colData            assays
## Class:            matrix            matrix         DataFrame    Assays_OR_NULL
##                                                             
## Name:              NAMES   elementMetadata          metadata
## Class: character_OR_NULL         DataFrame              list
## 
## Extends: 
## Class "SummarizedExperiment", directly
## Class "RectangularData", by class "SummarizedExperiment", distance 2
## Class "Vector", by class "SummarizedExperiment", distance 2
## Class "Annotated", by class "SummarizedExperiment", distance 3
## Class "vector_OR_Vector", by class "SummarizedExperiment", distance 3

We use the NCOL and NROW methods from BiocGenerics as these support various Bioconductor objects, whereas the base methods do not.

Creating a show method

The default show method will only display information about the SummarizedExperiment slots. We can augment it to display some relevant aspects of the custom slots. This is done by calling the base show method before printing additional fields as necessary.

#' @export
#' @importMethodsFrom SummarizedExperiment show
setMethod("show", "ExampleClass", function(object) {
    callNextMethod()
    cat(
        "rowToRowMat has ", ncol(rowToRowMat(object)), " columns\n",
        "colToColMat has ", nrow(colToColMat(object)), " rows\n",
        "rowToColMat has ", ncol(rowToRowMat(object)), " columns\n",
        "colToRowMat has ", ncol(rowToRowMat(object)), " rows\n",
        sep=""
    )
})

Creating setter methods

For 1D data structures

We define some setter methods for the custom slots containing the 1D structures. Again, this usually requires the creation of new generics.

#' @export
setGeneric("rowVec<-", function(x, ..., value) standardGeneric("rowVec<-"))
## [1] "rowVec<-"
#' @export
setGeneric("colVec<-", function(x, ..., value) standardGeneric("colVec<-"))
## [1] "colVec<-"

We define the class-specific methods for these generics. Note that use of validObject to ensure that the assigned input is still valid.

#' @export
setReplaceMethod("rowVec", "ExampleClass", function(x, value) {
    x@rowVec <- value
    validObject(x)
    x
})

#' @export
setReplaceMethod("colVec", "ExampleClass", function(x, value) {
    x@colVec <- value
    validObject(x)
    x
})

For 2D data structures

We repeat this process for the 2D structures.

#' @export
setGeneric("rowToRowMat<-", function(x, ..., value)
    standardGeneric("rowToRowMat<-")
)
## [1] "rowToRowMat<-"
#' @export
setGeneric("colToColMat<-", function(x, ..., value)
    standardGeneric("colToColMat<-")
)
## [1] "colToColMat<-"
#' @export
setGeneric("rowToColMat<-", function(x, ..., value) 
    standardGeneric("rowToColMat<-")
)
## [1] "rowToColMat<-"
#' @export
setGeneric("colToRowMat<-", function(x, ..., value)
    standardGeneric("colToRowMat<-")
)
## [1] "colToRowMat<-"

Again, we define class-specific methods for these generics.

#' @export
setReplaceMethod("rowToRowMat", "ExampleClass", function(x, value) {
    x@rowToRowMat <- value
    validObject(x)
    x
})

#' @export
setReplaceMethod("colToColMat", "ExampleClass", function(x, value) {
    x@colToColMat <- value
    validObject(x)
    x
})

#' @export
setReplaceMethod("rowToColMat", "ExampleClass", function(x, value) {
    x@rowToColMat <- value
    validObject(x)
    x
})

#' @export
setReplaceMethod("colToRowMat", "ExampleClass", function(x, value) {
    x@colToRowMat <- value
    validObject(x)
    x
})

For SummarizedExperiment slots

Again, we can use the setter methods defined in SummarizedExperiment to modify slots in the base class. These should generally not require any re-defining. However, if it is necessary, the methods should use callNextMethod internally:

#' @export
#' @importMethodsFrom SummarizedExperiment "rowData<-"
setReplaceMethod("rowData", "ExampleClass", function(x, ..., value) {
    y <- callNextMethod() # returns a modified ExampleClass
    
    # Do something extra here.
    message("hi!\n")

    y
})

Other types of modifying functions

Imagine that we want to write a function that returns a modified ExampleClass, e.g., where the signs of the *.vec fields are reversed. For example, we will pretend that we want to write a normalize function, using the generic from BiocGenerics.

#' @export
#' @importFrom BiocGenerics normalize
setMethod("normalize", "ExampleClass", function(object) {
    # do something exciting, i.e., flip the signs
    new.row <- -rowVec(object, withDimnames=FALSE) 
    new.col <- -colVec(object, withDimnames=FALSE)
    BiocGenerics:::replaceSlots(object, rowVec=new.row, 
        colVec=new.col, check=FALSE)
})

We use BiocGenerics:::replaceSlots instead of the setter methods that we defined above. This is because our setters perform validity checks that are unnecessary if we know that the modification cannot alter the validity of the object. The replaceSlots function allows us to skip these validity checks (check=FALSE) for greater efficiency.

Enabling subsetting operations

Getting a subset

A key strength of the SummarizedExperiment class is that subsetting is synchronized across the various (meta)data fields. This avoids book-keeping errors and guarantees consistency throughout an interactive analysis session. We need to ensure that the values in our custom slots are also subsetted.

#' @export
setMethod("[", "ExampleClass", function(x, i, j, drop=TRUE) {
    rv <- rowVec(x, withDimnames=FALSE)
    cv <- colVec(x, withDimnames=FALSE)
    rrm <- rowToRowMat(x, withDimnames=FALSE)
    ccm <- colToColMat(x, withDimnames=FALSE)
    rcm <- rowToColMat(x, withDimnames=FALSE)
    crm <- colToRowMat(x, withDimnames=FALSE)

    if (!missing(i)) {
        if (is.character(i)) {
            fmt <- paste0("<", class(x), ">[i,] index out of bounds: %s")
            i <- SummarizedExperiment:::.SummarizedExperiment.charbound(
                i, rownames(x), fmt
            )
        }
        i <- as.vector(i)
        rv <- rv[i]
        rrm <- rrm[i,,drop=FALSE]
        crm <- crm[,i,drop=FALSE]
    }

    if (!missing(j)) {
        if (is.character(j)) {
            fmt <- paste0("<", class(x), ">[,j] index out of bounds: %s")
            j <- SummarizedExperiment:::.SummarizedExperiment.charbound(
                j, colnames(x), fmt
            )
        }
        j <- as.vector(j)
        cv <- cv[j]
        ccm <- ccm[,j,drop=FALSE]
        rcm <- rcm[j,,drop=FALSE]
    }

    out <- callNextMethod()
    BiocGenerics:::replaceSlots(out, rowVec=rv, colVec=cv,
        rowToRowMat=rrm, colToColMat=ccm, 
        rowToColMat=rcm, colToRowMat=crm, check=FALSE)
})

Note the special code for handling character indices, and the use of callNextMethod to subset the base SummarizedExperiment slots.

Assigning a subset

Subset assignment can be similarly performed, though the signature needs to be specified so that the replacement value is of the same class. This is generally necessary for sensible replacement of the custom slots.

#' @export
setReplaceMethod("[", c("ExampleClass", "ANY", "ANY", "ExampleClass"),
        function(x, i, j, ..., value) {
    rv <- rowVec(x, withDimnames=FALSE)
    cv <- colVec(x, withDimnames=FALSE)
    rrm <- rowToRowMat(x, withDimnames=FALSE)
    ccm <- colToColMat(x, withDimnames=FALSE)
    rcm <- rowToColMat(x, withDimnames=FALSE)
    crm <- colToRowMat(x, withDimnames=FALSE)

    if (!missing(i)) {
        if (is.character(i)) {
            fmt <- paste0("<", class(x), ">[i,] index out of bounds: %s")
            i <- SummarizedExperiment:::.SummarizedExperiment.charbound(
                i, rownames(x), fmt
            )
        }
        i <- as.vector(i)
        rv[i] <- rowVec(value, withDimnames=FALSE)
        rrm[i,] <- rowToRowMat(value, withDimnames=FALSE)
        crm[,i] <- colToRowMat(value, withDimnames=FALSE)
    }

    if (!missing(j)) {
        if (is.character(j)) {
            fmt <- paste0("<", class(x), ">[,j] index out of bounds: %s")
            j <- SummarizedExperiment:::.SummarizedExperiment.charbound(
                j, colnames(x), fmt
            )
        }
        j <- as.vector(j)
        cv[j] <- colVec(value, withDimnames=FALSE)
        ccm[,j] <- colToColMat(value, withDimnames=FALSE)
        rcm[j,] <- rowToColMat(value, withDimnames=FALSE)
    }

    out <- callNextMethod()
    BiocGenerics:::replaceSlots(out, rowVec=rv, colVec=cv,
        rowToRowMat=rrm, colToColMat=ccm, 
        rowToColMat=rcm, colToRowMat=crm, check=FALSE)
})

Defining combining methods

By row

We need to define a rbind method for our custom class. This is done by combining the custom per-row slots across class instances.

#' @export
setMethod("rbind", "ExampleClass", function(..., deparse.level=1) {
    args <- list(...)
    all.rv <- lapply(args, rowVec, withDimnames=FALSE)
    all.rrm <- lapply(args, rowToRowMat, withDimnames=FALSE)
    all.crm <- lapply(args, colToRowMat, withDimnames=FALSE)

    all.rv <- do.call(c, all.rv)
    all.rrm <- do.call(rbind, all.rrm)
    all.crm <- do.call(cbind, all.crm)

    # Checks for identical column state.
    ref <- args[[1]]
    ref.cv <- colVec(ref, withDimnames=FALSE)
    ref.ccm <- colToColMat(ref, withDimnames=FALSE)
    ref.rcm <- rowToColMat(ref, withDimnames=FALSE)
    for (x in args[-1]) {
        if (!identical(ref.cv, colVec(x, withDimnames=FALSE)) 
            || !identical(ref.ccm, colToColMat(x, withDimnames=FALSE))
            || !identical(ref.rcm, rowToColMat(x, withDimnames=FALSE)))
        {
            stop("per-column values are not compatible")
        }
    }
 
    old.validity <- S4Vectors:::disableValidity()
    S4Vectors:::disableValidity(TRUE)
    on.exit(S4Vectors:::disableValidity(old.validity))

    out <- callNextMethod()
    BiocGenerics:::replaceSlots(out, rowVec=all.rv,
        rowToRowMat=all.rrm, colToRowMat=all.crm, 
        check=FALSE)
})

We check the other per-column slots across all elements to ensure that they are the same. This protects the user against combining incompatible objects. However, depending on the application, this may not be necessary (or too costly) for all slots, in which case it can be limited to critical slots.

We also use the disableValidity method to avoid the validity check in the base cbind method. This is because the object is technically invalid when the base slots are combined but before it is updated with the new combined values for the custom slots. The on.exit call ensures that the original validity setting is restored upon exit of the function.

By column

We similarly define a cbind method to handle the custom slots.

#' @export
setMethod("cbind", "ExampleClass", function(..., deparse.level=1) {
    args <- list(...)
    all.cv <- lapply(args, colVec, withDimnames=FALSE)
    all.ccm <- lapply(args, colToColMat, withDimnames=FALSE)
    all.rcm <- lapply(args, rowToColMat, withDimnames=FALSE)

    all.cv <- do.call(c, all.cv)
    all.ccm <- do.call(cbind, all.ccm)
    all.rcm <- do.call(rbind, all.rcm)

    # Checks for identical column state.
    ref <- args[[1]]
    ref.rv <- rowVec(ref, withDimnames=FALSE)
    ref.rrm <- rowToRowMat(ref, withDimnames=FALSE)
    ref.crm <- colToRowMat(ref, withDimnames=FALSE)
    for (x in args[-1]) {
        if (!identical(ref.rv, rowVec(x, withDimnames=FALSE)) 
            || !identical(ref.rrm, rowToRowMat(x, withDimnames=FALSE))
            || !identical(ref.crm, colToRowMat(x, withDimnames=FALSE)))
        {
            stop("per-row values are not compatible")
        }
    }

    old.validity <- S4Vectors:::disableValidity()
    S4Vectors:::disableValidity(TRUE)
    on.exit(S4Vectors:::disableValidity(old.validity))

    out <- callNextMethod()
    BiocGenerics:::replaceSlots(out, colVec=all.cv,
        colToColMat=all.ccm, rowToColMat=all.rcm, 
        check=FALSE)
})

Defining coercion methods

Coercion from SummarizedExperiment

We define a method to coerce SummarizedExperiment objects into our new ExampleClass class.

#' @exportMethods coerce
setAs("SummarizedExperiment", "ExampleClass", function(from) {
    new("ExampleClass", from, 
        rowVec=integer(nrow(from)), 
        colVec=integer(ncol(from)),
        rowToRowMat=matrix(0,nrow(from),0),
        colToColMat=matrix(0,0,ncol(from)),
        rowToColMat=matrix(0,ncol(from),0),
        colToRowMat=matrix(0,0,nrow(from)))
})

… which works as expected:

se <- SummarizedExperiment(matrix(rpois(100, lambda=1), ncol=5))
as(se, "CountSE")
## class: CountSE 
## dim: 20 5 
## metadata(0):
## assays(1): ''
## rownames: NULL
## rowData names(0):
## colnames: NULL
## colData names(0):

This was not strictly necessary for our previous CountSE class as no new slots were added. Of course, developers can still explicitly write a conversion method perform additional work to achieve a “sensible” conversion - for example, one might take the absolute values of all entries of the first matrix to ensure that the CountSE is valid for all input SummarizedExperiment objects.

Deriving from a RangedSummarizedExperiment

Note that, if we were deriving from a RangedSummarizedExperiment (e.g., for some ExampleClassRanged), it would be necessary to define explicit conversions from both RangedSummarizedExperiment and SummarizedExperment to ExampleClassRanged. In theory, we should only have to define a conversion from RangedSummarizedExperiment to ExampleClassRanged - then, any attempt to convert from a SummarizedExperiment to ExampleClassRanged would:

  1. Use the existing SummarizedExperiment to RangedSummarizedExperiment converter defined in SummarizedExperiment, and then
  2. Use the new RangedSummarizedExperiment to ExampleClassRanged converter that we just defined.

Unfortunately, in cases involving conversion to non-direct subclasses, the S4 system automatically creates methods for any conversions that are not explicitly defined. This means that the correct “chain” of methods listed above is not used when converting from a SummarizedExperiment to a ExampleClassRanged object. The automatically generated method is used instead, which may not yield a valid object when the specifics of the conversion are ignored. We avoid this scenario by explicitly defining converters for both SummarizedExperiment and RangedSummarizedExperiment to ExampleClassRanged.

Unit testing procedures

Overview

We test our new methods using the expect_* functions from the testthat package. Each function will test an expression and will raise an error if the output is not as expected. This can be used to construct unit tests for the tests/ subdirectory of the package. Unit testing ensures that the methods behave as expected, especially after any refactoring that may be performed in the future.

For testing, we will construct an instance of ExampleClass that has 10 rows and 7 columns:

RV <- 1:10
CV <- sample(50, 7)
RRM <- matrix(runif(30), nrow=10)
CCM <- matrix(rnorm(14), ncol=7)
RCM <- matrix(runif(21), nrow=7)
CRM <- matrix(rnorm(20), ncol=10)

thing <- ExampleClass(rowVec=RV, colVec=CV,
    rowToRowMat=RRM, colToColMat=CCM,
    rowToColMat=RCM, colToRowMat=CRM,
    assays=list(counts=matrix(rnorm(70), nrow=10)),
    colData=DataFrame(whee=LETTERS[1:7]),
    rowData=DataFrame(yay=letters[1:10])
)

We will also add some row and column names, which will come in handy later.

rownames(thing) <- paste0("FEATURE_", seq_len(nrow(thing)))
colnames(thing) <- paste0("SAMPLE_", seq_len(ncol(thing)))
thing
## class: ExampleClass 
## dim: 10 7 
## metadata(0):
## assays(1): counts
## rownames(10): FEATURE_1 FEATURE_2 ... FEATURE_9 FEATURE_10
## rowData names(2): yay extra
## colnames(7): SAMPLE_1 SAMPLE_2 ... SAMPLE_6 SAMPLE_7
## colData names(1): whee
## rowToRowMat has 3 columns
## colToColMat has 2 rows
## rowToColMat has 3 columns
## colToRowMat has 3 rows

Constructor

We test that the thing object we constructed is valid:

expect_true(validObject(thing))

Another useful set of unit tests involves checking that the default constructors (internal and exported) yield valid objects:

expect_true(validObject(.ExampleClass())) # internal
expect_true(validObject(ExampleClass())) # exported

We can also verify that the validity method fails on invalid objects:

expect_error(ExampleClass(rowVec=1), "rowVec")
expect_error(ExampleClass(colVec=1), "colVec")
expect_error(ExampleClass(rowToRowMat=rbind(1)), "rowToRowMat")
expect_error(ExampleClass(colToColMat=rbind(1)), "colToColMat")
expect_error(ExampleClass(rowToColMat=rbind(1)), "rowToColMat")
expect_error(ExampleClass(colToRowMat=rbind(1)), "colToRowMat")

Finally, we check that the coercion method yields a valid object.

se <- as(thing, "SummarizedExperiment")
conv <- as(se, "ExampleClass")
expect_true(validObject(conv))

Getters

Testing the 1D getter methods:

expect_identical(names(rowVec(thing)), rownames(thing))
expect_identical(rowVec(thing, withDimnames=FALSE), RV)

expect_identical(names(colVec(thing)), colnames(thing))
expect_identical(colVec(thing, withDimnames=FALSE), CV)

Testing the 2D getter methods:

expect_identical(rowToRowMat(thing, withDimnames=FALSE), RRM)
expect_identical(rownames(rowToRowMat(thing)), rownames(thing))

expect_identical(colToColMat(thing, withDimnames=FALSE), CCM)
expect_identical(colnames(colToColMat(thing)), colnames(thing))

expect_identical(rowToColMat(thing, withDimnames=FALSE), RCM)
expect_identical(rownames(rowToColMat(thing)), colnames(thing))

expect_identical(colToRowMat(thing, withDimnames=FALSE), CRM)
expect_identical(colnames(colToRowMat(thing)), rownames(thing))

Testing the custom rowData method:

expect_true("extra" %in% colnames(rowData(thing)))

Setters

Testing the 1D setter methods:

rowVec(thing) <- 0:9
expect_equivalent(rowVec(thing), 0:9)

colVec(thing) <- 7:1
expect_equivalent(colVec(thing), 7:1)

Testing the 2D setter methods:

old <- rowToRowMat(thing)
rowToRowMat(thing) <- -old
expect_equivalent(rowToRowMat(thing), -old)

old <- colToColMat(thing)
colToColMat(thing) <- 2 * old
expect_equivalent(colToColMat(thing), 2 * old)

old <- rowToColMat(thing)
rowToColMat(thing) <- old + 1
expect_equivalent(rowToColMat(thing), old + 1)

old <- colToRowMat(thing) 
colToRowMat(thing) <- old / 10
expect_equivalent(colToRowMat(thing), old / 10)

Testing our custom rowData<- method:

expect_message(rowData(thing) <- 1, "hi")

We ensure that we can successfully trigger errors on the validity method:

expect_error(rowVec(thing) <- 0, "rowVec")
expect_error(colVec(thing) <- 0, "colVec")
expect_error(rowToRowMat(thing) <- rbind(0), "rowToRowMat")
expect_error(colToColMat(thing) <- rbind(0), "colToColMat")
expect_error(rowToColMat(thing) <- rbind(0), "rowToColMat")
expect_error(colToRowMat(thing) <- rbind(0), "colToRowMat")

Other modifying functions

We test our new normalize method:

modified <- normalize(thing)
expect_equal(rowVec(modified), -rowVec(thing))
expect_equal(colVec(modified), -colVec(thing))

Subsetting methods

Subsetting by row:

subbyrow <- thing[1:5,]
expect_identical(rowVec(subbyrow), rowVec(thing)[1:5])
expect_identical(rowToRowMat(subbyrow), rowToRowMat(thing)[1:5,])
expect_identical(colToRowMat(subbyrow), colToRowMat(thing)[,1:5])

# columns unaffected...
expect_identical(colVec(subbyrow), colVec(thing)) 
expect_identical(colToColMat(subbyrow), colToColMat(thing))
expect_identical(rowToColMat(subbyrow), rowToColMat(thing))

Subsetting by column:

subbycol <- thing[,1:2]
expect_identical(colVec(subbycol), colVec(thing)[1:2])
expect_identical(colToColMat(subbycol), colToColMat(thing)[,1:2])
expect_identical(rowToColMat(subbycol), rowToColMat(thing)[1:2,])

# rows unaffected...
expect_identical(rowVec(subbycol), rowVec(thing)) 
expect_identical(rowToRowMat(subbycol), rowToRowMat(thing))
expect_identical(colToRowMat(subbycol), colToRowMat(thing))

Checking that subsetting to create an empty object is possible:

norow <- thing[0,]
expect_true(validObject(norow))
expect_identical(nrow(norow), 0L)

nocol <- thing[,0]
expect_true(validObject(nocol))
expect_identical(ncol(nocol), 0L)

Subset assignment:

modified <- thing
modified[1:5,1:2] <- thing[5:1,2:1]

rperm <- c(5:1, 6:nrow(thing))
expect_identical(rowVec(modified), rowVec(thing)[rperm])
expect_identical(rowToRowMat(modified), rowToRowMat(thing)[rperm,])
expect_identical(colToRowMat(modified), colToRowMat(thing)[,rperm])

cperm <- c(2:1, 3:ncol(thing))
expect_identical(colVec(modified), colVec(thing)[cperm])
expect_identical(colToColMat(modified), colToColMat(thing)[,cperm])
expect_identical(rowToColMat(modified), rowToColMat(thing)[cperm,])

Checking that we obtain the same object after trivial assignment operations:

modified <- thing
modified[0,] <- thing[0,]
expect_equal(modified, thing)
modified[1,] <- thing[1,]
expect_equal(modified, thing)
modified[,0] <- thing[,0]
expect_equal(modified, thing)
modified[,1] <- thing[,1]
expect_equal(modified, thing)

We double-check that we can get an error upon invalid assignment:

expect_error(modified[1,1] <- thing[0,0], "replacement has length zero")

Combining methods

Combining by row:

combined <- rbind(thing, thing)

rtwice <- rep(seq_len(nrow(thing)), 2)
expect_identical(rowVec(combined), rowVec(thing)[rtwice])
expect_identical(rowToRowMat(combined), rowToRowMat(thing)[rtwice,])
expect_identical(colToRowMat(combined), colToRowMat(thing)[,rtwice])

# Columns are unaffected:
expect_identical(colVec(combined), colVec(thing))
expect_identical(colToColMat(combined), colToColMat(thing))
expect_identical(rowToColMat(combined), rowToColMat(thing))

And combining by column. We use test_equivalent here for simplicity, as column names are altered to preserve uniqueness.

combined <- cbind(thing, thing)

ctwice <- rep(seq_len(ncol(thing)), 2)
expect_equivalent(colVec(combined), colVec(thing)[ctwice]) 
expect_equivalent(colToColMat(combined), colToColMat(thing)[,ctwice])
expect_equivalent(rowToColMat(combined), rowToColMat(thing)[ctwice,])

# Rows are unaffected:
expect_equivalent(rowVec(combined), rowVec(thing)) 
expect_equivalent(rowToRowMat(combined), rowToRowMat(thing))
expect_equivalent(colToRowMat(combined), colToRowMat(thing))

Checking that we get the same object if we combine a single object or an empty object:

expect_equal(thing, rbind(thing))
expect_equal(thing, rbind(thing, thing[0,]))

expect_equal(thing, cbind(thing))
expect_equal(thing, cbind(thing, thing[,0]))

And checking that the compatibility errors are properly thrown:

expect_error(rbind(thing, thing[,ncol(thing):1]), "not compatible")
expect_error(cbind(thing, thing[nrow(thing):1,]), "not compatible")

Documentation

We suggest creating at least two separate documentation (i.e. *.Rd) files. The first file would document the class and the constructor:

\name{ExampleClass class}

\alias{ExampleClass-class}
\alias{ExampleClass}

\title{The ExampleClass class}
\description{An overview of the ExampleClass class and constructor.}

\usage{
ExampleClass(rowVec=integer(0), colVec=integer(0),
    # etc., etc., I won't write it all out here.
)
}

\arguments{
    \item{rowVec}{An integer vector mapping to the rows, representing
        something important.}

    \item{colVec}{An integer vector mapping to the columns, representing
        something else that's important.}

    % And so on...
}

\details{
    % Some context on why this class and its slots are necessary.
    The ExampleClass provides an example of how to derive from the
    SummarizedExperiment class.  Its slots have no scientific meaning and
    are purely for demonstration purposes.
}

The second file would document all of the individual methods:

\name{ExampleClass methods}

% New generics:
\alias{rowVec}
\alias{rowVec,ExampleClass-method}
\alias{rowVec<-}
\alias{rowVec<-,ExampleClass-method}
%% And so on...

% Already have a generic:
\alias{[,ExampleClass-method}
\alias{[,ExampleClass,ANY-method}
\alias{[,ExampleClass,ANY,ANY-method}

\alias{rbind,ExampleClass-method}
%% And so on...

\title{ExampleClass methods}
\description{Methods for the ExampleClass class.}

\usage{
\S4method{rowVec}{ExampleClass}(x, withDimnames=FALSE)

\S4method{rowVec}{ExampleClass}(x) <- value

\S4method{[}{ExampleClass}(x, i, j, drop=TRUE)

\S4method{rbind}{ExampleClass}(..., , i, j, drop=TRUE)

%% And so on...
}

\arguments{
    \item{x}{An ExampleClass object.}

    \item{withDimnames}{A logical scalar indicating whether dimension names
        from \code{x} should be returned.}

    \item{value}{
        For \code{rowVec}, an integer vector of length equal to the number of 
        rows.

        For \code{colVec}, an integer vector of length equal to the number of 
        columns.
    }

    %% And so on...
}

\section{Accessors}{
    % Add some details about accessor behaviour here.
}

\section{Subsetting}{
    % Add some details about subsetting behaviour here.
}

\section{Combining}{
    % Add some details about combining behaviour here.
}

Session information

sessionInfo()
## R version 4.4.2 (2024-10-31)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 24.04.1 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=C              
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: Etc/UTC
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats4    stats     graphics  grDevices utils     datasets  methods  
## [8] base     
## 
## other attached packages:
##  [1] testthat_3.2.1.1            SummarizedExperiment_1.37.0
##  [3] Biobase_2.67.0              GenomicRanges_1.59.1       
##  [5] GenomeInfoDb_1.43.1         IRanges_2.41.1             
##  [7] S4Vectors_0.45.2            BiocGenerics_0.53.3        
##  [9] generics_0.1.3              MatrixGenerics_1.19.0      
## [11] matrixStats_1.4.1           BiocStyle_2.35.0           
## 
## loaded via a namespace (and not attached):
##  [1] sass_0.4.9              SparseArray_1.7.2       lattice_0.22-6         
##  [4] digest_0.6.37           magrittr_2.0.3          evaluate_1.0.1         
##  [7] grid_4.4.2              pkgload_1.4.0           fastmap_1.2.0          
## [10] rprojroot_2.0.4         jsonlite_1.8.9          Matrix_1.7-1           
## [13] brio_1.1.5              BiocManager_1.30.25     httr_1.4.7             
## [16] UCSC.utils_1.3.0        jquerylib_0.1.4         abind_1.4-8            
## [19] cli_3.6.3               rlang_1.1.4             crayon_1.5.3           
## [22] XVector_0.47.0          withr_3.0.2             cachem_1.1.0           
## [25] DelayedArray_0.33.2     yaml_2.3.10             S4Arrays_1.7.1         
## [28] tools_4.4.2             GenomeInfoDbData_1.2.13 buildtools_1.0.0       
## [31] R6_2.5.1                lifecycle_1.0.4         zlibbioc_1.52.0        
## [34] waldo_0.6.1             desc_1.4.3              bslib_0.8.0            
## [37] xfun_0.49               sys_3.4.3               knitr_1.49             
## [40] htmltools_0.5.8.1       rmarkdown_2.29          maketools_1.3.1        
## [43] compiler_4.4.2

  1. For simplicity’s sake, we won’t worry about enforcing integer type, as fractional values are possible, e.g., when dealing with expected counts.↩︎

  2. This allows us to turn off the validity checks in internal functions where intermediate objects may not be valid within the scope of the function.↩︎

  3. If you have an idea for a generally applicable generic that is not yet available, please contact the Bioconductor core team.↩︎

  4. The ... in the generic function definition means that custom arguments like withDimnames= can be provided for specific methods, if necessary.↩︎

  5. It does no harm to repeat the Roxygen tags, which explicitly specifies the imports required for each class and function.↩︎

  6. This protects against changes to the slot names, and simplifies development when the storage mode differs from the conceptual meaning of the data, e.g., for efficiency purposes.↩︎