Choice of methods

The nullranges package has two distinct branches of functionality: matching or bootstrapping to generate null sets of genomic ranges.

In the vignette sections below we describe these two branches and give more formal definitions, where each branch has associated vignettes and man page sections (see Reference tab from the package website). To give a high level overview, we first provide a decision tree that helps to indicate the considerations when choosing between these branches of functionality.

Define features as a set of genomic locations of interest, these are minimally represented with a chromosome, start and width, and optionally strand and metadata (genomic ranges). Given a set of features, suppose that we wish to create an alternate set that represents a null feature set, sometimes also referred to as “background” or a “control set”. For example, we may see that our primary features are close to transcription start sites, but are they closer than we would expect compared to a reasonable choice of null features?

Additionally, define pool as a much larger set of genomic locations compared to the primary features, and covariates as pieces of metadata attached to all of the considered features (stored as mcols(<GRanges>), which may be continuous, integer, factor, etc.).

Our choice of methods is informed by the following:

• Are the features defined with respect to a pool, e.g. open chromatin sites bound by a protein, defined with respect to all open chromatin in a particular cell type?
• Are there important, potentially confounding covariates, e.g. does GC content, distance to particular landmarks, expression level, LD score, etc. potentially explain distribution of features in a way that should be controlled for in downstream analysis?
• Do the primary feature set and pool have different distribution of these potential confounding covariates?
• Does the local genomic context matter (local defined in genomic coordinate space)? Are we asking if primary features are near in the genome to the gene start sites or some other genomic feature set?
• Do the features tend to be distributed in genomic coordinate space such that they are clustered, e.g. open chromatin sites tend to occur near each other? Or do features that are near each other in the genome tend to have similar metadata of interest to the biological question, e.g. CpG’s near each other having similar levels of methylation, which will be used in downstream computations of average methylation near gene start site? Or is there variable feature density that must be accounted for via genome segmentation?

The following decision tree then informs what methods to choose:

In summary, while matchRanges does not control for genomic distribution and clustering of features, bootRanges does not directly control for feature covariates independent of proximity and local context.

Related work

For general considerations of generation of null feature sets or segmentation for enrichment or colocalization analysis, consider the papers of De, Pedersen, and Kechris (2014), Haiminen, Mannila, and Terzi (2007), Huen and Russell (2010), Ferkingstad, Holden, and Sandve (2015), Dozmorov (2017), Kanduri et al. (2019) (with links in references below). Other Bioconductor packages that offer randomization techniques for enrichment analysis include LOLA (Sheffield and Bock 2016) and regioneR (Gel et al. 2016). Methods implemented outside of Bioconductor include GAT (Heger et al. 2013), GSC (Bickel et al. 2010), GREAT (McLean et al. 2010), GenometriCorr (Favorov et al. 2012), ChIP-Enrich (Welch et al. 2014), and OLOGRAM (Ferré et al. 2019).

We note that our block bootstrapping approach closely follows that of GSC, while offering additional features/visualizations, and is re-implemented within R/Bioconductor with efficient vectorized code for working with GRanges objects (Lawrence 2013). In addition, regioneR and OLOGRAM also offer randomization schemes that can preserve e.g. inter-feature distances.

In the context of these related works, a design choice of nullranges is its modularity: it solely generates null or background feature sets as Bioconductor objects, such that these can be used in workflows that compute and perform inference on arbitrary statistics within the plyranges (Stuart Lee, Cook, and Lawrence 2019) framework. We have referred to such workflows as “fluent genomics” (S. Lee, Lawrence, and Love 2020), or “tidy genomics”, as these workflows build up complex operations from commonly recognized verbs (filter, mutate, join, group by, summarize, etc.) strung together with pipes (%>%).

Further description of matching and bootstrapping

Suppose we want to examine the significance of overlaps of genomic sets of features x and y. To test the significance of this overlap, we calculate the overlap expected under the null by generating a null feature set y′ (potentially many times). The null features in y′ may be characterized by:

1. Drawing from a larger pool z (y′ ⊂ z), such that y and y′ have a similar distribution over one or more covariates. This is the “matching” case. Note that the features in y′ are original features, just drawn from a different pool than y. The matchRanges method is described in Davis et al. (2023) doi: 10.1101/2022.08.05.502985.
2. Generating a new set of genomic features y′, constructing them from the original set y by selecting blocks of the genome with replacement, i.e. such that features can be sampled more than once. This is the “bootstrapping” case. Note that, in this case, y′ is an artificial feature set, although the re-sampled features can retain covariates such as score from the original feature set y. The bootRanges method is described in Mu et al. (2023) doi: 10.1101/2022.09.02.506382.

In other words

1. Matching – drawing from a pool of features but controlling for certain characteristics, or covariates
2. Bootstrapping – placing a number of artificial features in the genome but controlling for the local dependence among features (e.g. features clustering in the genome and/or having correlated metadata)

Options and features

We provide a number of vignettes to describe the different matching and bootstrapping use cases. In the matching case, we have implemented a number of options, including nearest neighbor matching or rejection sampling based matching. In the bootstrapping case, we have implemented options for bootstrapping across or within chromosomes, and bootstrapping only within states of a segmented genome. We also provide a function to segment the genome by density of features. For example, supposing that x is a subset of genes, we may want to generate y′ from y such that features are re-sampled in blocks from segments across the genome with similar gene density. In both cases, we provide a number of functions for performing quality control via visual inspection of diagnostic plots.

Consideration of excluded regions

Finally, we recommend to incorporate list of regions where artificial features should not be placed, including the ENCODE Exclusion List (Amemiya, Kundaje, and Boyle 2019). This and other excluded ranges are made available in the excluderanges Bioconductor package by Mikhail Dozmorov et al. (Ogata et al. 2023). Use of excluded ranges is demonstrated in the segmented block bootstrap vignette.