scOverlay: layered visualization of single-cell embeddings

Introduction

Dimensionality reduction plots such as t-SNE and UMAP are commonly used to explore single-cell datasets. Point positions represent the global structure of the data, while colour commonly represents one variable such as gene expression, cluster, cell type or a QC metric. This traditional approach is limited to a single layer representing a single variable and makes it difficult to show a variable of interest while retaining the distribution of samples, cell types or clusters as a reference.

scOverlay addresses this by creating multi-layered plots for single-cell data. It colours a background layer according to one variable (for example, cell type), then draws a second foreground layer colored with a different variable on top using an independent colour scale (for example, gene expression). Optional background dimming partially mutes the background colors so that the foreground remains easy to interpret.

The background layer therefore provides context, while the foreground highlights a variable of interest such as gene expression, a score or a cell annotation.

A key additional feature of scOverlay is that subsetting can be applied to the foreground layer. This means that we can highlight the cells from a specific sample, sample type or condition while still showing the complete embedding in the background. This is useful when comparing where different groups of cells are located in the same reduced dimension space and is really helpful in identifying composition differences in related samples or sample types.

scOverlay can also represent multiple foreground features using bivariate blend plots and expression-combination plots.

scOverlay works with SingleCellExperiment objects and requires at least one reduced dimension representation, such as "TSNE" or "UMAP", plus the variables to plot in an assay or in colData.

The package focuses on visualizing already processed single-cell data and is not intended to perform preprocessing, normalization or dimensionality reduction. It complements general single-cell visualization packages such as scater and dittoSeq, and density-based expression views such as Nebulosa, by focusing on independently colored background and foreground layers.

Installation

scOverlay can be installed with BiocManager following Bioconductor’s standard procedure which will install scOverlay and all its dependencies:

if (!requireNamespace("BiocManager", quietly = TRUE)) {
    install.packages("BiocManager")
}

BiocManager::install("scOverlay")

After installation, the package can be loaded with:

library(scOverlay)

Alternatively, the development version can be installed from github:

if (!requireNamespace("BiocManager", quietly = TRUE)) {
    install.packages("BiocManager")
}

BiocManager::install("bernatgel/scOverlay")

Quick start

We will start by loading the package and the example data included (more information in the next section).

library(scOverlay)
library(SingleCellExperiment)

data("sce_mpnst_example")

and we can view the content of the example data, a SingleCellExperiment object created from a series of 10X scRNAseq experiments, with 22K cells and 76 genes (more details below).

sce_mpnst_example
#> class: SingleCellExperiment 
#> dim: 76 22285 
#> metadata(0):
#> assays(1): logcounts
#> rownames(76): NF1 CDKN2A ... EEF1A1 TUBB
#> rowData names(2): ID Symbol
#> colnames(22285): Nerve1_TGGCTATAGACAACGA-1 Nerve1_AGGCGGATCGTACCGA-1
#>   ... MPNST3_GGCCTAATCCTAAATG-1 MPNST3_GTACTTCGTTTAAAGC-1
#> colData names(11): lib_size n_detected ... celltype celltype.main
#> reducedDimNames(3): UMAP TSNE PCA
#> mainExpName: NULL
#> altExpNames(0):

The simplest scOverlay plot shows all cells in the background and overlays a foreground variable on top. In this first example, the background shows the sample type annotation (tumor type in this example) and the foreground shows the expression of CDH19.

plotOverlay(
    sce = sce_mpnst_example,
    foreground = "CDH19",
    background = "sample_type",
    bg_dimming = 0.7,
    reduced_dim = "TSNE"    
)

By default, the examples in this vignette use the "TSNE" reduced dimension. Other reduced dimensions can be selected with the reduced_dim argument. For example, the same plot could be drawn on the UMAP coordinates by setting reduced_dim = "UMAP".

We can also select different backgrounds: a solid colour, "none" for no visible background points, a gene or a cell annotation. For example, cells can be coloured in the background by cluster, while the foreground shows the doublet score stored in colData(sce_mpnst_example)$scDblFinder.score. Visual parameters such as palettes, point sizes and titles can be adjusted independently.

plotOverlay(
    sce = sce_mpnst_example,
    reduced_dim = "UMAP",
    foreground = "scDblFinder.score",
    fg_palette = "viridis",
    fg_point_size = 0.5,
    background = "cluster",
    bg_palette = "pastel",
    bg_point_size = 4,
    bg_dimming = 0.6,
    bg_legend = FALSE,
    title = "Doublet score over clusters"
)

One of the main uses of scOverlay is to highlight only a subset of cells while keeping the complete dataset visible in the background as context. Here we plot CDH19 expression only in the cells from one sample. The background still contains all cells.

plotOverlay(
    sce = sce_mpnst_example,
    foreground = "CDH19",
    fg_point_size = 0.8,
    background = "cluster",
    reduced_dim = "TSNE",
    fg_subset_cells = quote(sample == "ANF1"),
    title = "CDH19 in ANF1",
    bg_dimming = 0.9,
    bg_legend = FALSE
)

Loading the package and example data

scOverlay includes an example dataset that we’ll use throughout this vignette, including in the previous section.

data("sce_mpnst_example")
sce_mpnst_example
#> class: SingleCellExperiment 
#> dim: 76 22285 
#> metadata(0):
#> assays(1): logcounts
#> rownames(76): NF1 CDKN2A ... EEF1A1 TUBB
#> rowData names(2): ID Symbol
#> colnames(22285): Nerve1_TGGCTATAGACAACGA-1 Nerve1_AGGCGGATCGTACCGA-1
#>   ... MPNST3_GGCCTAATCCTAAATG-1 MPNST3_GTACTTCGTTTAAAGC-1
#> colData names(11): lib_size n_detected ... celltype celltype.main
#> reducedDimNames(3): UMAP TSNE PCA
#> mainExpName: NULL
#> altExpNames(0):

The object sce_mpnst_example is a reduced SingleCellExperiment included with the package to demonstrate its plotting functionality. The underlying data were generated by the package author’s research group as part of the single-cell MPNST progression study deposited at the European Genome-phenome Archive under accession EGAS50000001747. This example contains selected genes and cells from several sample types, with generic sample identifiers. It retains selected log-expression values, cell metadata and reduced dimensions, and includes no direct patient identifiers.

The most important elements for scOverlay are the reduced dimensions, assays and cell metadata stored in colData.

reducedDimNames(sce_mpnst_example)
#> [1] "UMAP" "TSNE" "PCA"
assayNames(sce_mpnst_example)
#> [1] "logcounts"
colnames(colData(sce_mpnst_example))
#>  [1] "lib_size"          "n_detected"        "mito_percent"     
#>  [4] "sample"            "sample_type"       "sizeFactor"       
#>  [7] "scDblFinder.score" "scDblFinder.class" "cluster"          
#> [10] "celltype"          "celltype.main"

The reduced dimensions define the coordinates used for plotting. The object contains "UMAP", "TSNE" and "PCA" coordinates. This vignette uses t-SNE and UMAP to demonstrate the plotting API.

The assay names indicate where gene expression values are stored. The examples below assume that the object contains a "logcounts" assay, which is the default assay used by plotOverlay() when the foreground is a gene.

The columns in colData(sce_mpnst_example) contain cell-level annotations that can be used as background variables, foreground variables or to select subsets of cells.

Finally, we can check that some of the genes used in the examples are present in the object:

c("SOX10", "S100B", "CDH19", "MKI67") %in% rownames(sce_mpnst_example)
#> [1] TRUE TRUE TRUE TRUE

The basic overlay model

The main idea behind scOverlay is to separate the information used as context from the information we want to highlight. The background layer shows the complete embedding. The foreground layer is drawn on top and can represent a gene, a cell metadata column, a score, etc in the whole dataset or a subset of cells.

As seen in the “Quick Start” section, a first example is to plot gene expression over the sample type annotation, which indicates the biological sample class associated with each cell.

plotOverlay(
    sce = sce_mpnst_example,
    foreground = "CDH19",
    background = "sample_type",
    reduced_dim = "TSNE"    
)

However, the foreground does not have to be a gene. It can also be a column in colData(sce), such as a cluster, sample, cell type or doublet score. The background can use the same data sources or a solid colour.

The foreground can also be restricted to a specific subset of cells. This makes it possible to ask questions such as where the cells from a particular sample type are located in the dimensionality reduction embedding, while still seeing the complete structure of the data in the background.

The following sections describe these components in more detail.

Choosing the foreground and background

The foreground layer contains the variable we want to highlight while the background provides the context. They can be a gene or feature in rownames(sce) or a column in colData(sce), but also "solid" for a fixed-colour selection or "none" for no visible points.

When the foreground is a gene, expression values are taken from the assay specified by fg_assay; when the background is a gene, bg_assay is used. Both default to "logcounts". Selection can also be explicit, for example with fg_assay = "logcounts" and bg_assay = "logcounts", which is useful for objects containing several assays. Assay arguments do not affect variables read from colData.

The data layers can also show cell annotations in colData(sce), including clusters, samples, sample types and other values such as doublet scores or other QC metrics. Value type is detected automatically, but it can be specified explicitly with fg_type = "categorical" or fg_type = "continuous" (and the corresponding bg_type values for the background). This determines the colour mapping used for each layer.

The foreground can be a vector of feature or column names only when using fg_type = "bivariate" or fg_type = "combination".

Important: type detection treats numeric vectors as continuous and other vectors as categorical. When categories such as cluster labels are stored as integers, convert the column in colData to a factor or specify fg_type = "categorical" explicitly.

plotOverlay(
    sce = sce_mpnst_example,
    foreground = "scDblFinder.score",
    fg_type = "continuous",
    background = "cluster",
    bg_type = "categorical",
    reduced_dim = "TSNE"
)

In addition to representing a variable, both foreground and background can be set to "solid" or "none". The former plots all points in that layer with one colour, while the latter omits the layer. Solid layers use grey by default; fg_palette and bg_palette can change this colour.

We can alter foreground plotting order with fg_order. This can reduce visual artefacts caused by the input cell order or improve the visibility of high values in dense plots. fg_order = "random" randomizes the order, while fg_order = "ascending" draws low values first so that high values remain visible on top.

Finally, we can subset the foreground layer (and only the foreground! background is not subsettable) with fg_subset_cells; the background remains complete. The most direct selection criterion is an expression evaluated in colData(sce), such as fg_subset_cells = quote(sample_type == "MPNST"). A logical vector, a vector of cell names or a function that receives the SingleCellExperiment object and returns a logical vector are also valid.

p4 <- plotOverlay(
    sce = sce_mpnst_example,
    foreground = "scDblFinder.score",
    fg_type = "continuous",
    fg_order = "ascending",
    fg_subset_cells = quote(sample_type != "MPNST"),
    reduced_dim = "TSNE",
    background = "cluster",
    bg_point_size = 3,
    title = "scDblFinder.score in non-MPNST samples",
    bg_legend = FALSE
)
p4

Plotting multiple genes on the foreground

Some questions involve more than one foreground feature. scOverlay supports this through two explicit foreground modes: bivariate blends for two numeric variables, and expression-combination plots for small sets of numeric variables.

Bivariate foregrounds

Bivariate foregrounds blend two numeric variables into a two-dimensional colour scale. The first foreground variable defines the x direction and maps to the high_x colour corner; the second foreground variable defines the y direction and maps to the high_y colour corner. Because this is not a regular one-dimensional ggplot colour scale, bivariate plots do not use a standard ggplot legend. Use plotBivariateLegend() to create a separate two-dimensional legend and combine it manually with the plot.

p_biv <- plotOverlay(
    sce = sce_mpnst_example,
    foreground = c("SOX10", "MKI67"),
    fg_type = "bivariate",
    fg_palette = "bivariateBright",
    background = "cluster",
    bg_dimming = 0.9,
    bg_legend = FALSE,
    reduced_dim = "TSNE"
)

leg_biv <- plotBivariateLegend(
    features = c("SOX10", "MKI67"),
    palette = "bivariateBright"
)

patchwork::wrap_plots(p_biv, leg_biv, widths = c(1, 0.25))

Here, the two-dimensional colour mapping distinguishes SOX10-high cells from MKI67-high proliferating cells and highlights cells with high values for both genes.

Custom bivariate palettes use four named corner colours.

custom_biv <- c(
    low_xy = "#e6e6e6",
    high_x = "#ff4346",
    high_y = "#55b2ff",
    high_xy = "#7a1fa2"
)

plotBivariateLegend(
    features = c("SOX10", "MKI67"),
    palette = custom_biv
)

Expression-combination foregrounds

Combination foregrounds convert between two and five numeric foreground features into expressed/not-expressed calls and plot the resulting combinations as categorical values. By default, fg_threshold = 0, and expression is called with the strict rule value > fg_threshold, so exact zero is not expressed. Labels such as SOX10-/MKI67+ are categorical values, and the combinationBright palette is used by default.

plotOverlay(
    sce = sce_mpnst_example,
    foreground = c("SOX10", "MKI67"),
    fg_type = "combination",
    fg_threshold = 0,
    background = "cluster",
    bg_dimming = 0.9,
    bg_legend = FALSE,
    reduced_dim = "TSNE"
)

For larger combinations, a data-driven threshold can be useful.

plotOverlay(
    sce = sce_mpnst_example,
    foreground = c("SOX10", "MKI67", "CDH19"),
    fg_type = "combination",
    fg_threshold = "median",
    background = "solid",
    reduced_dim = "TSNE"
)

Plotting several genes

When we want to inspect several genes, we can call plotOverlay() repeatedly, but plotGeneOverlay() provides a compact convenience function. It creates one overlay plot for each requested gene and returns the plots as a named list.

gene_plots <- plotGeneOverlay(
    sce = sce_mpnst_example,
    genes = c("S100B", "CDH19", "PRRX1", "EGFR"),
    background = "solid",
    reduced_dim = "TSNE",
    fg_order = "ascending"
)

names(gene_plots)
#> [1] "S100B" "CDH19" "PRRX1" "EGFR"

Each element of the list is a regular ggplot2 object and can be displayed independently.

gene_plots$CDH19

All additional arguments are passed to plotOverlay(). For example, we can use a metadata background and keep the same foreground ordering.

And the plots can also be combined for example with patchwork.

library(patchwork)

gene_plots_bg <- plotGeneOverlay(
    sce = sce_mpnst_example,
    genes = c("SOX10", "CDH19", "PRRX1", "EGFR"),
    background = "sample_type",
    reduced_dim = "TSNE",
    fg_order = "ascending",
    bg_dimming = 0.7
)

(gene_plots_bg$SOX10 + gene_plots_bg$CDH19) /
    (gene_plots_bg$PRRX1 + gene_plots_bg$EGFR)

If some of the requested genes are not present in the object, they are skipped with a warning. This makes it possible to use the same gene list with different objects, as long as at least one of the requested genes is present.

gene_plots_subset <- plotGeneOverlay(
    sce = sce_mpnst_example,
    genes = c("SOX10", "NOT_A_GENE"),
    background = "solid",
    reduced_dim = "TSNE",
    fg_order = "ascending"
)
#> Warning: `genes` contains features not present in rownames(sce) and they will
#> be skipped: NOT_A_GENE.

names(gene_plots_subset)
#> [1] "SOX10"

Splitting plots by groups

The function plotOverlayPerGroup() creates one overlay plot for each value of a metadata column. This is useful when we want to compare the same foreground variable across sample types, clusters, conditions or other cell annotations.

In each panel, the background still shows the complete embedding but the foreground layer is restricted to the cells belonging to the corresponding group.

By default, the group order follows the factor levels if the grouping column is a factor, or the order of appearance in the object otherwise. An explicit order can be supplied with group_order.

p2 <- plotOverlayPerGroup(
    sce = sce_mpnst_example,
    group_col = "sample_type",
    group_order = c("Nerve", "PNF", "ANF", "MPNST"),
    foreground = "CDH19",
    background = "cluster",
    reduced_dim = "TSNE",
    fg_order = "ascending",
    fg_legend = FALSE,
    bg_legend = FALSE,
    bg_dimming = 0.9
)
p2

An additional foreground subset can be combined with the group-wise split. This allows us to show, for example, one plot per sample type while restricting the foreground to cells with a particular annotation.

p4 <- plotOverlayPerGroup(
    sce = sce_mpnst_example,
    group_col = "sample_type",
    group_order = c("Nerve", "PNF", "ANF", "MPNST"),
    foreground = "S100B",
    background = "cluster",
    reduced_dim = "TSNE",
    fg_subset_cells = quote(celltype.main == "Schwann cell"),
    fg_order = "ascending",
    fg_legend = FALSE,
    bg_legend = FALSE
)
p4

Shared scales and legends

When plotting the same gene or other value in multiple panels, shared scale limits make colours comparable. For univariate continuous layers, fg_limits and bg_limits can specify exact limits, such as fg_limits = c(0, 6). Alternatively, fg_limits = "shared" computes one foreground range for all panels, while bg_limits = "shared" does the same for a continuous background. Shared limits are computed after applying any global foreground subset and before splitting into groups. For bivariate foregrounds, numeric limits are not supported, but fg_limits = "shared" provides comparable x and y scaling across grouped or nested panels. Compatible legends can be collected with shared_legend = TRUE and moved with regular patchwork syntax.

p_shared <- plotOverlayPerGroup(
    sce = sce_mpnst_example,
    group_col = "sample_type",
    group_order = c("Nerve", "PNF", "ANF", "MPNST"),
    foreground = "CDH19",
    background = "cluster",
    bg_dimming = 0.9,
    reduced_dim = "TSNE",
    fg_order = "ascending",
    fg_legend = TRUE,
    bg_legend = FALSE,
    fg_limits = "shared",
    shared_legend = TRUE
)

p_shared & ggplot2::theme(legend.position = "bottom")

For categorical variables, grouped and nested plots use global categorical values to keep colour assignments consistent across panels. Named categorical palettes are recommended when comparing panels, especially when some panels lack some categories.

Nested group plots

plotOverlayPerNestedGroup() is useful when there are two related grouping variables. A common situation is to have cells grouped by sample, with samples belonging to broader sample types or experimental conditions.

In this example, group_col = "sample" defines the individual panels and outer_col = "sample_type" arranges the samples according to their sample type.

Nested layouts can represent either true hierarchies or repeated inner labels. Every real panel shows the full dataset in the background; the nested grouping restricts only the foreground cells. Nested panels are titled by default as inner - outer, which can be changed with panel_titles; use panel_titles = "none" to suppress these titles. When inner labels are unique to one outer group, a compact layout is often easier to read, and shorter rows are padded on the right with blank spacers so panel widths remain consistent. When inner labels repeat across outer groups, a grid layout preserves a consistent column meaning and uses blank spacers for missing outer/inner combinations. The default layout = "auto" uses the grid layout when any inner label appears in more than one outer group and otherwise uses the compact layout.

In the sample type/sample example, samples belong to a single sample type, so layout = "auto" resolves to the compact layout.

The row and panel order can be controlled with outer_order and group_order. Here we explicitly order the sample types.

p1 <- plotOverlayPerNestedGroup(
    sce = sce_mpnst_example,
    group_col = "sample",
    outer_col = "sample_type",
    outer_order = c("Nerve", "PNF", "ANF", "MPNST"),
    foreground = "CDH19",
    background = "cluster",
    reduced_dim = "TSNE",
    bg_dimming = 0.9,
    fg_order = "ascending",
    fg_limits = "shared",
    shared_legend = TRUE,
    bg_legend = FALSE,
    layout = "auto"
)
p1

We can also use nested plots when the inner labels repeat across outer groups. Here we create a simple high/low detected-gene group from an existing QC metric. Because both labels can appear in several sample types, layout = "auto" chooses the grid layout.

sce_mpnst_example$detected_group <- ifelse(
    sce_mpnst_example$n_detected > median(sce_mpnst_example$n_detected),
    "High detected genes",
    "Low detected genes"
)

p_grid <- plotOverlayPerNestedGroup(
    sce = sce_mpnst_example,
    outer_col = "sample_type",
    group_col = "detected_group",
    foreground = "SOX10",
    background = "celltype.main",
    layout = "auto",
    fg_limits = "shared",
    shared_legend = TRUE,
    bg_legend = FALSE,
    reduced_dim = "TSNE"
)
p_grid

Nested plots can also be combined with an additional foreground subset. In this example, one plot is created per sample, but only cells whose main cell type is annotated as Mesenchymal are shown in the foreground.

p1 <- plotOverlayPerNestedGroup(
    sce = sce_mpnst_example,
    group_col = "sample",
    outer_col = "sample_type",
    outer_order = c("Nerve", "PNF", "ANF", "MPNST"),
    foreground = "PRRX1",
    background = "cluster",
    reduced_dim = "TSNE",
    bg_dimming = 0.9,
    fg_order = "ascending",
    fg_limits = "shared",
    shared_legend = TRUE,
    bg_legend = FALSE,
    fg_subset_cells = quote(celltype.main == "Mesenchymal"),
    layout = "auto"
)
p1

Palettes

scOverlay includes categorical, continuous and bivariate palettes. Palette definitions and metadata can be listed with listPalettes().

listPalettes()
#>                            name        type  n      colours is_default
#> 1                     scOverlay categorical 23 #05d165,....       TRUE
#> 2                          base categorical 24 #3B7EA1,....      FALSE
#> 3                          dark categorical 24 #1F4E79,....      FALSE
#> 4                          soft categorical 24 #4E79A7,....      FALSE
#> 5                        bright categorical 24 #0057FF,....      FALSE
#> 6                        pastel categorical 24 #8ECAE6,....      FALSE
#> 7             combinationBright categorical 32 #D9D9D9,....      FALSE
#> 8                         magma  continuous  6 #000004,....      FALSE
#> 9                     gray_blue  continuous  5 #c9c9c9,....      FALSE
#> 10                     gray_red  continuous  5 #c9c9c9,....       TRUE
#> 11                  gray_purple  continuous  5 #c9c9c9,....      FALSE
#> 12                    blue_gold  continuous  5 #081D58,....      FALSE
#> 13                purple_orange  continuous  5 #2D004B,....      FALSE
#> 14             black_red_yellow  continuous  6 #000000,....      FALSE
#> 15     diverging_blue_white_red  continuous  7 #053061,....      FALSE
#> 16 diverging_green_white_purple  continuous  7 #00441B,....      FALSE
#> 17  diverging_teal_gray_magenta  continuous  7 #004D40,....      FALSE
#> 18                      viridis  continuous NA           NA      FALSE
#> 19                      inferno  continuous NA           NA      FALSE
#> 20                       plasma  continuous NA           NA      FALSE
#> 21                      cividis  continuous NA           NA      FALSE
#> 22                       rocket  continuous NA           NA      FALSE
#> 23                         mako  continuous NA           NA      FALSE
#> 24                        turbo  continuous NA           NA      FALSE
#> 25              bivariateBright   bivariate  4 #e6e6e6,....       TRUE
#> 26          bivariateOrangeBlue   bivariate  4 #e8e8e8,....      FALSE
#> 27           bivariatePinkGreen   bivariate  4 #e8e8e8,....      FALSE
#> 28             bivariateRedCyan   bivariate  4 #e6e6e6,....      FALSE
#> 29            bivariateGreenRed   bivariate  4 #e6e6e6,....      FALSE
#> 30          bivariateBlueYellow   bivariate  4 #e6e6e6,....      FALSE
listPalettes(type = "bivariate")
#>                  name      type n      colours is_default
#> 1     bivariateBright bivariate 4 #e6e6e6,....       TRUE
#> 2 bivariateOrangeBlue bivariate 4 #e8e8e8,....      FALSE
#> 3  bivariatePinkGreen bivariate 4 #e8e8e8,....      FALSE
#> 4    bivariateRedCyan bivariate 4 #e6e6e6,....      FALSE
#> 5   bivariateGreenRed bivariate 4 #e6e6e6,....      FALSE
#> 6 bivariateBlueYellow bivariate 4 #e6e6e6,....      FALSE

A graphical preview can be generated with plotPalettes().

plotPalettes(n = 9)

Palettes are selected by name with bg_palette or fg_palette. Continuous palettes are stored internally as anchor colours and interpolated to the number of colours needed for the plot. User-defined continuous palettes can therefore be supplied as a small vector of colours that is interpolated automatically. Supported palettes from the viridisLite and RColorBrewer packages are also available by name.

Here are a few plots with different palettes.

my_palettes <- list(
    "gray_red",
    "blue_gold",
    "viridis",
    c("cornsilk", "khaki1", "yellow2", "darkgoldenrod")
)

plots <- lapply(my_palettes, function(pal) {
    plotOverlay(
        sce = sce_mpnst_example,
        foreground = "CDH19",
        background = "solid",
        reduced_dim = "TSNE",
        fg_palette = pal,
        fg_order = "ascending",
        fg_legend = FALSE,
        title = pal
    )
})


patchwork::wrap_plots(plots, ncol = 2)

Categorical and named categorical palettes

The package also includes categorical palettes that will be used for categorical data. Categorical palettes may have names. If all plotted values match those names, colours are matched by name. If none of the plotted values match the names, colours are assigned by position. If only some values match, scOverlay raises an error to avoid accidentally remapping colours.

sample_type_palette <- c(
    Nerve = "#4E79A7",
    PNF = "#59A14F",
    ANF = "#F28E2B",
    MPNST = "#E15759"
)

plotOverlay(
    sce = sce_mpnst_example,
    foreground = "sample_type",
    background = "solid",
    reduced_dim = "TSNE",
    fg_type = "categorical",
    fg_palette = sample_type_palette
)

The built-in categorical palettes keep integer-like names ("1", "2", "3" and so on) because these are useful for cluster labels. Use unname() to force positional matching, or setNames() to define a custom mapping.

sample_type_palette <- c(
    Nerve = "#4E79A7",
    PNF = "#59A14F",
    ANF = "#F28E2B",
    MPNST = "#E15759"
)

getPalette(
    palette = sample_type_palette,
    type = "categorical",
    values = c("Nerve", "PNF", "ANF", "MPNST")
)
#>     Nerve       PNF       ANF     MPNST 
#> "#4E79A7" "#59A14F" "#F28E2B" "#E15759"

pal <- getPalette("scOverlay", type = "categorical", n = 4)
setNames(
    unname(pal),
    c("Nerve", "PNF", "ANF", "MPNST")
)
#>     Nerve       PNF       ANF     MPNST 
#> "#05d165" "#e6194b" "#ebad00" "#be6af3"

Using named palettes is recommended to avoid misleading plots when positional matching is used and some plots might be missing some groups.

Rasterizing dense point layers

Single-cell embeddings can contain thousands or millions of points. When these plots are saved as vector graphics, the resulting PDF or SVG files can become large and slow to open or edit. scOverlay can rasterize only the point layers while keeping the rest of the plot as vector graphics.

This means that axes, titles, labels, legends and other plot elements remain editable, while the dense background and/or foreground points are stored as raster images.

Rasterization is controlled independently for the background and foreground layers with bg_raster and fg_raster.

p1 <- plotOverlay(
    sce = sce_mpnst_example,
    foreground = "SOX10",
    background = "sample_type",
    reduced_dim = "TSNE",
    fg_order = "ascending",
    bg_raster = TRUE,
    bg_dimming = 0.7
)
p1

A common use case is to rasterize the background, because it usually contains all cells, and keep the foreground as vector points. This can be useful when the foreground contains a smaller number of highlighted cells.

p2 <- plotOverlay(
    sce = sce_mpnst_example,
    foreground = "CDH19",
    background = "sample_type",
    reduced_dim = "TSNE",
    fg_subset_cells = quote(sample_type == "MPNST"),
    fg_order = "ascending",
    bg_raster = TRUE,
    fg_raster = FALSE
)
p2

When both layers are dense, both can be rasterized.

p3 <- plotOverlay(
    sce = sce_mpnst_example,
    foreground = "SOX10",
    background = "sample_type",
    reduced_dim = "TSNE",
    fg_order = "ascending",
    bg_raster = TRUE,
    fg_raster = TRUE,
    raster_dpi = 300
)
p3

The resolution of the rasterized layers is controlled with raster_dpi. Higher values, such as 600 or 1200, may produce sharper point layers but also larger output files.

Rasterization is particularly useful when saving plots to PDF or SVG for publication figures, because it reduces the number of vector elements while keeping the text and layout editable.

Saving plots

The objects returned by plotOverlay() are regular ggplot2 objects, so they can be saved with the standard ggplot2::ggsave() function.

p <- plotOverlay(
    sce = sce_mpnst_example,
    foreground = "SOX10",
    background = "sample_type",
    reduced_dim = "TSNE",
    fg_order = "ascending",
    bg_raster = TRUE
)

The same plot can be saved in different formats depending on its intended use, including PNG, PDF and SVG.

ggplot2::ggsave(
    filename = "SOX10_overlay.png",
    plot = p,
    width = 5,
    height = 5,
    dpi = 300
)

ggplot2::ggsave(
    filename = "SOX10_overlay.pdf",
    plot = p,
    width = 5,
    height = 5
)

ggplot2::ggsave(
    filename = "SOX10_overlay.svg",
    plot = p,
    width = 5,
    height = 5
)

Grouped plots are saved in the same way: create the plot object first, then pass it to ggplot2::ggsave(). For grouped plots, take into account the number of panels to define the image size.

p_grouped <- plotOverlayPerGroup(
    sce = sce_mpnst_example,
    group_col = "sample_type",
    group_order = c("Nerve", "PNF", "ANF", "MPNST"),
    foreground = "SOX10",
    background = "solid",
    reduced_dim = "TSNE",
    fg_order = "ascending"
)

panel_width <- 4
panel_height <- 4
n_panels <- length(unique(sce_mpnst_example$sample_type))

ggplot2::ggsave(
    filename = "SOX10_by_sample_type.pdf",
    plot = p_grouped,
    width = panel_width * n_panels,
    height = panel_height
)

For very dense plots, it is often a good idea to combine saving to PDF or SVG with point-layer rasterization using bg_raster, fg_raster or both.

Session information

sessionInfo()
#> R version 4.6.1 (2026-06-24)
#> Platform: x86_64-pc-linux-gnu
#> Running under: Ubuntu 26.04.1 LTS
#> 
#> Matrix products: default
#> BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.32.so;  LAPACK version 3.12.0
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> time zone: Etc/UTC
#> tzcode source: system (glibc)
#> 
#> attached base packages:
#> [1] stats4    stats     graphics  grDevices utils     datasets  methods  
#> [8] base     
#> 
#> other attached packages:
#>  [1] patchwork_1.3.2             SingleCellExperiment_1.35.2
#>  [3] SummarizedExperiment_1.43.0 Biobase_2.73.2             
#>  [5] GenomicRanges_1.65.4        Seqinfo_1.3.2              
#>  [7] IRanges_2.47.5              S4Vectors_0.51.10          
#>  [9] BiocGenerics_0.59.12        generics_0.1.4             
#> [11] MatrixGenerics_1.25.0       matrixStats_1.5.0          
#> [13] scOverlay_0.99.5            BiocStyle_2.41.0           
#> 
#> loaded via a namespace (and not attached):
#>  [1] sass_0.4.10         SparseArray_1.13.4  lattice_0.23-1     
#>  [4] digest_0.6.39       evaluate_1.0.5      grid_4.6.1         
#>  [7] RColorBrewer_1.1-3  fastmap_1.2.0       jsonlite_2.0.0     
#> [10] Matrix_1.7-6        ggnewscale_0.5.2    ggrastr_1.0.2      
#> [13] BiocManager_1.30.27 viridisLite_0.4.3   scales_1.4.0       
#> [16] jquerylib_0.1.4     abind_1.4-8         cli_3.6.6          
#> [19] rlang_1.3.0         XVector_0.53.0      withr_3.0.3        
#> [22] cachem_1.1.0        DelayedArray_0.39.8 yaml_2.3.12        
#> [25] otel_0.2.0          ggbeeswarm_0.7.3    S4Arrays_1.13.2    
#> [28] tools_4.6.1         ggplot2_4.0.3       buildtools_1.0.0   
#> [31] vctrs_0.7.3         R6_2.6.1            lifecycle_1.0.5    
#> [34] vipor_0.4.7         beeswarm_0.4.0      bslib_0.12.0       
#> [37] gtable_0.3.6        glue_1.8.1          xfun_0.62          
#> [40] sys_3.4.3           knitr_1.52          farver_2.1.2       
#> [43] htmltools_0.5.9     labeling_0.4.3      rmarkdown_2.32     
#> [46] maketools_1.3.2     Cairo_1.7-0         compiler_4.6.1     
#> [49] S7_0.2.2