In this vignette we cover the generation of a qseaSet using the
functions provided in mesa, rather than those in
qsea. Please see the qsea package vignette
vignette("qsea_tutorial", package = "qsea") for more
detail, although we cover much of it here.
The first thing to prepare is the sampleTable: a data
frame containing metadata for the samples to be analysed. This has three
required columns:
sample_name : a name for each sample, which must be
unique. This is the primary identifier used to describe each sample, so
it is good to be descriptive.group : a column to specify a group for each sample,
for use inside certain functions for averaging over samples of the same
type. We suggest only using this for replicates of the same (or very
similar) samples, and adding extra columns for other comparisons that
may be of interest.file_name : a path to a bam file, containing the
aligned, deduplicated reads mapped to the relevant reference genome,
from material which has undergone the methylation enrichment/capture
process (e.g. MeDIP-seq or MBD-seq). Ideally
samtools fixmate has been run to supply mate information
onto the read tags, but filtering for quality is not required.There are also two optional fields that may be used during generation:
input_file : a path to an additional bam file
containing aligned, deduplicated reads for material from the same
sample, without performing enrichment. This is whole-genome sequencing,
but may be sequenced at a much lower depth (e.g. ultra-shallow WGS, ~1-2
million fragments is sufficient for calling copy number variation at a
window size of 1Mb). If present, this will be used on a sample-by-sample
basis to calculate copy number variation, at a chromosome scale, in
order to correct for this in the expected read depths. This is most
relevant when studying cancer, as particular cancers often show
recurrent patterns of copy number changes. Note that below ~0.5 million
fragments the calculated copy number patterns will become especially
noisy (for human samples).
gender or sex : if one of these columns
is included, then any chromosome called X, Y,
chrX or chrY will be set with the appropriate
zygosity for samples with M, m or
male in this column. See qsea.createSet.R in
the qsea source for details; within the National Biomarker
Centre we only consider the autosomes.
In addition, any further metadata columns may be included as required, and we recommend that anything you might want to query or compare the data on is included here. This includes experimental conditions (tissue, treatment, tumour status, etc.).
We also highly recommend the use of a tool such as NGSCheckMate for verifying that (human) samples come from the same individual or not. This is useful both for ensuring that the enriched and non-enriched samples do indeed come from the same individual, and to check that paired samples are actually paired (for instance when comparing pre- and post-treatment).
For the example here, we use one of the example samples in the
MEDIPSData package. The package contains pairs of tumour
and normal tissue from the same patients, with MeDIP-seq performed as
the enrichment step. These bam files are downsampled to only contain
chromosomes 20-22, and are aligned to hg19. Note, we highly recommend
using the hg38/GRCh38 reference genome rather than hg19 as used here!
For speed, we only use the tumour sample from the first patient, and
only chromosome 21.
The qsea tutorial details the steps involved in
generating a qseaSet. We provide a function,
makeQset, which wraps these up into one function, although
you may still use the rest of the mesa functionality with
or without using this to create your qseaSet.
Before creating the qseaSet, you need to have:
sampleTable of metadata, including the file paths
to the bam files.BSgenome package, which must be
installed. You can check which BSgenome packages are available on
Bioconductor via BSgenome::available.genomes(), or make
your own if necessary
(vignette("BSgenomeForge", package = "BSgenome")).qseaSet, the
genome is split into fixed windows with equal size, and all analysis is
performed on these windows as independent units. We generally use 300bp
windows, as that is a little bigger than our average DNA fragment sizes,
which are in the 150-250bp range. If the DNA fragments are particularly
long, a larger window size may make sense. This should also be informed
by your depth of sequencing, as a smaller window size will be more prone
to noise.For our hg19 example data, we set
BSgenome = "BSgenome.Hsapiens.UCSC.hg19", specify the
chromosome used from the test dataset and give some approximate fragment
length and SD values.
makeQset(sampleTable,
BSgenome = "BSgenome.Hsapiens.UCSC.hg19",
chrSelect = "chr21",
windowSize = 300,
fragmentLength = 200,
fragmentSD = 50,
CNVmethod = "None")
#> Detected parallel setup with 2 workers.
#>
#> Attaching package: 'Biostrings'
#> The following object is masked from 'package:base':
#>
#> strsplit
#> Considering 160432 regions with total size 48129600
#> ==== Creating qsea set ====
#> restricting analysis on chr21
#> Dividing provided regions in 300nt windows
#> Scanning up to 2 files in parallel.
#> Generated objects
#> Calculated coverage
#> No CNV being calculated
#> Get genomic positions of "CG" ...
#> searching chr21 for "CG"...
#> found 380444 occurances of CG in chr21
#> estimating expected number of CGs per fragment for windows of chr21...
#> ...done
#> selecting windows with low CpG density for background read estimation
#> 6.415% of the windows have enrichment pattern density of at most 0.05 per fragment
#> and are used for background reads estimation
#> qseaSet object generated successfully
#> qseaSet
#> =======================================
#> 1 Samples:
#> Tumour1
#> 160432 Regions in 1 chromosomes of BSgenome.Hsapiens.UCSC.hg19A range of messages are produced as output during this process. Note
that by default the process is performed without parallelisation; this
can be controlled via the setMesaParallel function and will
vastly speed up the process when multiple cores are available (but use
more RAM).
This function combines several steps:
BSgenome, chrSelect, windowSize
and badRegions parameters.fragmentLength and fragmentSD parameters to
define the expected contribution of fragments to the likelihood that
they overlap the window (see the Calculating CpG Density section
below).The chosen chromosomes (via chrSelect) of the
BSgenome object will be tiled in windowSize
windows. These can be filtered for regions which have poor mappability
or are overrepresented in non-enriched sequencing, by providing a
GRanges object to the badRegions option. If provided, this
will remove any window which overlaps with the GRanges object; ensuring
that the windows remain fixed when this filtering is changed. The 2019
ENCODE exclusion regions are provided as a data object called
ENCODEbadRegions.
The copy number variation is calculated over a set of larger genomic
windows as set by CNVwindowSize; this is typically much
larger than the windowSize due to coverage limitations.
This will be performed using the same options as the enriched samples.
Once the coverage is determined, the R package HMMcopy will
be used, with default parameters, to calculate the copy number
variation. The non-enriched (also known as “Input”) bam files should be
specified by adding a column called input_file to the
sampleTable. To use this you need to provide
HMMcopy with a pair of data frames containing the GC
content and mappability per region you wish to calculate over, as
detailed in vignette("HMMcopy", package = "HMMcopy"). There
is an exception if you are using the human genome hg38 (or
GRCh38), in which case files are provided for window sizes
of 50kb, 500kb or 1000kb. These
are stored in objects called e.g. gc_hg38_1000kb and
map_hg38_1000kb; similar objects need to be generated for
other genomes or window sizes using the tools gcCounter or
mapCounter from the HMMcopy
suite.
Alternatively, copy number variation may be calculated using an
external method, for instance using the ichorCNA
package. This may then be added to the qseaSet object using the
qsea function addCNV.
As a further option, qsea provides a method for
determining the copy number from the methylation enriched samples, using
only reads which do not contain a CG; this may be selected using
CNVmethod = "MeCap". We do note that for our MBD-seq
experiments we found there was reasonable overall agreement, but this
method did give spurious CNV compared to the non-enriched reads,
although that may be due to the selectivity of our MBD-seq enrichment
step being relatively high.
No copy number variation will be calculated if
CNVmethod = "None".
One essential part of the qseaSet is the
CpG_density for each region. This is calculated using the
genome sequence, alongside an expected fragment size distribution via
the qsea function
qsea:::estimatePatternDensity().
This process works by first locating all CG
dinucleotides across the genome, before modelling a distribution around
each, representing the likelihood for a fragment including that
dinucleotide to reach adjacent positions. This distribution is assumed
to be normally distributed, with a mean and standard deviation. Thus
each CG contributes to the density of the base pairs around
it. Finally, these values are averaged across the windows to give a
CpG_density column on the regions of the
qseaSet. This method is designed to capture the fact that
some captured fragments of DNA may be heavily methylated at one end, but
if the fragment is long it may be counted in an adjacent window.
This calculation therefore requires the fragmentLength
and fragmentSD parameters to be specified, to define these
normal distributions, and is associated with the whole
qseaSet. This is therefore not sample dependent, and so if
samples have vastly different fragment distributions then there will be
some expected variation not captured in this process. This does have a
significant effect on the calculated beta values for each sample, so it
is important to set the fragmentLength and
fragmentSD values in an appropriate way. We also provide a
fragmentType option, which hardcodes these two parameters
to values derived from our initial sheared or cfDNA datasets. When
fragmentType = "Sheared" is used this sets
fragmentLength = 213, fragmentSD = 60, while
fragmentType = "cfDNA" sets
fragmentLength = 167, fragmentSD = 38, but we do emphasise
that you should check your own data. Ideally the
CpG_density would be a property of the fragment
distribution of each individual sample, but mesa does not
support this at this time.
Please note that these fragmentLength and
fragmentSD parameters are only used for calculating this CG
density metric; they are not used to control what reads are included in
the coverage calculations.
In any ME-seq experiment, there will be off-target fragments
sequenced due to the imperfect nature of the enrichment process. These
are used to calculate the offset; a measure of the
background level of fragments in windows that do not contain CGs. The
maximum CpG_density value for these windows is controlled
by maxPatternDensity. If this is set too low, then
qsea will give a warning saying that the offset cannot be
calculated. In particular, this could be an issue if using a targeted
enrichment step in your library preparation method.
Finally, the qsea beta value calculation is applied.
This requires the CpG_density and the coverage, as well as
some assumption about which regions are methylated, or at least how many
of them are. The qsea
manuscript details three methods for this assumption, two of which
require external data in the form of methylation arrays or
bisulphite/enzymatic sequencing. The third method, known as “blind
calibration” in the qsea manuscript, is the only one implemented here.
This works on a global assumption about the typical methylation levels
across the human genome: that the majority of isolated CGs are
methylated, while the majority of CpG islands are unmethylated. This
blind calibration method therefore assumes an average methylation level
of 76% in windows with a CpG_density of 1, falling along a
straight line to 25% in windows with a CpG_density of 15.
This method is the only one implemented here, as it does not require
additional data.
We have found this blind calibration sufficient for the human genome, including being able to build classifiers using methylation arrays as our data source.
If you wish to use array data for normalisation, then this Bioconductor support post may be a good source of information, but we have only used the blind calibration method.
One thing to note is that this function does not set the
library_factor for the qseaSet, which
qsea suggests doing via the Trimmed Mean of M-values (TMM)
method. The way qsea does this is to choose one sample (by
default the first sample), and use this as the reference sample to
calculate a scaling factor to adjust the total fragments by when
calculating the NRPM (normalised reads per million) values.
TMM normalisation is a method used in RNA-seq to compensate for genes which are excessively expressed in a particular condition. In RNA-seq, the distribution of read counts varies over ~4 orders of magnitude, with some highly expressed genes having tens of thousands of reads while others have tens or hundreds. If in a particular tissue there is a group of genes which are extremely highly expressed that are not expressed in another tissue, then the total pool of available reads will be dominated by those genes in the first tissue, comparatively depressing the other genes. So just using the total size ends up biased towards these highly expressed genes, making it look like the other genes are less expressed, and giving many more differentially expressed genes than might actually be the case.
For example, if there are 100 reads in a sample, and 3 genes, with values:
then we want to be careful not to call the last two genes as differentially expressed, when they are really just affected by the lack of the first gene. One option could be to normalise with the median of the read distribution, although, as many genes are not expressed, this is often implemented as an upper quartile.
TMM normalisation however takes some subset of the “genes” (windows in our case), having trimmed off the very high/very low expressed “genes”, to calculate this scaling factor for total number of fragments.
However, we find that for methylation, after filtering for excluded regions, extremely few windows in any sample have more than 100 reads. So the distribution of read counts only spans roughly 0-100, removing the main reason for TMM normalisation.
There are a couple of reasons why we therefore choose not to apply TMM:
calculateDMRs calculation), there is no effect
of the library_factor, as it is normalised out in the beta
value fitting process.qsea).
This therefore means that the normalisation of all the samples depends
on which other samples they were normalised with, meaning that comparing
between batches would need renormalising every time. This would also
mean that filtering samples pre- or post-normalisation would give
different results.However, we note that these considerations might not apply for
everyone; the qsea::addLibraryFactors() function may be
used to apply this normalisation if required.
#> R version 4.6.2 (2026-10-29)
#> Platform: x86_64-pc-linux-gnu
#> Running under: Ubuntu 26.04.1 LTS
#>
#> Matrix products: default
#> BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.32.so; LAPACK version 3.12.0
#>
#> locale:
#> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
#> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
#> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
#> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
#> [9] LC_ADDRESS=C LC_TELEPHONE=C
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
#>
#> time zone: Etc/UTC
#> tzcode source: system (glibc)
#>
#> attached base packages:
#> [1] stats4 stats graphics grDevices utils datasets methods
#> [8] base
#>
#> other attached packages:
#> [1] BSgenome.Hsapiens.UCSC.hg19_1.4.3 BSgenome_1.81.1
#> [3] rtracklayer_1.73.0 BiocIO_1.23.3
#> [5] Biostrings_2.81.9 XVector_0.53.0
#> [7] GenomicRanges_1.65.4 Seqinfo_1.3.2
#> [9] tibble_3.3.1 org.Mm.eg.db_3.24.6
#> [11] org.Hs.eg.db_3.24.6 AnnotationDbi_1.75.2
#> [13] IRanges_2.47.5 S4Vectors_0.51.10
#> [15] Biobase_2.73.2 BiocGenerics_0.59.13
#> [17] generics_0.1.4 stringr_1.6.0
#> [19] tidyr_1.3.2 ggplot2_4.0.4
#> [21] mesa_0.99.11 dplyr_1.2.1
#> [23] qsea_1.39.0 rmarkdown_2.32
#>
#> loaded via a namespace (and not attached):
#> [1] bitops_1.1-0
#> [2] ggplotify_0.1.4
#> [3] polyclip_1.10-7
#> [4] janitor_2.2.1
#> [5] enrichit_0.2.7
#> [6] XML_3.99-0.25
#> [7] lifecycle_1.0.5
#> [8] doParallel_1.0.17
#> [9] lattice_0.23-1
#> [10] MASS_7.3-66
#> [11] magrittr_2.0.5
#> [12] limma_3.99.5
#> [13] sass_0.4.10
#> [14] jquerylib_0.1.4
#> [15] yaml_2.3.12
#> [16] plotrix_3.8-14
#> [17] otel_0.3.0
#> [18] ggtangle_0.1.5
#> [19] DBI_1.3.0
#> [20] buildtools_1.0.0
#> [21] RColorBrewer_1.1-3
#> [22] lubridate_1.9.5
#> [23] abind_1.4-8
#> [24] purrr_1.2.2
#> [25] RCurl_1.98-1.20
#> [26] yulab.utils_0.2.6
#> [27] tweenr_2.0.3
#> [28] rappdirs_0.3.4
#> [29] gdtools_0.6.0
#> [30] circlize_0.4.18
#> [31] enrichplot_1.99.8
#> [32] ggrepel_0.9.8
#> [33] tidytree_0.4.9
#> [34] maketools_1.3.2
#> [35] ChIPseeker_1.49.5
#> [36] codetools_0.2-20
#> [37] DelayedArray_0.39.8
#> [38] DOSE_4.7.4
#> [39] ggforce_0.5.0
#> [40] shape_1.4.6.1
#> [41] tidyselect_1.2.1
#> [42] aplot_0.3.2
#> [43] UCSC.utils_1.9.1
#> [44] HMMcopy_1.55.1
#> [45] farver_2.1.2
#> [46] matrixStats_1.5.0
#> [47] GenomicAlignments_1.49.2
#> [48] jsonlite_2.0.0
#> [49] GetoptLong_1.1.1
#> [50] iterators_1.0.14
#> [51] systemfonts_1.3.3
#> [52] foreach_1.5.2
#> [53] tools_4.6.2
#> [54] ggnewscale_0.5.2
#> [55] progress_1.2.3
#> [56] treeio_1.37.1
#> [57] TxDb.Hsapiens.UCSC.hg19.knownGene_3.24.0
#> [58] ragg_1.5.2
#> [59] Rcpp_1.1.2
#> [60] glue_1.8.1
#> [61] gridExtra_2.3.1
#> [62] SparseArray_1.13.5
#> [63] BiocBaseUtils_1.15.1
#> [64] xfun_0.62
#> [65] MatrixGenerics_1.25.0
#> [66] GenomeInfoDb_1.49.3
#> [67] withr_3.0.3
#> [68] fastmap_1.2.0
#> [69] boot_1.3-32
#> [70] caTools_1.18.4
#> [71] digest_0.6.39
#> [72] timechange_0.4.0
#> [73] R6_2.6.1
#> [74] gridGraphics_0.5-1
#> [75] colorspace_2.1-4
#> [76] textshaping_1.0.5
#> [77] GO.db_3.24.0
#> [78] gtools_3.9.5
#> [79] RSQLite_3.53.3
#> [80] cigarillo_1.3.1
#> [81] UpSetR_1.4.1
#> [82] utf8_1.2.6
#> [83] fontLiberation_0.1.0
#> [84] data.table_1.18.6.1
#> [85] treelayout_0.1.0
#> [86] prettyunits_1.2.0
#> [87] httr_1.4.9
#> [88] htmlwidgets_1.6.4
#> [89] S4Arrays_1.13.2
#> [90] scatterpie_0.2.6
#> [91] pkgconfig_2.0.3
#> [92] gtable_0.3.6
#> [93] MEDIPSData_1.49.0
#> [94] blob_1.3.0
#> [95] ComplexHeatmap_2.29.0
#> [96] S7_0.2.2
#> [97] sys_3.4.3
#> [98] htmltools_0.5.9
#> [99] fontBitstreamVera_0.1.1
#> [100] clue_0.3-68
#> [101] plyranges_1.33.2
#> [102] scales_1.4.0
#> [103] hues_0.2.0
#> [104] TxDb.Hsapiens.UCSC.hg38.knownGene_3.24.0
#> [105] png_0.1-9
#> [106] snakecase_0.11.1
#> [107] ggfun_0.2.2
#> [108] knitr_1.52
#> [109] reshape2_1.4.5
#> [110] rjson_0.2.23
#> [111] nlme_3.1-172
#> [112] curl_8.1.0
#> [113] GlobalOptions_0.1.4
#> [114] cachem_1.1.0
#> [115] zoo_1.9-1
#> [116] KernSmooth_2.23-27
#> [117] parallel_4.6.2
#> [118] restfulr_0.0.17
#> [119] pillar_1.11.1
#> [120] grid_4.6.2
#> [121] vctrs_0.7.3
#> [122] gplots_3.3.0
#> [123] tidydr_0.0.7
#> [124] cluster_2.1.8.3
#> [125] evaluate_1.0.5
#> [126] GenomicFeatures_1.65.1
#> [127] cli_3.6.6
#> [128] compiler_4.6.2
#> [129] Rsamtools_2.29.0
#> [130] rlang_1.3.0
#> [131] crayon_1.5.3
#> [132] labeling_0.4.3
#> [133] plyr_1.8.9
#> [134] fs_2.1.0
#> [135] ggiraph_0.9.6
#> [136] stringi_1.8.9
#> [137] TxDb.Mmusculus.UCSC.mm10.knownGene_3.10.0
#> [138] BiocParallel_1.47.0
#> [139] lazyeval_0.2.3
#> [140] GOSemSim_2.39.4
#> [141] fontquiver_0.2.1
#> [142] Matrix_1.7-6
#> [143] hms_1.1.4
#> [144] patchwork_1.3.2
#> [145] bit64_4.8.6
#> [146] KEGGREST_1.53.6
#> [147] statmod_1.5.2
#> [148] SummarizedExperiment_1.43.0
#> [149] igraph_2.3.4
#> [150] memoise_2.0.1
#> [151] bslib_0.12.0
#> [152] ggtree_4.3.2
#> [153] bit_4.6.0
#> [154] ape_5.8-1