--- title: "Preprocessing an HT-SELEX study" author: "Marco Rota Negroni" date: "`r Sys.Date()`" output: BiocStyle::html_document vignette: > %\VignetteIndexEntry{Preprocessing an HT-SELEX study} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include=FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>") ``` # From public reads to a Bioconductor experiment High-throughput SELEX enriches binding sequences over successive selection rounds. Public deposits commonly provide raw reads without a uniform record of primer layout, trimming decisions, or the relationship between rounds. `selexprepR` combines these preprocessing steps and returns a sparse `SummarizedExperiment`: rows represent recovered sequences, columns represent rounds, and the `counts` assay contains their observed read counts. The experiment also retains the inferred library layout, extraction yields, quality-control diagnostics and a provenance manifest. Inspect these decisions before supplying the counts to downstream Bioconductor workflows. The existing `SELEX` package provides a complementary analysis workflow based on its own sample annotation model. Install with `BiocManager::install("selexprepR")` in a supported Bioconductor installation. All examples below use bundled files and run without downloading reads or installing software. ```{r load-package} library(selexprepR) ``` # Start with your own reads | What you have | Where to start | | --- | --- | | One FASTQ per round | Name a vector of paths `round_00`, `round_01`, and so on, using the actual round numbers. Pass it to `run_selexprep_files()`. | | Paired-end reads or several files per round | Pass a table with `path`, `round_number` and `mate` columns to `run_selexprep_files()`. Give each file its own row and use `"R1"` and `"R2"` to identify paired mates. | | Reads with inline round barcodes | Read the FASTQ with `read_selexprep_fastq()` and call `selexprep_demultiplex()` with your barcode-to-round map. Review unassigned reads and pass the result to `run_selexprep()`. | If you know the flanking sequences, supply `primer_5p` and `primer_3p` in the orientation of the reads you intend to count. To infer the layout first, load your round-separated reads with `read_selexprep_inputs()` and inspect the report from `selexprep_detect()` before relying on the inferred boundaries. The example below shows both layout inspection and explicit primers. # A small real-data example The example reads come from Zibin Zhao's hydrocortisone DL-SELEX deposit, [Zenodo 14272647](https://doi.org/10.5281/zenodo.14272647), associated with *Structure-enhanced Deep Learning Accelerates Aptamer Selection for Small Molecule Families like Steroids*. The deposit contains paired sequencing of selection rounds 3, 5 and 7. We distribute only the first 600 complete R1 records from each of those rounds, preserving their sequences, read identifiers and quality strings. The files are separately compressed subsets, not copies of the entire original FASTQ files. Taking the first records is deterministic, but is not random sampling of the selection pools. Such a small sample cannot establish binding affinity or the population's enrichment trajectory. R2 reads are not included in this single-read example. Source attribution, the data license, original filenames, checksums and the subsetting method are recorded in `REAL_DATA_PROVENANCE.md` in the installed package. ```{r real-fastq-paths} rounds <- c(3L, 5L, 7L) paths <- system.file( "extdata", sprintf("cs_round_%02d.fastq.gz", rounds), package = "selexprepR", mustWork = TRUE ) names(paths) <- sprintf("round_%02d", rounds) reads <- read_selexprep_fastq(paths[[1L]]) length(reads) head(reads) ``` `read_selexprep_fastq()` accepts plain or compressed FASTQ and returns a `Biostrings::BStringSet`. This container preserves the input alphabet, including RNA `U`. This step does not filter by quality; the quality strings remain in the original FASTQ for a separate quality-filtering step. The path-vector names supply the round numbers. Alternatively, use a table with explicit `path`, `round_number` and `mate` columns. The input bridge pools files belonging to the same round and mate and records their loaded read counts, file sizes and checksums. ```{r round-aware-inputs} inputs <- read_selexprep_inputs(paths) inputs$files[, c("round_number", "mate", "reads_loaded", "sha256")] vapply(inputs$sequences_by_round, length, integer(1)) ``` # Inspect the library layout Primer inference starts with the earliest supplied round and checks support across the remaining rounds. Here the earliest supplied sample is round 3; an unselected round 0 is not included. Run-specific outer tags may be removed while the inner 12 bases adjacent to the insert remain protected. ```{r detect-layout} report <- selexprep_detect(inputs$sequences_by_round) report[c( "status", "failure_reason", "primer_5p", "primer_3p", "extraction_mode" )] ``` Here the detector returns `UNABLE_TO_INFER`. Its `failure_reason` reports that no candidate 5' or 3' flank was found in the earliest round. The safe failure is useful information; it does not silently count untrimmed reads. When inference fails on your data, take the flanks from the library design or sequencing protocol and supply them as explicit primers, as shown below. The [article's supplement](https://doi.org/10.1093/bib/bbaf680), Table S8, marks the flanks of its aptamers in lowercase. These provide an external source for explicit primer overrides. We target the R1 orientation opposite to the aptamer sequences printed in that table, so we reverse-complement the two published flanks and exchange their sides: ```{r explicit-boundaries} published_5p <- "GGCTCTCGGGACGAC" published_3p <- "CTGCGTTCTCCATTCTGG" primer_5p <- reverse_complement(published_3p) primer_3p <- reverse_complement(published_5p) ``` This is an explicit choice to demonstrate one orientation of the raw R1 pool: reads in the published orientation are present in similar numbers, so either choice would be possible here. This example counts only reads containing both primers in the chosen orientation; opposite-orientation reads and other unmatched reads remain excluded and are not added to the counts. The result is therefore a documented subset, not a count of every molecule in the deposit. The counted sequences are the reverse complement of the aptamers printed in Table S8; use `reverse_complement(rownames(experiment))` to compare them with that table. Primers are located as cutadapt locates adapters: anywhere in a read, with up to 10% substitutions or indels, and possibly cut short at the read ends (see `?selexprep_extract`). The retention and QC warnings below remain part of the interpretation. `HIGH` and `MEDIUM` in an inferred report would describe read support, not independent confirmation of a biological constant/random boundary. Review the published library design before interpreting a selected pool, especially when its early unselected round is unavailable. # Extract, count and inspect provenance `run_selexprep_files()` runs detection, extraction, counting and QC from local FASTQ files. The same workflow accepts `inputs` through `run_selexprep(inputs)`. The source deposit is paired-end. All bundled R1 reads are 150 nucleotides long, enough to cover the expected 72-nucleotide construct (two flanks and a 39-nucleotide variable region), so this example supplies R1 only. Consider the same when your R1 reads are long enough to contain both flanks. For paired inputs, specify both mates in every round. When the primers are split across mates, the trimmed sides remain separate and the experiment has no counted rows. In that case, merge the read pairs with an external tool and supply the merged FASTQ as one file per round. ```{r file-workflow} experiment <- run_selexprep_files( paths, primer_5p = primer_5p, primer_3p = primer_3p, extraction_mode = "BOTH_PRIMERS_SINGLE_READ" ) experiment counts <- SummarizedExperiment::assay(experiment, "counts") round_info <- SummarizedExperiment::colData(experiment) stopifnot(identical(colnames(counts), rownames(round_info))) data.frame( round = round_info$round, counted_reads = unname(Matrix::colSums(counts)) ) head(SummarizedExperiment::rowData(experiment)) ``` Use `counts` and the aligned `round_info` as inputs for downstream work. The numeric round of each input file is recorded in the `round_number` column of `S4Vectors::metadata(experiment)$input_files`; the final section exports one count table per round. Before comparing rounds, check the round assignments against your study design, then review extraction yields and QC. The counts here cover the selected read orientation. A zero means a sequence was not observed among the retained reads for that round. ```{r provenance-and-yields} metadata <- S4Vectors::metadata(experiment) metadata$extraction$input_reads metadata$extraction$output_reads metadata$extraction$input_yields metadata$input_files[, c("round_number", "reads_loaded", "md5")] metadata$manifest$manifest_version ``` Extraction warns when a round retains less than half its reads or less than half the best round's retained fraction. Inspect `low_yield_rounds` and `low_yield_inputs` in the extraction metadata. File-level checks prevent pooling from hiding one poorly matching input. Retention measures extraction success; it does not assess affinity. # Quality control and its plot The QC report summarizes depth, diversity, sequence lengths, GC content and other diagnostics. We keep the default minimum read-depth threshold, so a low-depth flag is expected for this small example. Flags identify conditions to review; they do not automatically remove samples or reads. ```{r qc-tables} qc <- metadata$qc qc$per_round qc$flags[, c("name", "severity")] ``` The `evidence` column holds the values behind each flag. In this example: - `low_total_reads`: every round has fewer than the default 10,000 reads. - `strand_mix`: more than 20% of the reads in each round start with the reverse complement of `primer_3p`, i.e. the opposite orientation. - `low_primer_match`: with supplied primers, the match rates are measured for those primers on the earliest round (reads containing the primer with at most one substitution). Here they are 0.39 for `primer_5p` and 0.38 for `primer_3p`, below the 0.40 threshold, because about half of the reads are in the other orientation. - `adapter_contamination_high`: 489 of the 600 raw reads used for detection contain the TruSeq read-1 adapter. This describes the raw reads; the `truseq_reads_fraction` of the extracted sequences above is 0. Supply the experiment as the second argument to `plot()` so the plots can use extraction provenance and sequence-length distributions. ```{r qc-retention, fig.width=6, fig.height=3.6, dpi=160, out.width="100%", fig.cap="Input and retained reads by round."} plot(qc, experiment, which = "retention", cex = 1.2) ``` Each round starts with 600 reads. The retained fraction ranges from about 38% to 50%. Extraction targets one orientation in a mixed input, and reads without both matching primers are excluded. On your data, inspect the primer sequences and read orientation if a round has unexpectedly low retention. ```{r qc-length, fig.width=6, fig.height=3.6, dpi=160, out.width="100%", fig.cap="Read-weighted lengths of the extracted regions."} plot(qc, experiment, which = "length", cex = 1.2) ``` Each point shows the fraction of retained reads at an observed length in one round. All three rounds peak at 39 nucleotides (219/228, 258/278 and 270/297 reads). This agrees with the 39-nucleotide variable region reported for CS round 3 in Supplementary Note S8. A few reads extend past 100 nucleotides (2, 2 and 5 reads, respectively). The plot measures the sequences recovered between the chosen flanks. All nine sequences above 100 nucleotides contain a further copy of `primer_5p`, and eight contain the first 10 nucleotides of `primer_3p`; these are hints about their structure, not a determined origin. Each is counted as one sequence. Compare the peak with your library design and inspect the original reads and primer matches behind an unexpected tail before interpreting it. ```{r qc-diversity, fig.width=6, fig.height=9.6, dpi=160, out.width="100%", fig.cap="Sequence diversity within each retained round."} plot(qc, experiment, which = "diversity", mfrow = c(3, 1), cex = 1.15, mar = c(5.3, 5.5, 2.5, 1), mgp = c(3.7, 0.7, 0)) ``` The diversity panels show the number of distinct sequences, Shannon entropy and the fraction of reads contributed by the 100 most abundant sequences in each round. Compare them alongside read depth and retention when assessing how strongly a small set of sequences dominates a pool. In this example each round has only 208 to 247 distinct sequences among 228 to 297 reads, so the 100 most abundant sequences are already a large share of the distinct sequences observed. Read their coverage of about 0.48 to 0.53 in that context. # Inline-barcode demultiplexing For an R1 stream containing rounds distinguished by inline barcodes, supply a barcode-to-round map to `selexprep_demultiplex()`. Barcodes must have enough pairwise Hamming distance for the allowed mismatches. They are never inferred. Paired mates, when supplied, follow their R1 assignments. The deposited files are already separated by round. To demonstrate this step offline, we add **artificial teaching barcodes** to their real read sequences and append one deliberately unassignable read. These tags and the extra read are not part of the original experiment. Removing them must recover the three input pools exactly. ```{r demultiplex} barcodes <- c(AAAAA = 3L, CCCCC = 5L, GGGGG = 7L) tagged <- Map(function(pool, barcode) { paste0(barcode, as.character(pool)) }, inputs$sequences_by_round, names(barcodes)) multiplexed <- c(unlist(tagged, use.names = FALSE), "TTTTTACGT") demultiplexed <- selexprep_demultiplex(multiplexed, barcodes) demultiplexed$summary length(demultiplexed$unassigned$r1) sequences_only <- function(pools) { lapply(pools, function(pool) unname(as.character(pool))) } stopifnot(identical( sequences_only(demultiplexed$sequences_by_round), sequences_only(inputs$sequences_by_round) )) demultiplexed_experiment <- run_selexprep( demultiplexed, primer_5p = primer_5p, primer_3p = primer_3p, extraction_mode = "BOTH_PRIMERS_SINGLE_READ" ) stopifnot(identical( SummarizedExperiment::assay(demultiplexed_experiment, "counts"), counts )) ``` # Save decisions and discover a full study Save library reports and manifests as JSON alongside downstream results. This example writes them to a temporary directory. ```{r save-decisions} outdir <- tempfile("selexprep_example_") dir.create(outdir) write_library_report(metadata$library_report, file.path(outdir, "library_report.json")) write_selexprep_manifest(metadata$manifest, file.path(outdir, "selexprep_manifest.json")) list.files(outdir) ``` Count tables use a stable four-column format (`sequence`, `reads`, `rank`, `rpm`) and can be written as TSV, CSV, RDS or Parquet. Here we build one table per round from the extracted sequences, check that it matches the count matrix, and write it with its round number in the file name. ```{r export-counts} round_numbers <- sort(unique(metadata$input_files$round_number)) round_numbers count_tables <- lapply(metadata$extraction$sequences_by_round, selexprep_count) stopifnot(identical( vapply(count_tables, function(table) sum(table$reads), numeric(1)), Matrix::colSums(counts) )) for (round in names(count_tables)) { write_selexprep_counts(count_tables[[round]], file.path(outdir, paste0(round, "_counts.tsv"))) } list.files(outdir) head(read_selexprep_counts(file.path(outdir, "round_03_counts.tsv")), 3) ``` The bundled catalog is a frozen, offline discovery snapshot from Python selexprep v0.4.6. It contains 240 records with per-field curation status. A discovery entry is not a guarantee of read availability or complete experimental annotation. ```{r public-catalog} selexprep_catalog("hydrocortisone") data("selexprep_public_catalog", package = "selexprepR") S4Vectors::metadata(selexprep_public_catalog)$snapshot_version ``` For ENA studies, inspect current metadata with `inspect_selexprep_accession("PRJDB19098")`, then review the dry-run plan from `fetch_selexprep_reads(inspection, outdir)` before requesting a download with `dry_run = FALSE`. These network operations are described here rather than executed during vignette builds: they depend on a live external archive and the complete FASTQ files can be large. ENA inspection uses sample attributes and text metadata for round inference. Ambiguous or conflicting assignments need review, and multiple BioSamples assigned to one round should be checked before merging lanes, replicates or parallel selections. # Session information ```{r session-info} sessionInfo() ```