GExPipe is a Shiny application for bulk
RNA-seq and microarray analysis. This vignette walks through the guided
15-step interface end to end — download GEO data, run QC and
normalization, correct batch effects, discover differentially expressed
genes, build co-expression networks, refine candidates with PPI and
machine learning, and export a clinical summary report — without
manually wiring separate Bioconductor tools together. For mixed
microarray studies (Arraystar, Affymetrix, Agilent, custom probe IDs),
the download step automatically runs STEP 2b to map
probes to HGNC gene symbols before merging — see Gene ID mapping below.
| Section | Purpose |
|---|---|
| How the Shiny app is organised | Pipeline overview figure |
| Installation | Bioconductor and GitHub install |
| Run latest code for analysis | Avoid stale installs; use main /
pkgload |
| Launch the Shiny application | Start the app (runGExPipe() +
shiny::runApp()) |
| Step-by-step Shiny walkthrough | Actions and screenshots for all 15 steps |
| Gene ID mapping | Probe / accession → symbol conversion (STEP 2b) |
| Programmatic example | Optional scripting with bundled CSV data |
| Troubleshooting | Common install and runtime issues |
The package builds on GEOquery for data retrieval, limma, DESeq2, and edgeR for differential expression, WGCNA for co-expression networks, and clusterProfiler for functional enrichment.
The sidebar lists 15 sequential steps grouped into four phases. Each step unlocks only after the previous one succeeds. The welcome screen explains the workflow; click Go to Analysis to open Step 1.
| Phase | Steps | What you do |
|---|---|---|
| Data preparation | 1 – 5 | Download GEO data → QC & visualization → Normalize → Select groups → Batch correction |
| Gene discovery | 6 – 8 | Differential expression → WGCNA → DEG ∩ hub genes |
| Candidate refinement | 9 – 12 | PPI network → ML feature selection → Validation → ROC |
| Clinical translation | 13 – 15 | Nomogram → GSEA → Summary PDF report |
GExPipe 15-step pipeline. Each box corresponds to one sidebar tab in the Shiny app.
Follow the Bioconductor installation instructions and the official install guide to ensure your R version matches a supported Bioconductor release (e.g. Bioconductor 3.22 for R 4.6).
Install GExPipe and all runtime dependencies with
BiocManager:
if (!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
BiocManager::install("GExPipe", dependencies = TRUE)Verify the installation:
When installed from Bioconductor, dependencies are resolved at
install time; runGExPipe() does not
download packages at launch.
Until the package is on Bioconductor, install from GitHub. Requires R ≥ 4.5.0.
options(timeout = 3600)
if (!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
options(repos = BiocManager::repositories())
if (!requireNamespace("remotes", quietly = TRUE))
install.packages("remotes")
remotes::install_github(
"safarafique/GExPipe",
ref = "main",
dependencies = TRUE,
INSTALL_opts = "--no-staged-install"
)Quick start without installing: see Run latest code for analysis (recommended for mixed microarray merges).
For mixed microarray merges (probe IDs across platforms), also install annotation helpers once:
If you installed from GitHub some time ago, reinstall or launch from
the latest main branch so probe-ID mapping (STEP 2b) and
other fixes are included:
if (!requireNamespace("shiny", quietly = TRUE)) install.packages("shiny")
if (!requireNamespace("pkgload", quietly = TRUE)) install.packages("pkgload")
shiny::runGitHub(
"safarafique/GExPipe",
ref = "main",
subdir = "inst/shinyapp",
destdir = tempfile()
)Bioconductor Shiny apps return an app object from the package; users
start the server with shiny::runApp() (Bioconductor
Shiny guide). If you need probe-ID conversion on mixed platforms,
prefer Run latest code for analysis over
a months-old install.
The browser opens at the welcome screen. Click Go to Analysis to begin Step 1 - Download Data.
options(gexpipe.auto_install = TRUE) # GitHub installs only
app <- GExPipe::runGExPipe()
shiny::runApp(app, port = 3838L)For users without a local R installation, run the app in Google Colab
(Runtime → Change runtime type → R). Use a tunnel
(e.g. ngrok) to expose port 3838, then install and start the app with
host = "0.0.0.0". See the GitHub README for full
Colab cells.
The sections below mirror the sidebar tabs. For each step, read the goal, apply the settings under What to do, run the primary action button, and confirm the success check before continuing.
The sidebar tabs for this phase are 1. Download
Data, 2. QC & Visualization, 3.
Normalize Data, 4. Select Groups, and
5. Batch Correction. The walkthrough uses a two-study
RNA-seq example (GSE50760, GSE104836); any
valid GEO accessions work the same way.
Sidebar: 1. Download Data
Goal: Download GEO series, map row IDs to gene symbols, and build a combined genes × samples matrix for downstream steps.
What to do:
GSE50760, GSE104836).For multiple microarray platforms, install
org.Hs.eg.db and biomaRt (see Installation) so STEP 2b
converts probe IDs to HGNC symbols before merging.
Success check: Download log lists samples per GSE;
STEP 2b runs when needed; row names are symbols
(TP53, BRCA1, …); Common
genes > 0.
Step 1: Select platform, dataset mode, DE method, and GSE IDs.
Sidebar: 2. QC & Visualization
Goal: Check gene overlap, inspect raw expression, and detect outlier samples before normalization.
What to do:
Success check: Overlap and QC plots look reasonable; outliers reviewed.
Step 2a: Venn and UpSet gene overlap.
Step 2b: Raw expression boxplot.
Step 2c: PCA and connectivity outlier detection.
Sidebar: 3. Normalize Data
Goal: Make expression values comparable across samples and datasets.
What to do:
Count-based DE (DESeq2 / edgeR / limma-voom from Step 1): the panel shows Normalization Auto-Handled — proceed with Go to Step 4: Select Groups. Background TMM + quantile still runs for WGCNA and heatmaps.
limma path (microarray or merged):
Success check: Q–Q points follow the diagonal; sample medians align.
Step 3a: Overall density and Q-Q plots after normalization.
Step 3b: Median/range alignment and distribution overlap.
Sidebar: 4. Select Groups
Goal: Assign each sample to Normal, Disease, or Exclude.
What to do:
characteristics_ch1, or
title when treatment is only in sample
titles).Success check: Group Summary counts match your study design.
Step 4a: Select phenotype column per GSE.
Step 4b: Group Summary after Apply Categorization.
Sidebar: 5. Batch Correction
Goal: Remove technical variation between datasets while keeping biological Normal vs Disease signal.
What to do:
Note: Step 5 is skipped when only one GSE was loaded in Step 1.
Success check: After-correction PCA (panel b) shows datasets intermingled.
Step 5a: Variance filtering and batch correction method.
Step 5b: PCA by dataset before and after batch correction.
Sidebar:
6. Differential Expression Analysis
Goal: Identify genes differing between Normal and Disease.
What to do:
filterByExpr gene
counts, samples used, and batch-in-model note.Success check: Volcano shows expected direction; DEG count is plausible for your contrast; pipeline verification lists the correct method and samples.
Step 6a: DE parameters, statistical model, and pipeline verification.
Step 6b: Volcano plot and top DEG table.
Sidebar: 7. WGCNA Analysis
Goal: Find co-expression modules associated with disease and extract hub genes from significant modules.
What to do:
The WGCNA tab has internal sub-steps; complete them in order:
Success check: Soft-threshold R² ≥ 0.80; at least one module correlates with Disease; hub genes are listed for significant modules.
Step 7a: WGCNA data preparation and gene selection.
Step 7b: Soft-threshold power selection.
Step 7c: Module dendrogram and module-trait heatmap.
Step 7d: Significant modules and hub gene summary.
Sidebar:
8. Common Genes (DEG & WGCNA)
Goal: Intersect DEGs (Step 6) with genes in significant WGCNA modules (Step 7), then run GO/KEGG enrichment on the overlap.
What to do:
Requirements: Steps 6 and 7 (including Identify Significant Modules) must be complete.
Success check: Venn overlap > 0; GO/KEGG plots show interpretable terms.
Step 8a: Common genes Venn diagram (DEG intersect WGCNA).
Step 8b: GO enrichment of common genes.
Step 8c: KEGG enrichment of common genes.
Sidebar: 9. PPI Interaction
Goal: Map common genes onto a STRING protein–protein interaction network and identify hub proteins.
What to do:
Requirements: Step 8 common genes computed.
Success check: Network builds with mapped edges; hub genes listed; ML extract succeeds.
Step 9a: Run PPI and interactive vs non-interactive gene table.
Step 9b: PPI network graph and hub gene metrics.
Sidebar:
10. Machine Learning Process
Goal: Rank genes by predictive importance using multiple ML algorithms and find genes selected by ≥ 2 methods.
What to do:
Note: If glmnet fails after install, restart R and relaunch the app.
Success check: At least one common ML gene; Venn overlap across methods is non-empty.
Step 10a: Select ML methods and run analysis.
Step 10b: Method overlap Venn diagram and common ML genes.
Sidebar: 11. Validation Setup
Goal: Test generalization on an external GEO cohort or an internal 70/30 train–test split. Target accuracy ≥ 70%.
What to do:
Requirements: Step 10 ML analysis complete with at least one common ML gene.
Success check: External mode shows downloaded samples and validation DE results; internal mode confirms the 70/30 split is ready for ROC and nomogram.
Step 11a: Validation mode and external validation settings.
Step 11b: External validation DE results.
Sidebar: 12. ROC Curve Analysis
Goal: Per-gene AUC with 95% CI (pROC). Retain genes with AUC ≥ 0.80 in training and validation.
What to do:
Requirements: Step 10 common ML genes; Step 11 validation mode selected.
Success check: At least one gene with training AUC ≥ 0.80; validation AUC reported when external data are loaded.
Step 12a: Training ROC curves and AUC summary.
Step 12b: Training vs validation AUC comparison.
The sidebar tabs for this phase are 13. Diagnostic Nomogram Model, 14. GSEA Analysis, and Results Summary.
Sidebar:
13. Diagnostic Nomogram Model
Goal: Build a points-based clinical score
(rms::lrm()), calibration curve, and decision curve
analysis.
What to do:
Requirements: Batch correction (Step 5), group metadata, and common ML genes from Step 10 (or common DEG ∩ WGCNA genes from Step 8).
Success check: Nomogram renders; training AUC is reasonable; validation metrics match your chosen validation strategy.
Step 13: Nomogram, calibration, and decision curves.
Sidebar:
14. GSEA Analysis (Signature Genes)
Goal: Pathway-level enrichment (GO, KEGG, MSigDB Hallmark/C2/C5) stratified by final gene expression.
What to do:
Requirements: Batch-corrected expression and common ML genes from Step 10 (or Step 8 overlap).
Success check: GSEA completes for each target gene; enrichment plots and pathway tables are interpretable.
Step 14: GSEA enrichment plots per signature gene.
Sidebar: Results Summary
Goal: Review the full pipeline narrative and key figures, then download exports where available.
What to do:
Success check: Summary reflects your completed steps; key plots from earlier tabs appear in the ordered overview.
Step 15: Results summary overview.
When you merge multiple GEO microarray series, row names are often probe IDs, Ensembl IDs, or GenBank accessions — not HGNC symbols. GExPipe detects the format per GSE and converts to symbols in STEP 2b before computing the gene intersection.
library(GExPipe)
detect_fmt <- getFromNamespace("detect_gene_id_format", "GExPipe")
need_conv <- getFromNamespace("gexpipe_ids_need_symbol_conversion", "GExPipe")
examples <- list(
"HGNC symbols" = c("TP53", "BRCA1", "EGFR"),
"Arraystar / custom probes" = c("(+)E1A_r60_1", "ASHGV40000001"),
"Affymetrix Ensembl probes" = c("ENSG00000000003_at", "ENSG00000000005_at"),
"GenBank accessions" = c("AB000409", "AB000463", "AB000781")
)
id_table <- data.frame(
Example = names(examples),
Detected_format = vapply(examples, detect_fmt, character(1)),
Needs_symbol_conversion = vapply(examples, need_conv, logical(1)),
check.names = FALSE
)
knitr::kable(id_table, caption = "How GExPipe classifies common GEO row ID types.")| Example | Detected_format | Needs_symbol_conversion | |
|---|---|---|---|
| HGNC symbols | HGNC symbols | Gene symbol (HGNC) | FALSE |
| Arraystar / custom probes | Arraystar / custom probes | Microarray probe-like ID | TRUE |
| Affymetrix Ensembl probes | Affymetrix Ensembl probes | Ensembl ID | TRUE |
| GenBank accessions | GenBank accessions | GenBank/EMBL accession | TRUE |
Arraystar GPL21827 (e.g. GSE188653) has
no gene-symbol column on GEO; GExPipe crosswalks V4 probe IDs to
GPL26963 annotation (ORF / accession → symbol). Control
probes such as (+)E1A_r60_* are dropped. Ensure
org.Hs.eg.db and biomaRt are installed for
full conversion.
Example platforms in a four-study microarray merge:
| GSE | GPL | Row ID style |
|---|---|---|
| GSE188653 | GPL21827 | Arraystar V4 probes (+)E1A_r60_*,
ASHGV… |
| GSE207304 | GPL26963 | Arraystar V5 probes ASHG19AP…V5 |
| GSE211729 | GPL30033 | Affymetrix Ensembl ENSG*_at |
| GSE92252 | GPL16025 | GenBank accessions AB000409 |
Demonstration with two small matrices (symbols only — no network):
genes <- c("TP53", "BRCA1", "EGFR", "MYC", "GAPDH", "ACTB")
set.seed(11)
m1 <- matrix(abs(rnorm(length(genes) * 4)), nrow = length(genes),
dimnames = list(genes, paste0("GSE1_S", 1:4)))
m2 <- matrix(abs(rnorm(length(genes) * 4)), nrow = length(genes),
dimnames = list(genes, paste0("GSE2_S", 1:4)))
overlap_out <- gexp_download_normalize_ids_for_overlap(
micro_expr_list = list(GSE1 = m1, GSE2 = m2),
rna_counts_list = list()
)
fin <- gexp_download_finalize_common_genes(
micro_expr_list = overlap_out$micro_expr_list,
rna_counts_list = overlap_out$rna_counts_list,
all_genes_list = overlap_out$all_genes_list
)
cat(
overlap_out$log_text,
"\nCommon genes after merge:", length(fin$common_genes), "\n"
)
#>
#> STEP 2b: Normalize IDs to gene symbols for overlap...
#>
#> Gene symbols extracted per GSE (format detected + sample):
#> GSE1: 6 genes; format: Gene symbol (HGNC); sample: 'TP53', 'BRCA1', 'EGFR', 'MYC', 'GAPDH', 'ACTB'
#> GSE2: 6 genes; format: Gene symbol (HGNC); sample: 'TP53', 'BRCA1', 'EGFR', 'MYC', 'GAPDH', 'ACTB'
#>
#> Common genes after merge: 6The Shiny app is the recommended interface. The functions below let you script normalization and QC plots using the example data bundled with the package.
library(ggplot2)
expr_tab <- read.csv(
system.file("extdata", "vignette_expression.csv", package = "GExPipe"),
check.names = FALSE, stringsAsFactors = FALSE
)
meta_tab <- read.csv(
system.file("extdata", "vignette_sample_metadata.csv", package = "GExPipe"),
check.names = FALSE, stringsAsFactors = FALSE
)
expr <- as.matrix(expr_tab[, -1L, drop = FALSE])
storage.mode(expr) <- "numeric"
rownames(expr) <- expr_tab[[1L]]
rownames(meta_tab) <- meta_tab$SampleID
meta_tab <- meta_tab[colnames(expr), , drop = FALSE]
dim(expr)
#> [1] 120 12
head(meta_tab)
#> SampleID Dataset Condition
#> S01 S01 D1 Normal
#> S02 S02 D1 Normal
#> S03 S03 D1 Disease
#> S04 S04 D1 Disease
#> S05 S05 D1 Disease
#> S06 S06 D1 Diseasegexp_normalize_and_intersect() normalizes each dataset,
intersects genes, and returns a combined matrix.
if (!"Dataset" %in% colnames(meta_tab))
meta_tab$Dataset <- "D1"
datasets <- unique(meta_tab$Dataset)
micro_expr_list <- setNames(
lapply(datasets, function(ds) {
ids <- rownames(meta_tab)[meta_tab$Dataset == ds]
expr[, ids, drop = FALSE]
}),
datasets
)
norm_res <- gexp_normalize_and_intersect(
micro_expr_list = micro_expr_list,
rna_counts_list = list(),
micro_norm_method = "quantile",
rnaseq_norm_method = "TMM",
de_method = "limma"
)
cat("Combined matrix :", paste(dim(norm_res$combined_expr), collapse = " x "), "\n")
#> Combined matrix : 120 x 12
cat("Common genes :", length(norm_res$common_genes), "\n")
#> Common genes : 120expr_pca <- norm_res$combined_expr
keep <- apply(expr_pca, 1L, var, na.rm = TRUE) > 1e-8
pca <- prcomp(t(expr_pca[keep, ]), center = TRUE, scale. = TRUE)
pca_df <- data.frame(
PC1 = pca$x[, 1L],
PC2 = pca$x[, 2L],
Dataset = meta_tab[rownames(pca$x), "Dataset"],
stringsAsFactors = FALSE
)
var_pct <- round(summary(pca)$importance[2L, 1L:2L] * 100, 1)
ggplot(pca_df, aes(PC1, PC2, colour = Dataset)) +
geom_point(size = 3) +
theme_minimal() +
labs(
title = "PCA - bundled example data",
subtitle = "After quantile normalization and gene intersection",
x = paste0("PC1 (", var_pct[1L], "% variance)"),
y = paste0("PC2 (", var_pct[2L], "% variance)")
)top_var <- head(
order(apply(norm_res$combined_expr, 1L, var, na.rm = TRUE),
decreasing = TRUE),
min(30L, nrow(norm_res$combined_expr))
)
if (length(top_var) > 1L) {
pheatmap::pheatmap(
norm_res$combined_expr[top_var, , drop = FALSE],
scale = "row",
show_rownames = length(top_var) <= 30L,
main = "Top 30 most variable genes - bundled example data"
)
}| Problem | Solution |
|---|---|
| 0 common genes after download | Use latest code. Check STEP 2b in the
log. Install org.Hs.eg.db and biomaRt. For
Arraystar (GSE188653), allow GPL26963 to download once
(cached under micro_data/). |
| IDs still probe-like in log | Restart R; use Run latest code — an old install skips STEP 2b fixes. |
Download log says
Code source: installed |
You are not on the GitHub clone; use
runGitHub(..., ref = "main") or
pkgload::load_all(). |
| Locked DLL files (Windows) | Restart R (Ctrl+Shift+F10) before installing or
launching. |
| Missing C++ compiler | Install Rtools (Windows). |
| glmnet / ML step skipped | Restart R after install; ensure glmnet ≥ 4.x or 5.x loads
(packageVersion("glmnet")). |
| Version conflicts | remotes::install_github("safarafique/GExPipe", ref = "main", force = TRUE, INSTALL_opts = "--no-staged-install"). |
| Corrupt lazy-load database | Reinstall with
INSTALL_opts = "--no-staged-install". |
| Corporate proxy | Sys.setenv(https_proxy = "http://proxy:port") before
installing. |
options(download.file.method = "wininet")
unlink(
list.files(.libPaths()[1], pattern = "^00LOCK-", full.names = TRUE),
recursive = TRUE, force = TRUE
)
if (!requireNamespace("remotes", quietly = TRUE))
install.packages("remotes")
remotes::install_github(
"safarafique/GExPipe",
ref = "main",
dependencies = TRUE,
INSTALL_opts = "--no-staged-install"
)sessionInfo()
#> R version 4.6.1 (2026-06-24)
#> Platform: x86_64-pc-linux-gnu
#> Running under: Ubuntu 26.04 LTS
#>
#> Matrix products: default
#> BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.32.so; LAPACK version 3.12.0
#>
#> locale:
#> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
#> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
#> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
#> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
#> [9] LC_ADDRESS=C LC_TELEPHONE=C
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
#>
#> time zone: Etc/UTC
#> tzcode source: system (glibc)
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] ggplot2_4.0.3 GExPipe_0.99.43 BiocStyle_2.41.0
#>
#> loaded via a namespace (and not attached):
#> [1] KEGGREST_1.53.5 gtable_0.3.6 xfun_0.60
#> [4] bslib_0.11.0 htmlwidgets_1.6.4 lattice_0.22-9
#> [7] Biobase_2.73.1 vctrs_0.7.3 tools_4.6.1
#> [10] generics_0.1.4 stats4_4.6.1 tibble_3.3.1
#> [13] AnnotationDbi_1.75.2 RSQLite_3.53.3 blob_1.3.0
#> [16] pkgconfig_2.0.3 BiocBaseUtils_1.15.1 pheatmap_1.0.13
#> [19] RColorBrewer_1.1-3 S7_0.2.2 S4Vectors_0.51.5
#> [22] VennDiagram_1.8.2 lifecycle_1.0.5 compiler_4.6.1
#> [25] farver_2.1.2 Biostrings_2.81.5 statmod_1.5.2
#> [28] Seqinfo_1.3.0 httpuv_1.6.17 htmltools_0.5.9
#> [31] sys_3.4.3 buildtools_1.0.0 sass_0.4.10
#> [34] yaml_2.3.12 later_1.4.8 pillar_1.11.1
#> [37] crayon_1.5.3 jquerylib_0.1.4 DT_0.34.0
#> [40] cachem_1.1.0 limma_3.69.2 org.Hs.eg.db_3.23.1
#> [43] mime_0.13 locfit_1.5-9.12 tidyselect_1.2.1
#> [46] digest_0.6.39 dplyr_1.2.1 labeling_0.4.3
#> [49] maketools_1.3.2 fastmap_1.2.0 grid_4.6.1
#> [52] cli_3.6.6 magrittr_2.0.5 edgeR_4.11.4
#> [55] withr_3.0.3 promises_1.5.0 scales_1.4.0
#> [58] bit64_4.8.2 rmarkdown_2.31 lambda.r_1.2.4
#> [61] XVector_0.53.0 httr_1.4.8 bit_4.6.0
#> [64] otel_0.2.0 futile.logger_1.4.9 png_0.1-9
#> [67] shiny_1.14.0 memoise_2.0.1 evaluate_1.0.5
#> [70] knitr_1.51 IRanges_2.47.2 rlang_1.3.0
#> [73] futile.options_1.0.1 Rcpp_1.1.2 xtable_1.8-8
#> [76] glue_1.8.1 DBI_1.3.0 BiocManager_1.30.27
#> [79] formatR_1.14 BiocGenerics_0.59.10 shinydashboard_0.7.3
#> [82] jsonlite_2.0.0 R6_2.6.1