---
title: "6. Case study: small cohorts"
author: "Abdelkarim Hani Ghrieb"
date: "`r Sys.Date()`"
output:
  BiocStyle::html_document:
    toc: true
    toc_float: true
vignette: >
  %\VignetteIndexEntry{6. Case study: small cohorts}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r small-cohort-setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

# Status note

This is a synthetic-data walkthrough demonstrating how the
`events_per_parameter` guardrail and fold-level reporting expose unstable
signatures. It is a vignette, not a study. The narrative follows the
*three-tier* structure used across the case studies: a **naive analysis**
first, then **rnaSentry standing guard**, then the **counterfactual**.

# Setup

We build a 30-sample cohort with 23 events and a real (if weak) survival
signal driven by five genes, then ask for a 5-gene signature. This is 4.6
events per parameter -- below the guardrail of 5.

```{r build-signature, warning = TRUE}
library(rnaSentry)
library(SummarizedExperiment)

set.seed(30)
n_sm <- 30
counts <- matrix(rpois(60 * n_sm, lambda = 500), nrow = 60, ncol = n_sm,
                 dimnames = list(paste0("gene", 1:60), paste0("S", 1:n_sm)))
sig_expr <- colMeans(counts[1:5, , drop = FALSE])
risk <- scale(sig_expr)[, 1] * 0.5
event_time <- rexp(n_sm, rate = 0.05 * exp(0.9 * risk))
censor_time <- rexp(n_sm, rate = 0.02)
cd <- S4Vectors::DataFrame(time = pmin(event_time, censor_time),
                           event = as.integer(event_time < censor_time),
                           row.names = colnames(counts))
se <- SummarizedExperiment(assays = list(counts = counts), colData = cd)

sig <- build_signature(se, "time", "event", top_n = 5,
                       repeats = 5, folds = 5, seed = 30)
```

# Tier 1: the naive analysis

A standard analysis reports the single number the summary returns and stops
there:

```{r naive-summary}
sig$cv_summary
```

The mean cross-validated concordance is 0.74 -- which a careless reader would
call "decent." A naive manuscript would report *"cross-validated C = 0.74"*
as evidence that the signature is prognostic, with no sense of how much that
single mean is hiding.

# Tier 2: rnaSentry standing guard

## The guardrail fires

```{r guardrail-flags}
sig$flags
```

The `events_per_parameter` warning fires immediately -- 23 events for 5 genes
is 4.6 per parameter, below the guardrail of 5 -- and is recorded in the
stage's flag ledger so it propagates into the rendered report.

## The mean hides the spread

```{r per-fold-results}
sig$cv_results
```

The per-fold concordances range from **0.50 to 1.00** across repeats and
folds. With only a handful of held-out samples per fold, single-fold C
estimates are nearly coin flips, and the standard deviation (0.14) is
enormous relative to the mean. A signature like this is unstable and should
be treated as hypothesis-generating only.

```{r small-fold-spread, fig.width = 7, fig.height = 5, fig.align = "center", fig.cap = "Per-fold cross-validated concordance across repeats and folds (one point per fold). The vertical dashed line marks the mean; the spread from 0.50 to 1.00 is the instability the mean hides."}
set.seed(30)
cv <- sig$cv_results
stripchart(cv$c_index, method = "jitter", jitter = 0.12, pch = 19, cex = 1.1,
           ylim = c(0, 1.05), ylab = "Cross-validated C-index (per fold)",
           main = sprintf("Per-fold C-index: mean %.2f, sd %.2f",
                          mean(cv$c_index), stats::sd(cv$c_index)))
abline(v = mean(cv$c_index), lty = 2, col = "grey40")
```

# Tier 3: the counterfactual

| | Naive analysis (Tier 1) | rnaSentry (Tier 2) |
|---|---|---|
| Reported statistic | mean CV C = 0.74, quoted alone | mean + fold-level distribution (range 0.50-1.00, sd 0.14) |
| Events per parameter | not computed | 4.6 < 5, `events_per_parameter` warning in the ledger |
| Scientific status | "a decent prognostic signature" | "hypothesis-generating only; unstable" |

**What a guarded analysis can claim:** the exact same cross-validated
concordance that a naive analysis would headline as 0.74 is shown by
`build_signature()` to be an average over near-coin-flip folds -- and the
guardrail warns rather than silently proceeding. This is the documented
"small-cohort trap," quantified in our simulation analysis
(`validation/simulate_study.R`): with fewer than ~5 events per signature
gene, neither the concordance nor the hazard ratios are trustworthy, and a
*mean* C-index hides the instability that the fold-level table exposes.

# Reading

The takeaway is exactly the documented "small-cohort trap": with fewer than
~5 events per signature gene, neither the concordance nor the hazard ratios
are trustworthy. This is why `build_signature()` reports the per-fold
distribution, not just the mean, and why the guardrail warns rather than
silently proceeding.

# Session info

```{r session-info}
sessionInfo()
```