---
title: "IntegratedLearner"
author: "Nalin Arora, Anupreet Porwal and Himel Mallick"
date: "2026-10-27"
output:
  BiocStyle::html_document:
    toc: true
    toc_float: true
vignette: >
  %\VignetteIndexEntry{IntegratedLearner}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE, warning=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

java_major_version <- function() {
  out <- tryCatch(
    suppressWarnings(system2("java", "-version", stdout = TRUE, stderr = TRUE)),
    error = function(e) character()
  )
  line1 <- out[1]
  if (length(line1) == 0 || is.na(line1)) return(NA_integer_)
  m <- regmatches(line1, regexec('"([0-9]+)(\\.[0-9]+)?', line1))[[1]]
  if (length(m) < 2) return(NA_integer_)
  as.integer(m[2])
}

has_java_vector_module <- function() {
  out <- tryCatch(
    suppressWarnings(system2("java", "--list-modules", stdout = TRUE, stderr = TRUE)),
    error = function(e) character()
  )
  any(grepl("^jdk.incubator.vector@", out))
}

# Set Java flags before any potential bartMachine/rJava namespace load.
java_params <- "-Xmx6G"
if (has_java_vector_module()) {
  java_params <- c(java_params, "--add-modules=jdk.incubator.vector")
}
options(java.parameters = java_params)

can_run_sl_bart <- function() {
  have_bart <- requireNamespace("bartMachine", quietly = TRUE)
  jv <- java_major_version()
  have_bart && !is.na(jv) && jv >= 21 && has_java_vector_module()
}

use_sl_bart <- isTRUE(getOption("IntegratedLearner.tutorial_bart", FALSE)) &&
  isTRUE(can_run_sl_bart())
```

## Introduction

This vignette is a practical tutorial for binary, multiclass, continuous, and survival outcome workflows in `IntegratedLearner`.

The examples use seven small MAE RDS files in `inst/extdata`, accessed directly
with `readRDS(system.file("extdata", filename, package = "IntegratedLearner"))`.
Each assay retains at most 25 variable features selected
from training data in original order; validation uses identical feature IDs.
All original samples and outcomes are kept. These subsets demonstrate the API;
they are not intended to reproduce full-cohort scientific results.

`IntegratedLearner` is designed for multi-omics studies where each assay layer can carry complementary signal and where users want a unified interface for standard integrated learning or standalone cooperative multiview learning. This makes the package a natural fit for Bioconductor workflows built around `MultiAssayExperiment`, `SummarizedExperiment`, and curated assay containers.

Within the Bioconductor ecosystem, the package is intended to sit after assay assembly and harmonization, and before downstream interpretation. IntegratedLearner focuses on supervised multi-omics prediction/integration rather than general container management or unsupervised integration.

### How this fits in a Bioconductor workflow

In a standard Bioconductor analysis, you would typically import or construct assay-level data, store those assays in a `MultiAssayExperiment`, inspect the object with standard accessors, fit `IntegratedLearner()`, and then carry predictions, layer weights, and feature-level summaries into downstream interpretation or visualization steps.

### Workflow at a Glance

A compact workflow is:

1. Construct aligned assays in `MultiAssayExperiment`.
2. Inspect experiments and metadata with `experiments()`, `assay()`, `colData()`, and `sampleMap()`.
3. Fit single-layer and fused predictive models with `IntegratedLearner()`.
4. Feed predictions, layer weights, and feature summaries into downstream Bioconductor interpretation workflows.

Compared with packages that primarily focus on assay containers, preprocessing, or narrower outcome families, `IntegratedLearner` is centered on the modeling layer itself: it provides a common interface for continuous, binary, multiclass, and survival prediction while remaining compatible with standard Bioconductor data structures. In that sense, it is intended to complement upstream Bioconductor tools rather than replace them.

The goal is to show a complete end-to-end pattern you can adapt to your own multi-omics study:

1. Build correctly formatted training/validation inputs.
2. Fit per-layer, stacked, and concatenated learners.
3. Interpret model outputs (AUC, accuracy, balanced accuracy, R2, survival discrimination curves, layer weights, and feature signals).

`IntegratedLearner` supports two execution modes:

- Standard IL mode: per-layer base learners, early fusion from concatenated features, and late fusion through a meta-learner.
- Intermediate/cooperative mode: standalone `multiview` cooperative learning over feature matrices from each layer.

Cooperative learning is enabled with `run_intermediate = TRUE`. In this mode, IntegratedLearner skips base learners, early fusion, late fusion, and supervised screening. It is currently supported for continuous, binary, and survival outcomes. For multiclass outcomes, `IntegratedLearner` prints a message and ignores `run_intermediate`.

Optional feature selection workflow used in this vignette:

1. Filtering first (`filter_method`, `filter_pct`) on training features.
2. Screening second (`run_screening = TRUE`, `screen_pct`) in a fold-safe manner:
   - selected on fold-training only,
   - applied to fold-validation,
   - repeated for each fold,
   - repeated once on full training data for final model fit.

## Installation

Install the release version from Bioconductor with:

```r
if (!requireNamespace("BiocManager", quietly = TRUE)) {
  install.packages("BiocManager")
}
BiocManager::install("IntegratedLearner")
```

For the development version, use:

```r
BiocManager::install("himelmallick/IntegratedLearner")
```

## Load Packages

```{r chunk-01, warning=FALSE, message=FALSE}
# Main package
library(IntegratedLearner)

# Tutorial dependencies
library(dplyr)
library(ggplot2)
library(SuperLearner)
library(caret)
library(cowplot)
library(bayesplot)
library(S4Vectors)
library(SummarizedExperiment)
library(MultiAssayExperiment)
library(survival)
if (requireNamespace("multiview", quietly = TRUE)) {
  library(multiview)
}
if (use_sl_bart) {
  library(bartMachine)
}
```

```{r chunk-02, include=FALSE}
if (!use_sl_bart) {
  message(
    "sl_bart sections are in fallback mode: Java >= 21 with bartMachine is required for sl_bart."
  )
}
```

## Input Data Contract

This vignette uses `MultiAssayExperiment` throughout and treats MAE as the primary user-facing entry point. The older PCL input mode is still supported for backward compatibility, but the recommended workflow is to stay inside standard Bioconductor containers for data inspection and model handoff.

For `IntegratedLearner`, each MAE example should satisfy:

- each omics layer is stored as a separate experiment, typically a `SummarizedExperiment`
- samples are aligned across experiments through MAE `colData`
- the outcome column lives in top-level `colData`
- the subject identifier column also lives in top-level `colData`

For survival workflows, `colData` should contain `time` and `event` columns (with `event` coded as `0/1`). You can also keep a convenience survival column such as `Y` or `outcome_surv` as a `Surv(time, event)` object.

You can keep your own column names and map them in the wrapper:

```r
fit <- IntegratedLearner(
  MAE_train = mae_train,
  outcome_col = "disease_status",
  subject_id_col = "participant_id",
  family = stats::binomial()
)
```

Automatic coercion in the wrapper:

- `family = gaussian()`: outcome is coerced to numeric (errors if conversion fails).
- binary `family = binomial()`: two classes are mapped internally to `{0,1}`.
- multiclass `family = binomial()`: class labels are retained.

## Standard Bioconductor Accessors

`IntegratedLearner` is intended to plug into a standard Bioconductor workflow,
not replace one. In practice, users should continue to inspect and manipulate
their inputs with the usual `MultiAssayExperiment` and `SummarizedExperiment`
accessors before modeling:

```{r chunk-bioc-accessors}
PRISM_MAE <- readRDS(system.file(
  "extdata", "PRISM_MAE.rds", package = "IntegratedLearner", mustWork = TRUE
))

mae_example <- PRISM_MAE

# Top-level sample annotations used by IntegratedLearner
head(as.data.frame(S4Vectors::DataFrame(MultiAssayExperiment::colData(mae_example)))[, 1:4])

# Available experiments/layers in the MAE
names(MultiAssayExperiment::experiments(mae_example))

# Pull one layer as a SummarizedExperiment
se_example <- MultiAssayExperiment::experiments(mae_example)[[1]]
class(se_example)

# Standard assay extraction from the layer
assay_names <- SummarizedExperiment::assayNames(se_example)
assay_names
head(SummarizedExperiment::assay(se_example, assay_names[1]))

# MAE sample-to-assay mapping
head(MultiAssayExperiment::sampleMap(mae_example))
```

This is the expected Bioconductor pipeline pattern:

1. Build assay-level objects as `SummarizedExperiment` instances.
2. Combine them into a `MultiAssayExperiment`.
3. Store outcome and subject identifiers in top-level `colData`.
4. Pass the MAE into `IntegratedLearner()` for layer-wise, early-fusion, and
   late-fusion modeling.

## Minimal MAE Workflow Example

The shortest end-to-end pattern starts from packaged MAE data, uses standard
Bioconductor accessors for inspection, and then fits `IntegratedLearner()`.

```{r chunk-mae-minimal, warning=FALSE}
PRISM_MAE <- readRDS(system.file(
  "extdata", "PRISM_MAE.rds", package = "IntegratedLearner", mustWork = TRUE
))

mae_min <- PRISM_MAE

names(MultiAssayExperiment::experiments(mae_min))
head(as.data.frame(S4Vectors::DataFrame(MultiAssayExperiment::colData(mae_min)))[, 1:4])
head(MultiAssayExperiment::sampleMap(mae_min))

se_min <- MultiAssayExperiment::experiments(mae_min)[[1]]
SummarizedExperiment::assayNames(se_min)
dim(SummarizedExperiment::assay(se_min, SummarizedExperiment::assayNames(se_min)[1]))

fit_min <- IntegratedLearner(
  MAE_train = mae_min,
  experiment = names(MultiAssayExperiment::experiments(mae_min))[1:2],
  assay.type = rep("abundance", 2),
  folds = 2,
  base_learner = "SL.mean",
  run_stacked = FALSE,
  run_concat = FALSE,
  print_learner = FALSE,
  family = stats::binomial()
)

names(fit_min)
```

## Parameter Reference (Conbin, Multiclass, and Survival)

### Common Wrapper Parameters (`IntegratedLearner`)

| Parameter | Default | Applies to | Description |
|---|---|---|---|
| `MAE_train`, `MAE_valid` | `NULL` | all | MAE-mode inputs (training and optional validation). |
| `PCL_train`, `PCL_valid` | `NULL` | all | Legacy tabular inputs retained for backward compatibility, but not used in this vignette. |
| `experiment` | `NULL` | MAE mode | Selected MAE experiment names/indices; defaults to all experiments. |
| `assay.type` | `NULL` | MAE mode | Assay names per selected experiment. |
| `outcome_col` | `"Y"` | all | Outcome column name in MAE `colData` (or tabular sample metadata, if using PCL mode). |
| `subject_id_col` | `"subjectID"` | all | Subject identifier column name in MAE `colData` (or tabular sample metadata, if using PCL mode). |
| `na.rm` | `FALSE` | all | Drop features with missing values after extraction/prep. |
| `folds` | `5` | all | Outer CV folds. |
| `seed` | `1234` | all | Reproducibility seed. |
| `base_learner` | `"sl_bart"` | standard IL mode | Base learner. Standard `SuperLearner` learners still use `SL.*` IDs for continuous/binary, while IntegratedLearner wrappers use snake_case IDs such as `sl_bart`; multiclass uses native IDs (for example `randomforest`, `xgboost`, `mbart`), and survival runs should use a supported `surv.*` ID. Not used when `run_intermediate = TRUE`. |
| `filter_method` | `NULL` | all | Optional feature filtering method: `"prevalence"` or `"variance"`. |
| `filter_pct` | `NULL` | all | Optional retention percentage in `(0,100]` for filtering. |
| `run_screening` | `FALSE` | standard IL mode | Enable supervised screening. Ignored when `run_intermediate = TRUE`. |
| `screen_pct` | `NULL` | all | Retention percentage in `(0,100]` for screening. Required when screening is enabled. |
| `prevalence_pct` | `NULL` | all | Deprecated alias for prevalence filtering (`filter_method = "prevalence"`). |
| `drop_poor_performing_layers` | `FALSE` | standard IL mode | If `TRUE`, removes layers with poor single-layer performance from early and late fusion only (AUC < 0.5 for binary, R² < 0.5 for continuous, C-index < 0.5 for survival). Single-layer results are still retained. Ignored when `run_intermediate = TRUE`. |
| `run_intermediate` | `FALSE` | continuous, binary, survival | Enables standalone direct cooperative learning through `multiview`. Base learners, early fusion, late fusion, and supervised screening are skipped. Multiclass fits print a message and ignore this option. Requires at least 3 CV folds. |
| `cooperative_rho` | `c(0, 0.1, 0.25, 0.5, 1)` | continuous, binary, survival | Candidate agreement-penalty values. `IntegratedLearner` fits one `cv.multiview()` model per `rho` using the same fold assignments, then keeps the best `rho` by CV score. |
| `cooperative_s` | `"lambda.min"` | continuous, binary, survival | Lambda selection used for cooperative predictions. Can be `"lambda.min"`, `"lambda.1se"`, or a numeric lambda. |
| `cooperative_type_measure` | `NULL` | continuous, binary, survival | Optional `multiview` CV metric. Defaults are MSE for gaussian, AUC for binomial, and Harrell's C for survival. |
| `verbose` | `FALSE` | all | Print progress. |
| `family` | `gaussian()` | all | Non-survival: `gaussian()`/`binomial()`. Multiclass is auto-detected when `family = binomial()` and outcome has more than two classes. Survival is auto-detected from metadata or family. |
| `...` | — | all | Passed to the selected backend (`IL_conbin` or `ILsurv`). |

### Conbin-Specific Parameters (Continuous/Binary Path)

| Parameter | Default | Description |
|---|---|---|
| `base_screener` | `"All"` | Deprecated compatibility parameter. Prefer `run_screening` + `screen_pct`. |
| `meta_learner` | `"sl_nnls_auc"` | Stacked meta learner for late fusion. |
| `run_stacked` | `TRUE` | Enables late-fusion stacked model. |
| `run_concat` | `TRUE` | Enables early-fusion concatenated model. |
| `print_learner` | `TRUE` | Prints fit summary. |
| `refit.stack` | `FALSE` | Refit stacked learner on full data for final predictions. |

### Multiclass-Specific Parameters (Native Multiclass Path)

| Parameter | Default | Description |
|---|---|---|
| `base_learner` | `"glmnet"` | Native multiclass learner per layer and for concatenated fit. Supported: `glmnet`, `randomforest`, `ranger`, `xgboost`, `mbart`, `multinom`. |
| `meta_learner` | `"glmnet"` | Native multiclass learner used for stacked fusion. |
| `base_screener` | `"All"` | Deprecated compatibility parameter. Prefer `run_screening` + `screen_pct`. |
| `run_stacked` | `TRUE` | Enables late-fusion stacked multiclass model. |
| `run_majority` | `TRUE` | Returns hard voting across layers, independently of `run_stacked`. Ties use the first training class. |
| `run_weighted` | `TRUE` | Returns learned probability averaging, independently of `run_stacked`. Nonnegative layer weights sum to one and minimize OOF multiclass log loss. |
| `run_concat` | `TRUE` | Enables early-fusion concatenated multiclass model. |
| `run_intermediate` | `FALSE` | Not supported for multiclass outcomes; if set to `TRUE`, a message is printed and the option is ignored. |
| `folds` | `5` | Subject-level CV folds for OOF multiclass probabilities. |
| `run_screening`, `screen_pct` | `FALSE`, `NULL` | Fold-safe multiclass screening (glmnet-based) after optional filtering. |

### Survival-Specific Parameters (via `...`)

| Parameter | Default | Description |
|---|---|---|
| `do_early_fusion` | `TRUE` | Train an early-fusion survival model on all features. |
| `t_vec`, `t_vec_probs` | `NULL`, quantiles | Time grid used in COX-style weighting summaries. |
| `layer_score` | `"sum"` | Aggregation of cumulative hazard increments (`sum`, `mean`, `l2`). |
| `weight_lambda` | `0.02` | Regularization strength for COX weighting optimizer. |
| `weight_penalty` | `"l2_to_uniform"` or `"entropy"` | Penalty used while learning survival late-fusion weights. |
| `weight_cap` | `1.0` | Optional cap on individual layer weights. |
| `optim_maxit_cox` | `4000` | Max iterations for COX weighting optimization. |
| `optim_maxit_ibs` | `300` | Max iterations for IBS weighting optimization. |
| `ibs_shrink_to_uniform` | `0` | Shrink IBS weights toward uniform blend. |

## Supported Models and Fusion Modules

### Supported Models

| Path | Supported base models |
|---|---|
| Continuous/Binary (`IL_conbin`) | Any `SuperLearner`-compatible learner available in your R session. Package wrappers include `sl_bart`, `sl_lasso`, `sl_enet`, `sl_glmnet2`, and `sl_horseshoe`, while standard SuperLearner learners such as `SL.glm` and `SL.randomForest` keep their original `SL.*` names. |
| Multiclass (`IL_multiclass`) | Native multiclass learner IDs: `glmnet`, `randomforest`, `ranger`, `xgboost`, `mbart`, `multinom`. |
| Survival (`ILsurv`) | Built-in survival learner IDs: `surv.coxph`, `surv.glmnet`, `surv.ranger`, `surv.ranger.extratrees`, `surv.ranger.maxstat`, `surv.ranger.C`, `surv.rfsrc`, `surv.coxboost`, `surv.gbm`, `surv.xgboost.cox`, `surv.xgboost.aft`, `surv.mboost`, `surv.bart`. |

### Supported Fusion Outputs

| Path | Single-layer | Early fusion | Late fusion | Cooperative/intermediate fusion |
|---|---|---|---|---|
| Continuous/Binary | Yes in standard IL mode | `run_concat = TRUE` in standard IL mode | `run_stacked = TRUE` with `meta_learner` in standard IL mode | Standalone `run_intermediate = TRUE` via `multiview` |
| Multiclass | Yes | `run_concat = TRUE` | Stacking, majority voting, and learned probability averaging, all enabled by default with independent flags | Not supported; prints a message and is ignored |
| Survival | Yes (`train_out$single`) in standard IL mode | `do_early_fusion = TRUE` in standard IL mode | Weighted layer blending with both `IBS` and `COX` outputs returned in standard IL mode | Standalone `run_intermediate = TRUE` via `multiview` Cox model |

## Output Reference: What You Get and How to Access It

This section summarizes the outputs produced by each integration method and where to find weights/importance values.

### Conbin Outputs (Binary/Continuous)

| Method | What it returns | Where to access |
|---|---|---|
| Single-layer (per omics layer) | Layer-specific predictions and metrics | `fit$yhat.train[, layer_name]`, `fit$yhat.test[, layer_name]` (if validation), `fit$AUC.train` / `fit$AUC.test`, `fit$accuracy.train` / `fit$accuracy.test`, `fit$balanced_accuracy.train` / `fit$balanced_accuracy.test` (binomial), `fit$R2.train` / `fit$R2.test` (gaussian) |
| Early fusion (concatenated) | One model on all features concatenated | Enable with `run_concat = TRUE`; outputs in `fit$yhat.train[, "concatenated"]`, `fit$model_fits$model_concat`, `fit$SL_fits$SL_fit_concat` |
| Late fusion (stacked) | Meta-model over layer-level predictions | Enable with `run_stacked = TRUE`; outputs in `fit$yhat.train[, "stacked"]`, `fit$model_fits$model_stacked`, `fit$SL_fits$SL_fit_stacked` |
| Cooperative/intermediate fusion | Standalone cooperative `multiview` model trained on layer feature matrices | Enable with `run_intermediate = TRUE`; outputs in `fit$yhat.train[, "cooperative"]`, `fit$yhat.test[, "cooperative"]`, `fit$model_fits$model_cooperative`, `fit$cooperative_rho`, `fit$cooperative_rho_scores`, and coefficient-based cooperative importance fields. No base learner, early-fusion, or late-fusion objects are returned in this mode. |
| Layer weights (stacked) | Relative contribution of each layer in late fusion | `fit$weights` (available when `meta_learner = "sl_nnls_auc"` and `run_stacked = TRUE`) |
| Binary metric table | Per-model AUC, accuracy, and balanced accuracy | `fit$metrics.train` and `fit$metrics.test` (if validation provided) |

### Multiclass Outputs

| Method | What it returns | Where to access |
|---|---|---|
| Single-layer (per omics layer) | Layer-wise multiclass probability and class predictions | `fit$prob.train[[layer_name]]`, `fit$class.train[, layer_name]`, plus validation analogs `fit$prob.test[[layer_name]]`, `fit$class.test[, layer_name]` |
| Early fusion (concatenated) | One multiclass model on concatenated features | Enable with `run_concat = TRUE`; outputs in `fit$prob.train$concatenated`, `fit$class.train[, "concatenated"]`, `fit$model_fits$model_concat` |
| Late fusion (stacked) | Multiclass meta-model over OOF layer probabilities | Enable with `run_stacked = TRUE`; outputs in `fit$prob.train$stacked`, `fit$class.train[, "stacked"]`, `fit$model_fits$model_stacked` |
| Late fusion (majority) | Per-class vote fractions and the class with the most votes | Enable with `run_majority = TRUE`; `fit$prob.train$majority`, `fit$class.train$majority`, `fit$train_out$late$majority` |
| Late fusion (learned average) | Probability average with nonnegative layer weights summing to one | Enable with `run_weighted = TRUE`; `fit$prob.train$weighted_average`, `fit$class.train$weighted_average`, `fit$train_out$late$weighted_average` |
| Grouped results | Survival-style `single`, `early`, and `late` entries for all enabled methods | `fit$train_out`, `fit$valid_out`; `fit$late_methods` lists the late-fusion methods |
| Learned layer weights | Final weights and weights fitted outside each training fold | `fit$model_fits$model_weighted$weights`, `fit$train_out$late$weighted_average$weights_by_fold` |
| Cooperative/intermediate fusion | Not supported for multiclass outcomes | `run_intermediate = TRUE` prints a message and is ignored; no cooperative probability object is returned |
| Multiclass performance metrics | Accuracy, balanced accuracy, one-vs-rest AUC, and log-loss | `fit$metrics.train` and `fit$metrics.test` (if validation provided) |
| Feature-selection metadata | Filtering/screening settings used in fit | `fit$filter_method`, `fit$filter_pct`, `fit$prevalence_pct`, `fit$screening_used`, `fit$screen_method`, `fit$screen_pct` |
| Screened feature sets | Features retained by fold-safe screening | `fit$selected_features_by_layer`, `fit$selected_features_concat` |

Every grouped fusion entry contains `train_prob` and `train_class` (or
`valid_prob` and `valid_class`) and a `metrics` table. Scalar metrics are also
available as `train_accuracy`, `train_balanced_accuracy`, `train_auc`, and
`train_logloss`, with analogous `valid_` names for validation. Single-layer
entries hold probability lists and class data frames covering all layers.
Prediction returns the same `valid_out` structure; without outcome labels,
predictions remain available, `metrics` is `NULL`, and scalar metrics are absent.
Without validation data, `fit$valid_out` is `NULL`. Disabled early fusion gives
`early = NULL`; disabled late methods are omitted from `late`. Flat outputs
remain available for compatibility.

Both the final stack and the final learned average train on OOF layer
probabilities. Training predictions for the average fit weights outside each
held-out fold. These procedures do not replace fully nested evaluation of the
complete pipeline; use an untouched validation set or outer CV for assessment.
Majority vote fractions are not calibrated probabilities, so its log loss should
be interpreted accordingly. Both layer-level and final voting ties use the
first class in `fit$class_levels`.

### Survival Outputs (Single/Early/Late)

| Method | Training outputs | Validation outputs |
|---|---|---|
| Single-layer | `fit$train_out$single$metrics`, `fit$train_out$single$train_risk` | `fit$valid_out$single$valid_cindex`, `fit$valid_out$single$valid_auc`, `fit$valid_out$single$valid_risk` |
| Early fusion | `fit$train_out$early$train_cindex`, `fit$train_out$early$train_auc`, `fit$train_out$early$train_risk` | `fit$valid_out$early$valid_cindex`, `fit$valid_out$early$valid_auc`, `fit$valid_out$early$valid_risk` |
| Late fusion | `fit$train_out$late$IBS` and `fit$train_out$late$COX`, each containing weights, C-index, time-dependent AUC, and risks | `fit$valid_out$late$IBS` and `fit$valid_out$late$COX`, with the analogous validation metrics |
| Cooperative/intermediate fusion | Standalone `fit$train_out$cooperative`, containing the selected `multiview` model, C-index, time-dependent AUC, risk scores, and coefficient-based feature-importance scores | `fit$valid_out$cooperative`, with analogous validation metrics and risk scores. `single`, `early`, and `late` outputs are `NULL` in this mode. |
| Survival plotting payload | `fit$surv_plot_data$train` | `fit$surv_plot_data$valid` |

### Importance Outputs (Conbin, Multiclass, and Survival)

| Importance type | Where to access | Notes |
|---|---|---|
| Conbin signed global feature importance | `fit$feature_importance_signed` | Returned for standard continuous/binary fits; named numeric vector sorted by effect magnitude/sign. |
| Conbin signed per-layer importance | `fit$feature_importance_signed_by_layer` | List split by `featureType`. |
| Cooperative absolute importance | `fit$cooperative_feature_importance` and `fit$cooperative_feature_importance_by_layer` | Absolute selected-model `multiview` coefficients; available when `run_intermediate = TRUE`. |
| Cooperative signed importance | `fit$cooperative_feature_importance_signed` and `fit$cooperative_feature_importance_signed_by_layer` | Signed selected-model `multiview` coefficients; available when `run_intermediate = TRUE`. |
| Multiclass global feature importance (unsigned) | `fit$feature_importance_global` | Named vector of mean absolute class-association scores, ranked highest first. |
| Multiclass signed importance by class | `fit$feature_importance_signed_by_class` | Feature-by-class matrix of signed class-association scores. |
| Multiclass signed importance by layer and class | `fit$feature_importance_signed_by_layer_by_class` | List of feature-by-class matrices, one per omics layer; features ranked by mean absolute score across classes. |
| Survival signed per-layer importance | `fit$train_out$single$feature_importance_signed_by_layer` | List of unweighted signed feature-importance scores from each single-layer survival base model. |
| Cooperative survival importance | `fit$train_out$cooperative$feature_importance` and `fit$train_out$cooperative$feature_importance_by_layer` | Absolute selected-model `multiview` Cox coefficients; available when `run_intermediate = TRUE`. |
| Cooperative survival signed importance | `fit$train_out$cooperative$feature_importance_signed` and `fit$train_out$cooperative$feature_importance_signed_by_layer` | Signed selected-model `multiview` Cox coefficients; available when `run_intermediate = TRUE`. |
| Survival early-fusion combined importance | `fit$train_out$early$combined_importance` | Available when `do_early_fusion = TRUE`. |
| Survival late-fusion combined importance | `fit$train_out$late$IBS$combined_importance` or `fit$train_out$late$COX$combined_importance` | Weighted signed importance; names are prefixed like `layer::feature`. |
| BART-specific layer importance (optional) | `bartMachine::investigate_var_importance(fit$model_fits$model_layers[[layer]], plot = FALSE)` | Only for BART-based conbin fits (`base_learner = "sl_bart"`). |

### Quick Access Snippets

```r
# ---- Conbin: weights + top features ----
fit$weights
fit$cooperative_rho
fit$cooperative_rho_scores
head(fit$yhat.train[, "cooperative"], 10)
head(fit$cooperative_feature_importance, 20)
head(fit$cooperative_feature_importance_by_layer[[1]], 20)
head(fit$feature_importance_signed, 20)
names(fit$feature_importance_signed_by_layer)
head(fit$feature_importance_signed_by_layer[[1]], 20)

# ---- Multiclass: metrics + class probabilities + importance ----
fit_mc$metrics.train
fit_mc$metrics.test
head(fit_mc$class.train)
head(fit_mc$class.test)
head(fit_mc$prob.train$stacked)
head(fit_mc$feature_importance_global, 20)
head(fit_mc$feature_importance_signed_by_class[, 1, drop = FALSE], 20)
head(fit_mc$feature_importance_signed_by_layer_by_class[[1]], 20)

# ---- Survival: late-fusion weights + top combined features ----
fit_surv$train_out$late$IBS$weights
fit_surv$train_out$late$COX$weights
fit_surv$train_out$cooperative$train_cindex
fit_surv$cooperative_rho
head(fit_surv$train_out$cooperative$feature_importance, 20)
head(fit_surv$train_out$cooperative$feature_importance_by_layer[[1]], 20)
names(fit_surv$train_out$single$feature_importance_signed_by_layer)
head(fit_surv$train_out$single$feature_importance_signed_by_layer[[1]], 20)
head(fit_surv$train_out$late$IBS$combined_importance, 20)
head(fit_surv$train_out$late$COX$combined_importance, 20)

# ---- Survival: inspect all fusion branches ----
fit_surv$train_out$single
fit_surv$train_out$early
fit_surv$train_out$late
```

## Example 1: Binary Outcome (IBD Classification)

This section uses the packaged PRISM and NLIBD `MultiAssayExperiment` fixtures for classifying IBD status.

### Step 1: Load and Inspect Training and Validation Data

```{r chunk-05}
PRISM_MAE <- readRDS(system.file(
  "extdata", "PRISM_MAE.rds", package = "IntegratedLearner", mustWork = TRUE
))
NLIBD_MAE <- readRDS(system.file(
  "extdata", "NLIBD_MAE.rds", package = "IntegratedLearner", mustWork = TRUE
))
mae_train <- PRISM_MAE
mae_valid <- NLIBD_MAE

head(MultiAssayExperiment::colData(mae_train))
names(MultiAssayExperiment::experiments(mae_train))
table(MultiAssayExperiment::colData(mae_train)$Y)
```

### Step 2: Inspect Layer Assays

```{r chunk-06}
vapply(
  MultiAssayExperiment::experiments(mae_train),
  function(se) nrow(SummarizedExperiment::assay(se, "abundance")),
  integer(1)
)
```

### Step 3: Fit the Model

`IntegratedLearner` fits one model per layer (`base_learner`) and then combines layer-level predictions with a meta-learner (`meta_learner`).

```{r chunk-07, warning=FALSE}
fit <- IntegratedLearner(
  MAE_train = mae_train,
  MAE_valid = mae_valid,
  folds = 2,
  base_learner = "SL.randomForest",
  meta_learner = "sl_nnls_auc",
  filter_method = "prevalence",
  filter_pct = 40,
  run_screening = TRUE,
  screen_pct = 30,
  verbose = TRUE,
  family = binomial()
)
```

### Step 4: Inspect and Interpret Outputs

Core outputs for binary tasks include:

- `fit$AUC.train` and `fit$AUC.test`: AUC per layer and fusion model.
- `fit$accuracy.train` and `fit$accuracy.test`: thresholded accuracy per layer and fusion model.
- `fit$balanced_accuracy.train` and `fit$balanced_accuracy.test`: balanced accuracy per layer and fusion model.
- `fit$metrics.train` and `fit$metrics.test`: compact metric tables with AUC, accuracy, and balanced accuracy.
- `fit$weights`: layer contributions in the stacked model (when `sl_nnls_auc` is used).
- `fit$yhat.train` and `fit$yhat.test`: predicted probabilities.

```{r chunk-08}
fit$AUC.train
fit$AUC.test
fit$accuracy.train
fit$balanced_accuracy.train
fit$metrics.test
fit$weights
```

Plot ROC summaries for train and validation sets:

```{r chunk-09, warning=FALSE, fig.height=10, fig.width=7}
plot.obj <- IntegratedLearner:::plot.learner(fit)
plot.obj$plot
```

In this PRISM setting, you can compare which single layer is strongest and whether stacked fusion outperforms both individual layers and simple concatenation.

### Optional: Direct Cooperative Learning with `multiview`

Set `run_intermediate = TRUE` to run a standalone direct cooperative multiview model. This branch uses the original layer feature matrices as views, fits `cv.multiview()` once per candidate `rho` using the same fold assignments, and keeps the `rho` with the best CV score. Within each `rho`, `cv.multiview()` selects the lambda path in the usual way; predictions use `cooperative_s`, which defaults to `"lambda.min"`.

The chunk below is not evaluated during vignette builds because `multiview` fits can be expensive on real omics matrices. It can be run directly in an analysis session.

```{r chunk-09b, warning=FALSE, message=FALSE, eval=FALSE}
fit_coop <- IntegratedLearner(
  MAE_train = mae_train,
  MAE_valid = mae_valid,
  experiment = names(MultiAssayExperiment::experiments(mae_train))[1:2],
  assay.type = rep("abundance", 2),
  folds = 3,
  run_intermediate = TRUE,
  cooperative_rho = c(0, 0.25, 1),
  cooperative_s = "lambda.min",
  filter_method = "prevalence",
  filter_pct = 20,
  print_learner = FALSE,
  family = binomial()
)

fit_coop$cooperative_rho
fit_coop$cooperative_rho_scores
fit_coop$metrics.train
fit_coop$metrics.test
head(fit_coop$yhat.test[, "cooperative"])
head(fit_coop$cooperative_feature_importance, 20)
head(fit_coop$cooperative_feature_importance_by_layer[[1]], 20)
```

## Example 2: Continuous Outcome (Gestational Age)

This section uses the pregnancy dataset (Ghaemi et al., 2019), converted into `MultiAssayExperiment` format. Here `Y` is continuous gestational age.

### Step 1: Load and Inspect Data

```{r chunk-10}
pregnancy_MAE <- readRDS(system.file(
  "extdata", "pregnancy_MAE.rds", package = "IntegratedLearner", mustWork = TRUE
))
mae_train <- pregnancy_MAE

head(MultiAssayExperiment::colData(mae_train))
names(MultiAssayExperiment::experiments(mae_train))
length(unique(MultiAssayExperiment::colData(mae_train)$subjectID))
```

### Step 2: Fit Continuous Model

The tutorial uses the non-Java `SL.randomForest` learner by default. To run
optional BART and uncertainty examples, set
`options(IntegratedLearner.tutorial_bart = TRUE)` before rendering; these require
a compatible Java installation.

If you hit:

`java.lang.UnsupportedClassVersionError ... class file version 65.0 ... recognizes up to 61.0`

your Java runtime is older than the version used by your installed `bartMachine` build (typically Java 17 runtime vs Java 21 bytecode). In that case, either:

- upgrade Java runtime to 21 and restart R, or
- use a non-Java learner (example fallback shown below).

```{r chunk-12, warning=FALSE, eval=use_sl_bart}
options(java.parameters = c("-Xmx6G", "--add-modules=jdk.incubator.vector"))

fit <- IntegratedLearner(
  MAE_train = mae_train,
  folds = 2,
  base_learner = "sl_bart",
  meta_learner = "sl_nnls_auc",
  filter_method = "variance",
  filter_pct = 40,
  run_screening = TRUE,
  screen_pct = 30,
  family = gaussian()
)
```

Fallback (non-Java) run:

```{r chunk-13, eval=!use_sl_bart}
fit <- IntegratedLearner(
  MAE_train = mae_train,
  folds = 2,
  base_learner = "SL.randomForest",
  meta_learner = "sl_nnls_auc",
  filter_method = "variance",
  filter_pct = 40,
  run_screening = TRUE,
  screen_pct = 30,
  family = gaussian()
)
```

### Step 3: Evaluate Predictive Accuracy

For continuous outcomes, `IntegratedLearner` reports `R2.train` (and `R2.test` if validation is provided).

```{r chunk-14}
fit$R2.train
```

```{r chunk-21, fig.asp=0.8, fig.width=7}
plot.obj <- IntegratedLearner:::plot.learner(fit)
plot.obj$plot
```

### Step 4: Uncertainty and Feature-Level Interpretation (BART)

When using `sl_bart`, you can inspect posterior predictive distributions and derive weighted posterior summaries.
`r if (!use_sl_bart) "This section is skipped in this session because Java < 21 (or bartMachine is unavailable)." else ""`

```{r chunk-15, eval=use_sl_bart}
weights <- fit$weights

dataX <- fit$X_train_layers
dataY <- fit$Y_train

post.samples <- vector("list", length(weights))
names(post.samples) <- names(dataX)

for (i in seq_along(post.samples)) {
  post.samples[[i]] <- bartMachine::bart_machine_get_posterior(
    fit$model_fits$model_layers[[i]],
    dataX[[i]]
  )$y_hat_posterior_samples
}

weighted.post.samples <- Reduce("+", Map("*", post.samples, weights))
rownames(weighted.post.samples) <- rownames(dataX[[1]])
names(dataY) <- rownames(dataX[[1]])
```

Visualize 68% and 95% credible intervals for observations:

```{r chunk-22, fig.width=9, fig.height=7, eval=use_sl_bart}
ord_names <- names(sort(rowMeans(weighted.post.samples), decreasing = TRUE))

mcmc_intervals(t(weighted.post.samples), prob = 0.68, prob_outer = 0.95) +
  scale_y_discrete(limits = ord_names) +
  geom_point(aes(x = dataY[ord_names], y = ord_names), shape = 1, size = 3, color = "black") +
  coord_flip() +
  theme_bw() +
  labs(
    x = "Gestational age (in months)",
    y = "Observations"
  ) +
  theme(axis.text.x = element_text(angle = 90, hjust = 1))
```

Layer weights and feature-level inclusion proportions can also be examined for biological interpretation.

```{r chunk-16, eval=use_sl_bart}
omicsEye_theme <- function() {
  angle <- 45
  hjust <- 1
  ggplot2::theme_bw() +
    ggplot2::theme(
      axis.text.x = ggplot2::element_text(size = 8, vjust = 1, hjust = hjust, angle = angle),
      axis.text.y = ggplot2::element_text(size = 8, hjust = 1),
      axis.title = ggplot2::element_text(size = 10),
      plot.title = ggplot2::element_text(size = 10),
      plot.subtitle = ggplot2::element_text(size = 8),
      legend.title = ggplot2::element_text(size = 6, face = "bold"),
      legend.text = ggplot2::element_text(size = 7),
      axis.line = ggplot2::element_line(colour = "black", linewidth = 0.25),
      axis.line.x = ggplot2::element_line(colour = "black", linewidth = 0.25),
      axis.line.y = ggplot2::element_line(colour = "black", linewidth = 0.25),
      panel.border = ggplot2::element_blank(),
      panel.grid.major = ggplot2::element_blank(),
      panel.grid.minor = ggplot2::element_blank()
    )
}

safe_var_importance <- function(model, layer_label) {
  tryCatch({
    qq <- bartMachine::investigate_var_importance(model, plot = FALSE)
    df <- cbind.data.frame(qq$avg_var_props, qq$sd_var_props)
    colnames(df) <- c("mean", "sd")
    df$type <- layer_label
    df
  }, error = function(e) {
    warning(sprintf("Skipping variable importance for %s: %s", layer_label, conditionMessage(e)))
    data.frame(mean = numeric(), sd = numeric(), type = character())
  })
}

vimp_stack <- cbind.data.frame(fit$weights)
colnames(vimp_stack) <- "mean"
vimp_stack$sd <- NA
vimp_stack$type <- "stack"

layer_names <- names(fit$model_fits$model_layers)
vimp_layers <- lapply(layer_names, function(layer_nm) {
  safe_var_importance(fit$model_fits$model_layers[[layer_nm]], layer_nm)
})

vimp_layers <- vimp_layers[lengths(vimp_layers) > 0]
vimp_top <- do.call(
  rbind,
  lapply(vimp_layers, function(df) head(df[order(-df$mean), , drop = FALSE], 20))
)

VIMP <- as.data.frame(rbind.data.frame(vimp_stack, vimp_top))
VIMP <- tibble::rownames_to_column(VIMP, "ID")

p4 <- VIMP %>%
  dplyr::filter(type == "stack") %>%
  dplyr::arrange(desc(mean)) %>%
  ggplot(aes(y = mean, x = reorder(ID, -mean))) +
  geom_bar(stat = "identity", fill = "darkseagreen") +
  theme_bw() +
  omicsEye_theme() +
  ylab("Layer Weights") +
  xlab("")

p5 <- VIMP %>%
  dplyr::filter(type != "stack") %>%
  dplyr::arrange(mean) %>%
  dplyr::mutate(ID = gsub("_", " ", ID, fixed = TRUE)) %>%
  ggplot(aes(reorder(ID, -mean), mean, fill = type)) +
  facet_wrap(. ~ type, scales = "free") +
  geom_bar(stat = "identity", fill = "lightsalmon") +
  geom_errorbar(aes(ymin = ifelse(mean - sd > 0, mean - sd, 0), ymax = mean + sd),
                width = 0.2,
                position = position_dodge(0.9)) +
  theme_bw() +
  coord_flip() +
  omicsEye_theme() +
  theme(strip.background = element_blank()) +
  ylab("Inclusion proportion") +
  xlab("")
```

```{r chunk-23, fig.width=7, fig.height=4, eval=use_sl_bart}
plot_grid(
  p4,
  ncol = 1,
  labels = c("Estimated IntegratedLearner Layer Weights"),
  label_size = 8,
  vjust = 0.1
) + theme(plot.margin = unit(c(0.5, 0.5, 0.5, 0.5), "cm"))
```

```{r chunk-24, fig.width=9, fig.height=7, eval=use_sl_bart}
plot_grid(
  p5,
  ncol = 1,
  labels = c("Top Features by Layer (BART Inclusion Proportions)"),
  label_size = 8,
  vjust = 0.1
) + theme(plot.margin = unit(c(0.5, 0.5, 0.5, 0.5), "cm"))
```

## Example 3: Multiclass Outcome (Franzosa MAE with External Validation)

This section shows a full multiclass MAE workflow using packaged local fixtures.
Here we keep the original outcome column name (`diseaseCat`) and subject ID
column (`sample_id`) and pass them through `outcome_col` and
`subject_id_col`.

The small multiclass example uses a ranger base model and returns multinomial
stacking, majority voting, and learned probability averaging together.
Supervised screening is already demonstrated above; it is disabled here because
the fixture has only 25 features per assay.

```{r chunk-17}
Franzosa_MAE_train <- readRDS(system.file(
  "extdata", "Franzosa_MAE_train.rds", package = "IntegratedLearner", mustWork = TRUE
))
Franzosa_MAE_valid <- readRDS(system.file(
  "extdata", "Franzosa_MAE_valid.rds", package = "IntegratedLearner", mustWork = TRUE
))

MAE_train <- Franzosa_MAE_train
MAE_valid <- Franzosa_MAE_valid

fit <- IntegratedLearner::IntegratedLearner(
  MAE_train = MAE_train,
  MAE_valid = MAE_valid,
  experiment = c("metabolome", "species"),
  assay.type = c("abundance", "abundance"),
  outcome_col = "diseaseCat",
  subject_id_col = "sample_id",
  family = stats::binomial(),
  base_learner = "ranger",
  meta_learner = "multinom",
  run_stacked = TRUE,
  run_concat = TRUE,
  run_majority = TRUE,
  run_weighted = TRUE,
  filter_method = "variance",
  filter_pct = 50,
  run_screening = FALSE,
  folds = 2,
  verbose = TRUE
)

```

Useful multiclass outputs:

- `fit$train_out$single`, `fit$train_out$early`, `fit$train_out$late`
- `fit$valid_out$single`, `fit$valid_out$early`, `fit$valid_out$late`
- `fit$late_methods`
- `fit$metrics.train`
- `fit$metrics.test`
- `fit$class.train`
- `fit$class.test`
- `fit$prob.train`
- `fit$prob.test`
- `fit$feature_importance_signed_by_class`
- `fit$feature_importance_signed_by_layer_by_class`
- `fit$feature_importance_global`
- `fit$filter_method`, `fit$filter_pct`
- `fit$screening_used`, `fit$screen_pct`

The multiclass metric tables now report accuracy, balanced accuracy, one-vs-rest AUC, and log-loss. The plotting helper also returns a single one-vs-rest ROC figure with all class curves overlaid for each fitted model.

Multiclass feature importance retains the existing univariate calculation:
for each feature and class, subtract the mean in the other classes from the
mean in that class, then divide by the feature's standard deviation across all
training samples (with a small numerical stabilizer). Positive scores indicate
higher values in the class; negative scores indicate lower values. The global
ranking averages the absolute scores across classes and is therefore unsigned.
The per-layer output groups the same signed feature-by-class matrices by omics
layer; it does not compute a single importance value for an entire layer.
These scores use features remaining after optional filtering, including features
excluded by supervised screening. They are shared across the fitted models and
describe class associations rather than model-specific prediction attribution.

```{r chunk-17a}
fit$late_methods
lapply(fit$valid_out$late, function(result) result$metrics)
fit$train_out$late$weighted_average$weights
fit$train_out$late$weighted_average$weights_by_fold
head(fit$valid_out$late$majority$valid_class)
head(fit$valid_out$late$weighted_average$valid_prob)
head(fit$feature_importance_signed_by_class[, 1, drop = FALSE])
head(fit$feature_importance_signed_by_layer_by_class[[1]])
head(fit$feature_importance_global)
```

```{r chunk-17b, warning=FALSE, fig.height=9, fig.width=8}
plot.obj.mc <- IntegratedLearner:::plot.learner(fit)
plot.obj.mc$plot
```

## Example 4: Survival Outcome (Time-to-event)

For survival tasks, `IntegratedLearner` dispatches to `ILsurv` when survival metadata are detected. The expected fields are:

- `time`: follow-up time (non-negative).
- `event`: event indicator (`0/1`).
- optional `Y`: `Surv(time, event)` convenience column.

This path uses the package-native survival backend (no `mlr3` dependency required).

Survival probabilities are calibrated using only training outcomes and OOF
risks. The resulting Cox calibration model, including baseline survival and
missing-risk replacement, is held fixed for validation. When calibration is
not estimable, the training Kaplan-Meier fallback is also held fixed.
Validation outcomes enter metric computation and IPCW censoring-weight
estimation, but never prediction calibration. IBS fusion Brier scores use the
actual weighted survival curves rather than recalibrated scalar risk summaries.
All models are scored on the same training-derived `fit_surv_mae$brier_time_grid`
for Brier/IBS, separately from the event-time AUC grid. Survival matrices are
available as `train_surv` and `valid_surv` within the single, early, late, or
cooperative output entries, and fitted calibration objects are stored in
`survival_calibration`. Training probability-error summaries are internal
diagnostics; a complete outer evaluation or independent validation is still
required to assess the full fitted pipeline.

For plotting, the survival backend now stores:

- a time-dependent AUC table evaluated over a denser event-time grid (rather than only a few summary quantiles), and
- Kaplan-Meier curve payloads built from predicted risk groups for the best fused survival model.

This section provides a complete MAE workflow.

```{r chunk-18}
TCGA_survival_MAE_train <- readRDS(system.file(
  "extdata", "TCGA_survival_MAE_train.rds", package = "IntegratedLearner", mustWork = TRUE
))
TCGA_survival_MAE_valid <- readRDS(system.file(
  "extdata", "TCGA_survival_MAE_valid.rds", package = "IntegratedLearner", mustWork = TRUE
))

mae_train <- TCGA_survival_MAE_train
mae_valid <- TCGA_survival_MAE_valid

fit_surv_mae <- IntegratedLearner(
  MAE_train = mae_train,
  MAE_valid = mae_valid,
  experiment = c("gene", "mirna"),
  assay.type = c("abundance", "abundance"),
  outcome_col = "outcome_surv",
  subject_id_col = "patient_id",
  folds = 2,
  base_learner = "surv.coxph",
  filter_method = "variance",
  filter_pct = 40,
  run_screening = TRUE,
  screen_pct = 25,
  verbose = TRUE
)
```

To run standalone direct cooperative survival modeling, use at least 3 folds and set the cooperative options:

```{r chunk-18b, eval=FALSE}
fit_surv_coop <- IntegratedLearner(
  MAE_train = mae_train,
  MAE_valid = mae_valid,
  experiment = c("gene", "mirna"),
  assay.type = c("abundance", "abundance"),
  outcome_col = "outcome_surv",
  subject_id_col = "patient_id",
  folds = 3,
  filter_method = "variance",
  filter_pct = 40,
  run_screening = TRUE,
  screen_pct = 25,
  run_intermediate = TRUE,
  cooperative_rho = c(0, 0.25, 1),
  cooperative_s = "lambda.min",
  verbose = TRUE
)
```




### Interpret Survival Outputs

```{r chunk-19}
# Single-layer training metrics
# fit_surv_mae$train_out$single$metrics

# Training-derived probability-evaluation grid and directly scored IBS curves
fit_surv_mae$brier_time_grid
head(fit_surv_mae$valid_out$late$IBS$valid_surv)
fit_surv_mae$valid_out$late$IBS$valid_ibs

# Late fusion (two weighted integration summaries)
fit_surv_mae$train_out$late$IBS$weights
fit_surv_mae$train_out$late$IBS$train_cindex
fit_surv_mae$train_out$late$IBS$train_auc
fit_surv_mae$train_out$late$COX$weights
fit_surv_mae$train_out$late$COX$train_cindex
fit_surv_mae$train_out$late$COX$train_auc

# Early fusion summary
# fit_surv_mae$train_out$early
# Validation metrics
fit_surv_mae$valid_out$late$IBS$valid_cindex
fit_surv_mae$valid_out$late$IBS$valid_auc
fit_surv_mae$valid_out$late$COX$valid_cindex
fit_surv_mae$valid_out$late$COX$valid_auc
fit_surv_mae$valid_out$single$valid_cindex

# Cooperative survival branch, if run_intermediate = TRUE
# fit_surv_coop$cooperative_rho
# fit_surv_coop$cooperative_rho_scores
# fit_surv_coop$train_out$cooperative$train_cindex
# fit_surv_coop$valid_out$cooperative$valid_cindex
# head(fit_surv_coop$train_out$cooperative$feature_importance, 20)
# head(fit_surv_coop$train_out$cooperative$feature_importance_by_layer[[1]], 20)
```

The `train_auc` and `valid_auc` objects for both `IBS` and `COX` are data frames with `time` and `AUC` columns, so they can be plotted as true time-dependent discrimination curves. In practice, these two late-fusion outputs provide complementary summaries over the same single-layer and early-fusion backbone, making side-by-side comparison possible within one fit object.

```{r chunk-19b, warning=FALSE, fig.height=9, fig.width=8}
plot.obj.surv <- IntegratedLearner:::plot.learner(fit_surv_mae)
plot.obj.surv$plot
```

## Session Information

```{r chunk-20}
sessionInfo()
```

## References

Franzosa EA et al. (2019).
[Gut microbiome structure and metabolic activity in inflammatory bowel disease](
https://pubmed.ncbi.nlm.nih.gov/30531976/
).
*Nature Microbiology* 4(2):293-305.

Ghaemi MS et al. (2019).
[Multiomics modeling of the immunome, transcriptome, microbiome, proteome and
metabolome adaptations during human pregnancy](
https://pubmed.ncbi.nlm.nih.gov/30561547/
).
*Bioinformatics* 35(1):95-103.

## Citation

Mallick et al. (2024).
[An integrated Bayesian framework for multi-omics prediction and
classification](
https://pubmed.ncbi.nlm.nih.gov/38146838/
).
*Statistics in Medicine* 43(5):983-1002.