Run the Web Application with Docker (Optional)

Alternatively, if docker is installed, the DiscoRhythm container on Docker Hub can be pulled and used to run the web application locally ⁴ avoiding the need to install DiscoRhythm and its dependencies

Cosinor

Cornelissen (2014)

“Conceived as a regression problem, the method is applicable to non-equidistant data, a major advantage.”

and referring to least squares techniques more generally:

“useful in curve-fitting problems, where it is desirable to obtain a functional form that best fits a given set of measurements.”

JTK Cycle

Hughes, Hogenesch, and Kornacker (2010)

“JTK_CYCLE’s increased resistance to outliers results in considerably greater sensitivity and specificity”

Lomb-Scargle

Glynn, Chen, and Mushegian (2006)

“Our approach should be applicable for detection and quantification of periodic patterns in any unevenly spaced gene expression time-series data.”

ARSER

Yang and Su (2010)

“ARSER was superior to two existing and widely-used methods, COSOPT and Fisher’s G-test, during identification of sinusoidal and non-sinusoidal periodic patterns in short, noisy and non-stationary time-series”

Processed Metadata Table

The Time and Replicate Id data extracted from the column names will be seen throughout the DiscoRhythm application ⁸ as:

Note: If there is more relevant metadata in the experiment that does not fit into the design specified in this section, it may be appropriate to split the dataset into several parts and use each subgroup as a separate input for DiscoRhythm. Example: If the experiment includes multiple conditions, each condition may be an independent input dataset for DiscoRhythm.

Specify Dataset Parameters

Time Type - See @ref(circular-and-linear-time).

Period of Interest - The main hypothesized period should be specified in order to set appropriate defaults throughout the application. If unknown, set it to the range of the sample collection times.

Time Unit/Observation Unit - Units to display in the axis labels throughout the application.

If the “Sampling Summary” table does not seem to accurately reflect the data, please refer back to section @ref(column-names). It is also a good idea to expand the “Inspect Input Data Matrix” and “Inspect Parsed Metadata” boxes to ensure the data has been read correctly.

In “Inspect Parsed Metadata” it is possible to override the metadata extracted from the column names with an independent CSV table which matches the metadata table format outlined in @ref(processed-metadata-table), however, this is not recommended for regular use.

Inter-sample Correlation

Purpose: In order to detect outliers, samples are pairwise correlated using either the Pearson or Spearman method.

Heatmap: The values of these pairwise correlations can be visualized in this tab, where samples with similar correlation values are grouped together using clustering.

Outlier Detection: The average correlation value for each sample is used as a metric of its overall similarity to all other samples and is summarized in this figure.

Samples with average correlations significantly below the mean will be flagged as outliers and the user may specify a number of standard deviations below the mean to use as a threshold.

Screenshot of the ‘Inter-sample Correlation’ section of the DiscoRhythm interface.

Principal Component Analysis

Purpose: Utilize principal component analysis (PCA) to detect outliers.

PCA is used to extract the strongest recurring patterns in the dataset. Outliers detected in these patterns (PC scores) are flagged by their deviation from the mean where again the user may specify a threshold in units of standard deviations.

Scale Before PCA: Whether to scale rows to a standard deviation of 1 prior to PCA, such that all rows are on an equal scale.

PCs to Use For Outlier Detection: Click to change the list of PCs to use for outlier removal in the case the scores of a PC seem inappropriate for use in outlier detection. You can remove unwanted PCs by pressing “delete” and add extra ones by typing their number.

Before CSV/After CSV: Downloadable summaries of the PCA before and after the detected outliers are removed.

Figures:

Distributions: The distributions of PC scores used to detect outliers. Only the PCs colored darkly are used for outlier detection. Outliers will be shown with an ‘x’.

Scree: PCs are numbered, where the amount of variance explained by each PC (their ‘importance’) decreases with increasing PC number. This can be seen in the “Scree” figure. Users should choose an appropriate number of PCs to use for outlier detection by the shape of this scree plot.

One Pair and All Pairs: Plotting the PC scores of the components versus one another may reveal grouping that cannot be determined from simple analysis of individual PCs.

Screenshot of the ‘PCA’ section of the DiscoRhythm interface.

Filtering Summary

Purpose: Determine how to proceed with outlier removal.

The user may, at this point, choose to remove the flagged outliers or may disregard these flags if it is suspected the dissimilarity of these samples may be biologically relevant. The user may also manually remove samples they deem unreliable for further analysis.

Raw Distributions: A boxplot for the values of each individual sample that can be used to further evaluate sample selection.

Input and Final: Shows summary tables for data before and after outlier removal.

Screenshot of the ‘Filtering Summary’ section of the DiscoRhythm interface.

Period Detection

Available periods will be limited ¹⁰ starting from a smallest period of 3 times the sampling-interval up to the sampling duration. 12 periods, evenly spaced across this range, will be used for period detection.

Screenshot of the ‘Period Detection’ section of the DiscoRhythm interface.

PC Fits

To obtain an abstract representation of the rhythmic patterns seen across the dataset, PCA is performed to reduce dimensionality and test the summarized temporal signal for rhythmicity (using the Cosinor method). Using an a priori hypothesized period, a significant fit on a single PC indicates the presence of few phases ¹¹ in the dataset, while strong fits to multiple PCs suggests that there exist multiple phases of oscillation. Oscillations detected among the first principal components would be more meaningful than oscillations in the last components. When a period is selected based on period detection results, more caution should be taken as the reliability of the findings might suffer due to selection bias and overfitting.

Screenshot of the ‘PC Cosinor Fits’ section of the DiscoRhythm interface.

Inspection of the patterns seen in the principal component scores may hint at other effects in the data that one should be aware of (e.g. batch effects or unobserved confounders).

Using Results to Proceed

The two statistical summaries of the periodicity provided in the period detection section may be used to proceed with the analysis in “Oscillation Detection” in multiple ways. Three scenarios are provided below to aid in illustrating some approaches for utilizing the results. The aim often should be to proceed with a hypothesis driven analysis; however, data driven exploratory approaches may also be used.

1. An exact periodicity is hypothesized. The goal is to determine which features fit this hypothesis.

   • Use period detection for data summary purposes only.
   • Use PC fits to summarize dominant rhythms and inspect for any odd patterns.
   • Proceed to oscillation detection using the hypothesized periodicity to detect rhythmic features.

2. An approximate periodicity is hypothesized, such as when subjects are in free running conditions. The goal is to determine which features fit this approximate hypothesis.

   • Use period detection to find an accurate estimate of the period.
   • Use the dominant period in oscillation detection to detect feature-wise rhythms.
   • Proceed to oscillation detection with the strongest period seen in period detection. Some caution should be taken for an increased false positive rate due to the period tuning procedure.

3. The period is entirely unknown. The goal may be to find a dominant periodicity with a secondary goal of identifying which features follow this period.

   • Use period detection to find a dominant period (period with highest median r-squared).
   • Use PC fits to test for a dominant periodic pattern and inspect for other patterns.
   • Proceed to oscillation detection with the dominant period to determine presence of feature-wise rhythms. Caution should be taken in these findings as the selection of period using the data will not be taken into account during feature-wise oscillation detection and inferences will have an increased false positive rate. Validation datasets are recommended to confirm feature-wise rhythms.

Rhythmicity Calculation Configuration

The user must choose a single period ¹² of oscillation to test across all rows of the dataset. The application may show warnings/messages regarding the choice of period. By default, the period specified in the ‘Select Data’ section will be chosen.

JTK Cycle, Lomb-Scargle, and ARSER results are all obtained through the MetaCycle R package (meta2d function using minper=maxper). Cosinor is provided by DiscoRhythm. A brief summary of each method:

Cosinor - a.k.a “Harmonic Regression” Fits a sinusoid with a free phase parameter.
JTK Cycle - non-parametric test of rhythmicity robust to outliers.
Lomb-Scargle - an approach using spectral power density.
ARSER - removes linear trends and performs the Cosinor test.

Exclusion Criteria Matrix: A table is presented describing the criteria that exclude a method from being used. All exclusion criteria which are true for the loaded dataset are shown ¹³. The reasons for exclusion may be due to either computational (causes errors under given conditions) or statistical restrictions (requirements of study design) of the method.

Screenshot of the ‘Oscillation Detection (Preview)’ section of the DiscoRhythm interface.

Job Submission Modes

Two modes of execution are available for oscillation detection:

1. Interactive
2. Report

On the public server only, email features are available for receiving notifications and results.

Visualizing Results

Once rhythmicity computation is completed, 3 sections become available for inspecting the results:

Individual Models: Allows inspection of individual rows from the raw dataset. User may click a row on the table in order to display the raw data values along with a fitted periodic curve. If the Cosinor method is being viewed, the line will be the Cosinor fit, all other methods utilize a loess fit. If the error bar option is selected, a 95% confidence interval on the mean will be displayed for each timepoint.

Summary: Summarizes calculated rhythm parameters across all tested rows by all executed methods.

Method Comparison: Offers pairwise comparison of rhythmic parameters calculated by each method to determine the degree of agreement between them.

Screenshot of the ‘Oscillation Detection’ section of the DiscoRhythm interface.

Note on method comparison: This section should not be used as evidence of rhythmicity, and rather should be used for testing purposes with simulated datasets to determine the degree of agreement of the detection methods under various conditions. For real datasets, this section could be used to compare the results of each method to determine which features are sensitive to method choice.

discoDFtoSE

The CSV inputs to the DiscoRhythm web interface described in section @ref(input-format) can be read into R as a data.frame. To allow the users of the web application to use the same input for their analysis in R, the function discoDFtoSE() can convert the tabular input into an apporpriate format described in @ref(data-import) above.

The sample metadata will be extracted from the column names by matching the format ¹⁶ described in section @ref(column-names). The checks for validity and uniqueness mentioned in section @ref(select-data) will also be performed. Alternatively, this metadata may be provided directly to discoDFtoSE as a data.frame.

Loading in the example dataset described in section @ref(input-format) using discoGetSimu() to get the demo dataset as a data.frame:

library(DiscoRhythm)
indata <- discoGetSimu()
knitr::kable(head(indata[,1:6]), format = "markdown") # Inspect the data

And importing as a SummarizedExperiment.

se <- discoDFtoSE(indata)

discoSEtoDF

The reverse operation, discoSEtoDF, is also available and is mainly intended for exporting the data to a “csv” format to be used as an input to the web application.

write.csv(discoSEtoDF(se),file = "DiscoRhythmInputFile.csv",row.names = FALSE)

discoCheckInput

This function performs the row-wise checks for missing values and constant values as mentioned in section @ref(select-data).

selectDataSE <- discoCheckInput(se)

discoDesignSummary

The sample collection information present in colData(selectDataSE) can be summarized by the discoDesignSummary() function to detail the number of biological and technical replicates available at each collection time. Number of technical replicates is shown in brackets.

library(SummarizedExperiment)
Metadata <- colData(selectDataSE)
knitr::kable(discoDesignSummary(Metadata),format = "markdown")

discoInterCorOutliers

Performs the analysis described in @ref(inter-sample-correlation) to return some intermediate results and a vector indicating which samples were flagged as outliers using the inter-sample correlation method.

CorRes <- discoInterCorOutliers(selectDataSE,
                                cor_method="pearson",
                                threshold=3,
                                thresh_type="sd")

discoPCAoutliers

Performs the analysis described in @ref(principal-component-analysis) to return some intermediate results and a vector indicating which samples were flagged as outliers using the PCA method.

PCAres <- discoPCAoutliers(selectDataSE,
                           threshold=3,
                           scale=TRUE,
                           pcToCut = c("PC1","PC2","PC3","PC4"))

discoPCA

A simple wrapper was written for the stats::prcomp function for better use with the web application and it can be utilized as:

discoPCAres <- discoPCA(selectDataSE)

This returns the same output as prcomp with the addition of a reformatted summary table (available as PCAresAfter$table).

Filtering Summary

The results of the outlier detection analysis (CorRes and PCAres) are used to remove outliers by subsetting the data.

FilteredSE <- selectDataSE[,!PCAres$outliers & !CorRes$outliers]

DT::datatable(as.data.frame(
  colData(selectDataSE)[PCAres$outliers | CorRes$outliers,]
))

knitr::kable(discoDesignSummary(colData(FilteredSE)),format = "markdown")

discoRepAnalysis

Performs the analysis described in @ref(row-selection) returning the results of the ANOVA test and the se data object where technical replicates are combined.

ANOVAres <- discoRepAnalysis(FilteredSE,
                             aov_method="Equal Variance",
                             aov_pcut=0.05,
                             aov_Fcut=1,
                             avg_method="Median")

FinalSE <- ANOVAres$se

discoPeriodDetection

Performs the analysis described in @ref(period-detection) to return a data.frame of Cosinor results across a range of periods.

PeriodRes <- discoPeriodDetection(FinalSE,
                                  timeType="linear",
                                  main_per=24)

The main period of interest is fit using a Cosinor model to principal component scores as described in @ref(period-detection).

OVpca <- discoPCA(FinalSE)
OVpcaSE <- discoDFtoSE(data.frame("PC"=1:ncol(OVpca$x),t(OVpca$x)),
                                  colData(FinalSE))
knitr::kable(discoODAs(OVpcaSE,period = 24,method = "CS")$CS,
             format = "markdown")

discoODAs

Performs the analysis described in @ref(rhythmicity-calculation-configuration) using just the Cosinor method. discoODAs() will automatically run all appropraite methods if none are provided.

discoODAres <- discoODAs(FinalSE,
                         period=24,
                         method="CS",
                         ncores=1,
                         circular_t=FALSE)

discoBatch

The entire analysis performed in section @ref(discorhythm-r-usage) may be run through a single call to discoBatch() to obtain the final discoODAres results.

discoBatch(indata=indata,
  report="discoBatch_example.html",
  ncores=1,
  main_per=24,
  timeType="linear",
  cor_threshold=3,
  cor_method="pearson",
  cor_threshType="sd",
  pca_threshold=3,
  pca_scale=TRUE,
  pca_pcToCut=paste0("PC",seq_len(4)),
  aov_method="None",
  aov_pcut=0.05,
  aov_Fcut=0,
  avg_method="Median",
  osc_method="CS",
  osc_period=24)

This command will generate an html report called “discoBatch_example.html” , which includes the visualizations seen in the DiscoRhythm application. indata may be in either of the two input formats described in @ref(data-import) (data.frame or SummarizedExperiment).

Importing Archived Sessions

Section @ref(report) describes how to archive results in a DiscoRhythm web session. The RDS file produced by this feature contains the parameters used in the session which can be used by one of the methods below to continue an analysis in R.

First the CSV input dataset can be read into R. This may be done as:

indata <- read.csv(path_to_csv_file)

And the R data file can be read into R with:

discorhythm_inputs <- readRDS('discorhythm_inputs.RDS')

discoODAres <- do.call(discoBatch, c(list(indata=indata,
                                          report='discorhythm_report.html'),
                                          discorhythm_inputs))

list2env(discorhythm_inputs,envir=.GlobalEnv)

######################################################################
# Intended for use by discoBatch or through the DiscoRhythm_report.Rmd
# Includes all R code for the DiscoRhythm data processing
# Expects all arguments to discoBatch in the environment
#####################################################################

library(DiscoRhythm)

# Preprocess inputs
selectDataSE <- discoCheckInput(discoDFtoSE(indata))

# Intersample correlations
CorRes <- discoInterCorOutliers(selectDataSE,cor_method,
                                cor_threshold,cor_threshType)

# PCA for outlier detection
PCAres <- discoPCAoutliers(selectDataSE,pca_threshold,pca_scale,pca_pcToCut)
PCAresAfter <- discoPCA(selectDataSE[,!PCAres$outliers])

# Removing the outliers from the main data.frame and metadata data.frame
FilteredSE <- selectDataSE[,!PCAres$outliers & !CorRes$outliers]

# Running ANOVA and merging replicates
ANOVAres <- discoRepAnalysis(FilteredSE, aov_method,
                             aov_pcut, aov_Fcut, avg_method)

# Data to be used for Period Detection and Oscillation Detection
FinalSE <- ANOVAres$se

# Perform PCA on the final dataset
OVpca <- discoPCA(FinalSE)

# Period Detection
PeriodRes <- discoPeriodDetection(FinalSE,
                     timeType,
                     main_per)

# Oscillation Detection
discoODAres <- discoODAs(FinalSE,
                        circular_t = timeType=="circular",
                        period=osc_period,
                        osc_method,ncores)