Introduction_1_loadmetadata

Progenetix is an open data resource that provides curated individual cancer copy number variation (CNV) profiles along with associated metadata sourced from published oncogenomic studies and various data repositories. This vignette provides a comprehensive guide on accessing and utilizing metadata for samples or their corresponding individuals within the Progenetix database.

If your focus lies in cancer cell lines, you can access data from cancercelllines.org by setting the domain parameter to “https://cancercelllines.org” in pgxLoader function. This data repository originates from CNV profiling data of cell lines initially collected as part of Progenetix and currently includes additional types of genomic mutations.

Load library

library(pgxRpi)

pgxLoader function

This function loads various data from Progenetix database via the Beacon v2 API with some extensions (BeaconPlus).

The parameters of this function used in this tutorial:

  • type: A string specifying output data type. “biosamples”, “individuals”, “analyses”, and “sample_count” are used in this tutorial.
  • filters: Identifiers used in public repositories, bio-ontology terms, or custom terms such as c(“NCIT:C7376”, “PMID:22824167”). When multiple filters are used, they are combined using AND logic when the parameter type is “biosamples”, “individuals”, or “analyses”; OR logic when the parameter type is “sample_count”.
  • biosample_id: Identifiers used in the query database for identifying biosamples.
  • individual_id: Identifiers used in the query database for identifying individuals.
  • codematches: A logical value determining whether to exclude samples from child concepts of specified filters in the ontology tree. If TRUE, only samples exactly matching the specified filters will be included. Do not use this parameter when filters include ontology-irrelevant filters such as PMID and cohort identifiers. Default is FALSE.
  • limit: Integer to specify the number of returned profiles. Default is 0 (return all).
  • skip: Integer to specify the number of skipped profiles. E.g. if skip = 2, limit=500, the first 2*500 =1000 profiles are skipped and the next 500 profiles are returned. Default is NULL (no skip).
  • dataset: A string specifying the dataset to query from the Beacon response. Default is NULL, which includes results from all datasets.
  • domain: A string specifying the domain of the query data resource. Default is “http://progenetix.org”.
  • entry_point: A string specifying the entry point of the Beacon v2 API. Default is “beacon”, resulting in the endpoint being “http://progenetix.org/beacon”.

Retrieve biosamples information

Search by filters

Filters are a significant enhancement to the Beacon query API, providing a mechanism for specifying rules to select records based on their field values. To learn more about how to utilize filters in Progenetix, please refer to the documentation.

The pgxFilter function helps access available filters used in Progenetix by default. It is also possible to query available filters used in other resources via the Beacon v2 API by setting the domain and entry_point parameters accordingly. Here is an example usage:

# access all filters
all_filters <- pgxFilter()
head(all_filters)
#> [1] "PATO:0020000"               "PATO:0020001"              
#> [3] "PATO:0020002"               "EDAM:operation_3227"       
#> [5] "EDAM:operation_3961"        "labelSeg-based calibration"
# get all prefix
all_prefix <- pgxFilter(return_all_prefix = TRUE)
all_prefix
#>  [1] "PATO"                       "EDAM"                      
#>  [3] "labelSeg-based calibration" "NCIT"                      
#>  [5] "EFO"                        "OBI"                       
#>  [7] "geo"                        "arrayexpress"              
#>  [9] "UBERON"                     "cbioportal"                
#> [11] "cellosaurus"                "pgx:icdom"                 
#> [13] "pgx:icdot"                  "pgx:TCGA"                  
#> [15] "pgx:cohort"
# access specific filters based on prefix
ncit_filters <- pgxFilter(prefix="NCIT")
head(ncit_filters)
#> [1] "NCIT:C28076" "NCIT:C18000" "NCIT:C14158" "NCIT:C14161" "NCIT:C14167"
#> [6] "NCIT:C28077"

The following query retrieves metadata in Progenetix related to all samples of retinoblastoma, utilizing a specific filter based on an NCIt code as a disease identifier.

biosamples <- pgxLoader(type="biosamples", filters = "NCIT:C7541")
# data looks like this
biosamples[1:5,]
#>     biosample_id   individual_id biosample_status_id biosample_status_label
#> 1 pgxbs-kftvh1n1 pgxind-kftx2vtw         EFO:0009656      neoplastic sample
#> 2 pgxbs-kftvh1n3 pgxind-kftx2vty         EFO:0009656      neoplastic sample
#> 3 pgxbs-kftvh1n4 pgxind-kftx2vu0         EFO:0009656      neoplastic sample
#> 4 pgxbs-kftvh1n6 pgxind-kftx2vu2         EFO:0009656      neoplastic sample
#> 5 pgxbs-kftvh1n8 pgxind-kftx2vu4         EFO:0009656      neoplastic sample
#>   sample_origin_type_id sample_origin_type_label histological_diagnosis_id
#> 1           OBI:0001479   specimen from organism                NCIT:C7541
#> 2           OBI:0001479   specimen from organism                NCIT:C7541
#> 3           OBI:0001479   specimen from organism                NCIT:C7541
#> 4           OBI:0001479   specimen from organism                NCIT:C7541
#> 5           OBI:0001479   specimen from organism                NCIT:C7541
#>   histological_diagnosis_label sampled_tissue_id sampled_tissue_label
#> 1               Retinoblastoma    UBERON:0000966               retina
#> 2               Retinoblastoma    UBERON:0000966               retina
#> 3               Retinoblastoma    UBERON:0000966               retina
#> 4               Retinoblastoma    UBERON:0000966               retina
#> 5               Retinoblastoma    UBERON:0000966               retina
#>   pathological_stage_id pathological_stage_label tnm_id tnm_label
#> 1           NCIT:C92207            Stage Unknown     NA        NA
#> 2           NCIT:C92207            Stage Unknown     NA        NA
#> 3           NCIT:C92207            Stage Unknown     NA        NA
#> 4           NCIT:C92207            Stage Unknown     NA        NA
#> 5           NCIT:C92207            Stage Unknown     NA        NA
#>   tumor_grade_id tumor_grade_label age_iso          notes icdo_morphology_id
#> 1             NA                NA    <NA> Retinoblastoma    pgx:icdom-95103
#> 2             NA                NA    <NA> Retinoblastoma    pgx:icdom-95103
#> 3             NA                NA    <NA> Retinoblastoma    pgx:icdom-95103
#> 4             NA                NA    <NA> Retinoblastoma    pgx:icdom-95103
#> 5             NA                NA    <NA> Retinoblastoma    pgx:icdom-95103
#>   icdo_morphology_label icdo_topography_id icdo_topography_label     pubmed_id
#> 1   Retinoblastoma, NOS    pgx:icdot-C69.2                Retina PMID:15834944
#> 2   Retinoblastoma, NOS    pgx:icdot-C69.2                Retina PMID:15834944
#> 3   Retinoblastoma, NOS    pgx:icdot-C69.2                Retina PMID:15834944
#> 4   Retinoblastoma, NOS    pgx:icdot-C69.2                Retina PMID:15834944
#> 5   Retinoblastoma, NOS    pgx:icdot-C69.2                Retina PMID:15834944
#>   cellosaurus_id cbioportal_id tcga_project_id analysis_info_experiment_id
#> 1           <NA>          <NA>              NA                        <NA>
#> 2           <NA>          <NA>              NA                        <NA>
#> 3           <NA>          <NA>              NA                        <NA>
#> 4           <NA>          <NA>              NA                        <NA>
#> 5           <NA>          <NA>              NA                        <NA>
#>   analysis_info_series_id analysis_info_platform_id                cohort_ids
#> 1                    <NA>                      <NA> pgx:cohort-2021progenetix
#> 2                    <NA>                      <NA> pgx:cohort-2021progenetix
#> 3                    <NA>                      <NA> pgx:cohort-2021progenetix
#> 4                    <NA>                      <NA> pgx:cohort-2021progenetix
#> 5                    <NA>                      <NA> pgx:cohort-2021progenetix
#>   biosample_legacy_id geoprov_city geoprov_country geoprov_iso_alpha3
#> 1                  NA   Heidelberg         Germany                DEU
#> 2                  NA   Heidelberg         Germany                DEU
#> 3                  NA   Heidelberg         Germany                DEU
#> 4                  NA   Heidelberg         Germany                DEU
#> 5                  NA   Heidelberg         Germany                DEU
#>   geoprov_long_latitude geoprov_long_longitude                    updated
#> 1                 49.41                   8.69 2020-09-10 17:44:29.148000
#> 2                 49.41                   8.69 2020-09-10 17:44:29.150000
#> 3                 49.41                   8.69 2020-09-10 17:44:29.151000
#> 4                 49.41                   8.69 2020-09-10 17:44:29.152000
#> 5                 49.41                   8.69 2020-09-10 17:44:29.154000

The data contains many columns representing different aspects of sample information.

Search by biosample id and individual id

In Progenetix, biosample id and individual id serve as unique identifiers for biosamples and the corresponding individuals. You can obtain these IDs through metadata search with filters as described above, or through website interface query.

biosamples_2 <- pgxLoader(type="biosamples", biosample_id = "pgxbs-kftvki7h",individual_id = "pgxind-kftx6ltu")

biosamples_2
#>     biosample_id   individual_id biosample_status_id biosample_status_label
#> 1 pgxbs-kftvki7h pgxind-kftx6ltd         EFO:0009656      neoplastic sample
#> 2 pgxbs-kftvki7v pgxind-kftx6ltu         EFO:0009656      neoplastic sample
#>   sample_origin_type_id sample_origin_type_label histological_diagnosis_id
#> 1           OBI:0001479   specimen from organism                NCIT:C3512
#> 2           OBI:0001479   specimen from organism                NCIT:C3512
#>   histological_diagnosis_label sampled_tissue_id sampled_tissue_label
#> 1          Lung Adenocarcinoma    UBERON:0002048                 lung
#> 2          Lung Adenocarcinoma    UBERON:0002048                 lung
#>   pathological_stage_id pathological_stage_label
#> 1           NCIT:C27976                 Stage Ib
#> 2           NCIT:C27977               Stage IIIa
#>                                tnm_id
#> 1 NCIT:C48706,NCIT:C48714,NCIT:C48724
#> 2 NCIT:C48706,NCIT:C48714,NCIT:C48728
#>                                            tnm_label tumor_grade_id
#> 1 N1 Stage Finding,N3 Stage Finding,T2 Stage Finding             NA
#> 2 N1 Stage Finding,N3 Stage Finding,T3 Stage Finding             NA
#>   tumor_grade_label age_iso                 notes icdo_morphology_id
#> 1                NA    P56Y adenocarcinoma [lung]    pgx:icdom-81403
#> 2                NA    P75Y adenocarcinoma [lung]    pgx:icdom-81403
#>   icdo_morphology_label icdo_topography_id icdo_topography_label     pubmed_id
#> 1   Adenocarcinoma, NOS    pgx:icdot-C34.9             Lung, NOS PMID:19607727
#> 2   Adenocarcinoma, NOS    pgx:icdot-C34.9             Lung, NOS PMID:19607727
#>   cellosaurus_id cbioportal_id tcga_project_id analysis_info_experiment_id
#> 1             NA            NA              NA               geo:GSM417055
#> 2             NA            NA              NA               geo:GSM417063
#>   analysis_info_series_id analysis_info_platform_id
#> 1            geo:GSE16597               geo:GPL8690
#> 2            geo:GSE16597               geo:GPL8690
#>                                                                              cohort_ids
#> 1 pgx:cohort-arraymap,pgx:cohort-2021progenetix,pgx:cohort-carriocordo2021heterogeneity
#> 2                                         pgx:cohort-arraymap,pgx:cohort-2021progenetix
#>   biosample_legacy_id  geoprov_city          geoprov_country geoprov_iso_alpha3
#> 1                  NA New York City United States of America                USA
#> 2                  NA New York City United States of America                USA
#>   geoprov_long_latitude geoprov_long_longitude                    updated
#> 1                 40.71                 -74.01 2020-09-10 17:46:45.105000
#> 2                 40.71                 -74.01 2020-09-10 17:46:45.115000

It’s also possible to query by a combination of filters, biosample id, and individual id.

Access a subset of samples

By default, it returns all related samples (limit=0). You can access a subset of them via the parameter limit and skip. For example, if you want to access the first 10 samples , you can set limit = 10, skip = 0.

biosamples_3 <- pgxLoader(type="biosamples", filters = "NCIT:C7541",skip=0, limit = 10)
# Dimension: Number of samples * features
print(dim(biosamples))
#> [1] 256  37
print(dim(biosamples_3))
#> [1] 10 37

Parameter codematches use

Some filters, such as NCIt codes, are hierarchical. As a result, retrieved samples may include not only the specified filters but also their child terms.

unique(biosamples$histological_diagnosis_id)
#> [1] "NCIT:C7541" "NCIT:C8714" "NCIT:C8713"

Setting codematches as TRUE allows this function to only return biosamples that exactly match the specified filter, excluding child terms.

biosamples_4 <- pgxLoader(type="biosamples", filters = "NCIT:C7541",codematches = TRUE)
unique(biosamples_4$histological_diagnosis_id)
#> [1] "NCIT:C7541"

Query the number of samples in Progenetix

The number of samples in specific filters can be queried as follows:

pgxLoader(type="sample_count",filters = "NCIT:C7541")
#>      filters          label total_count exact_match_count
#> 1 NCIT:C7541 Retinoblastoma         256               215

Retrieve individuals information

If you want to query metadata (e.g. survival data) of individuals where the samples of interest come from, set the parameter type to “individuals” and follow the same steps as above.

individuals <- pgxLoader(type="individuals",individual_id = "pgxind-kftx26ml",filters="NCIT:C7541")
# data looks like this
individuals[173:174,]
#>       individual_id      sex_id sex_label age_iso histological_diagnosis_id
#> 173 pgxind-kftx7htu NCIT:C16576    female    P69Y                NCIT:C7541
#> 174 pgxind-m3io41rt NCIT:C20197      male    <NA>                NCIT:C7541
#>     histological_diagnosis_label followup_time followup_state_id
#> 173               Retinoblastoma          NULL       EFO:0030039
#> 174               Retinoblastoma          <NA>       EFO:0030039
#>     followup_state_label diseases_notes
#> 173   no followup status           <NA>
#> 174   no followup status           <NA>
#>                          individual_legacy_id                    updated
#> 173 PGX_IND_MSK_IMPACT_2017-P_0009865_T01_IM5 2024-11-19T03:30:44.900602
#> 174                                      <NA> 2024-11-19T03:41:17.507085

Retrieve analyses information

If you want to know more details about data analyses, set the parameter type to “analyses”. The other steps are the same, except the parameter codematches is not available because analyses data do not include filter information, even though it can be searched by filters.

analyses <- pgxLoader(type="analyses",biosample_id = c("pgxbs-kftvik5i","pgxbs-kftvik96"))

analyses
#>      analysis_id   biosample_id   individual_id analysis_operation_id
#> 1 pgxcs-kftw8qme pgxbs-kftvik5i pgxind-kftx4963   EDAM:operation_3961
#> 2 pgxcs-kftw8rrh pgxbs-kftvik96 pgxind-kftx49ao   EDAM:operation_3961
#>          analysis_operation_label experiment_id   series_id platform_id
#> 1 Copy number variation detection geo:GSM115217 geo:GSE5051 geo:GPL2826
#> 2 Copy number variation detection geo:GSM120460 geo:GSE5359 geo:GPL3960
#>                              platform_label calling_pipeline
#> 1             VUMC MACF human 30K oligo v31       progenetix
#> 2 MPIMG Homo sapiens 44K aCGH3_MPIMG_BERLIN       progenetix
#>                      updated
#> 1 2024-11-20T07:24:51.839782
#> 2 2024-11-20T07:24:53.496612

Visualization of survival data

Suppose you want to investigate whether there are survival differences associated with a particular disease, for example, between younger and older patients, or based on other variables. You can query and visualize the relevant information using the pgxMetaplot function.

pgxMetaplot function

This function generates a survival plot using metadata of individuals obtained by the pgxLoader function.

The parameters of this function:

  • data: The data frame returned by the pgxLoader function, containing survival data for individuals. The survival state is represented by Experimental Factor Ontology in the “followup_state_id” column, and the survival time is represented in ISO 8601 duration format in the “followup_time” column.
  • group_id: A string specifying which column is used for grouping in the Kaplan-Meier plot.
  • condition: A string for splitting individuals into younger and older groups, following the ISO 8601 duration format. Only used if group_id is “age_iso”.
  • return_data: A logical value determining whether to return the metadata used for plotting. Default is FALSE.
  • ...: Other parameters relevant to KM plot. These include pval, pval.coord, pval.method, conf.int, linetype, and palette (see ggsurvplot function from survminer package)

Example usage

# query metadata of individuals with lung adenocarcinoma
luad_inds <- pgxLoader(type="individual",filters="NCIT:C3512")
# use 65 years old as the splitting condition
pgxMetaplot(data=luad_inds, group_id="age_iso", condition="P65Y", pval=TRUE)

It’s noted that not all individuals have available survival data. If you set return_data to TRUE, the function will return the metadata of individuals used for the plot.

Session Info

#> R version 4.4.2 (2024-10-31)
#> Platform: x86_64-pc-linux-gnu
#> Running under: Ubuntu 24.04.1 LTS
#> 
#> Matrix products: default
#> BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=C              
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> time zone: Etc/UTC
#> tzcode source: system (glibc)
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] pgxRpi_1.3.0     BiocStyle_2.35.0
#> 
#> loaded via a namespace (and not attached):
#>  [1] sass_0.4.9          utf8_1.2.4          generics_0.1.3     
#>  [4] tidyr_1.3.1         rstatix_0.7.2       lattice_0.22-6     
#>  [7] digest_0.6.37       magrittr_2.0.3      evaluate_1.0.1     
#> [10] attempt_0.3.1       grid_4.4.2          timechange_0.3.0   
#> [13] fastmap_1.2.0       jsonlite_1.8.9      Matrix_1.7-1       
#> [16] backports_1.5.0     Formula_1.2-5       survival_3.7-0     
#> [19] gridExtra_2.3       BiocManager_1.30.25 httr_1.4.7         
#> [22] purrr_1.0.2         fansi_1.0.6         scales_1.3.0       
#> [25] jquerylib_0.1.4     abind_1.4-8         cli_3.6.3          
#> [28] KMsurv_0.1-5        rlang_1.1.4         munsell_0.5.1      
#> [31] splines_4.4.2       withr_3.0.2         cachem_1.1.0       
#> [34] yaml_2.3.10         tools_4.4.2         ggsignif_0.6.4     
#> [37] dplyr_1.1.4         colorspace_2.1-1    ggplot2_3.5.1      
#> [40] ggpubr_0.6.0        km.ci_0.5-6         broom_1.0.7        
#> [43] curl_6.0.1          buildtools_1.0.0    vctrs_0.6.5        
#> [46] R6_2.5.1            zoo_1.8-12          lifecycle_1.0.4    
#> [49] lubridate_1.9.3     car_3.1-3           pkgconfig_2.0.3    
#> [52] survminer_0.5.0     bslib_0.8.0         pillar_1.9.0       
#> [55] gtable_0.3.6        data.table_1.16.2   glue_1.8.0         
#> [58] xfun_0.49           tibble_3.2.1        tidyselect_1.2.1   
#> [61] sys_3.4.3           knitr_1.49          farver_2.1.2       
#> [64] xtable_1.8-4        survMisc_0.5.6      htmltools_0.5.8.1  
#> [67] labeling_0.4.3      carData_3.0-5       rmarkdown_2.29     
#> [70] maketools_1.3.1     compiler_4.4.2