In this document we illustrate some approaches to working with UK
Biobank summary statistics. Be sure that that the python module
ukbb_pan_ancestry has been installed where reticulate can
find it. (We don’t use basilisk as of 12/24/2022 because of issues in
the terra spark cluster.)
If the above command indicates that BiocHail is not available, see the Installation section near the end of this document.
We have produced a representation of summary statistics for a sample of 9888 loci. This 5GB resource can be retrieved and cached with the following code:
hl = hail_init()
ss = get_ukbb_sumstat_10kloci_mt(hl) # can take about a minute to unzip 5GB
ss$count() # but if a persistent MatrixTable is at the location given
# by env var HAIL_UKBB_SUMSTAT_10K_PATH it goes quicklyTo get a description of available content, we need a python chunk:
Here’s a basic description of the summary stats table, with code that works in terra.bio:
hl = bare_hail()
hl$init(idempotent=TRUE, spark_conf=list(
'spark.hadoop.fs.gs.requester.pays.mode'= 'CUSTOM',
'spark.hadoop.fs.gs.requester.pays.buckets'= 'ukb-diverse-pops-public',
'spark.hadoop.fs.gs.requester.pays.project.id'= Sys.getenv("GOOGLE_PROJECT")))We need to use a python chunk to get the output, using gs:// storage references.
We’ll collect the column metadata to start to understand details of annotation of phenotypes.
sscol = ss$cols()$collect() # OK because we are just working over column metadata
ss1 = sscol[[1]]
names(ss1)
ss1$get("phenocode")
ss1$get("description")We’ve introduced a function that extracts selected fields for a given phenotype, that accommodates availability of results for specific populations.
This can be iterated over all the elements of sscol to
produce a comprehensive searchable table. Here’s a small
illustration:
We’ll trim the material to be worked with by sampling both rows and columns. (2023.01.08: In future revisions we will be able to control the seed for random sampling.)
Row metadata are simple to collect:
The summary statistics themselves reside in entries of the MatrixTable. This can be expensive to collect and so filtering methods beyond random sampling must be mastered. But here is a basic view.
The summary_stats component has the association p-values
– log10 transformed?
BiocHailBiocHail should be installed as follows:
## R version 4.4.2 (2024-10-31)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 24.04.1 LTS
##
## Matrix products: default
## BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so; LAPACK version 3.12.0
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_US.UTF-8 LC_COLLATE=C
## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## time zone: Etc/UTC
## tzcode source: system (glibc)
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] ggplot2_3.5.1 BiocHail_1.7.1 BiocStyle_2.35.0
##
## loaded via a namespace (and not attached):
## [1] sass_0.4.9 generics_0.1.3 RSQLite_2.3.9
## [4] lattice_0.22-6 digest_0.6.37 magrittr_2.0.3
## [7] evaluate_1.0.1 grid_4.4.2 fastmap_1.2.0
## [10] blob_1.2.4 jsonlite_1.8.9 Matrix_1.7-1
## [13] DBI_1.2.3 BiocManager_1.30.25 httr_1.4.7
## [16] scales_1.3.0 jquerylib_0.1.4 cli_3.6.3
## [19] rlang_1.1.4 dbplyr_2.5.0 basilisk.utils_1.19.0
## [22] munsell_0.5.1 bit64_4.5.2 withr_3.0.2
## [25] cachem_1.1.0 yaml_2.3.10 tools_4.4.2
## [28] dir.expiry_1.15.0 parallel_4.4.2 memoise_2.0.1
## [31] dplyr_1.1.4 colorspace_2.1-1 filelock_1.0.3
## [34] basilisk_1.19.0 BiocGenerics_0.53.3 curl_6.0.1
## [37] reticulate_1.40.0 png_0.1-8 buildtools_1.0.0
## [40] vctrs_0.6.5 R6_2.5.1 BiocFileCache_2.15.0
## [43] lifecycle_1.0.4 bit_4.5.0.1 pkgconfig_2.0.3
## [46] gtable_0.3.6 pillar_1.10.0 bslib_0.8.0
## [49] glue_1.8.0 Rcpp_1.0.13-1 xfun_0.49
## [52] tibble_3.2.1 tidyselect_1.2.1 sys_3.4.3
## [55] knitr_1.49 htmltools_0.5.8.1 rmarkdown_2.29
## [58] maketools_1.3.1 compiler_4.4.2