[1] ".cred not detected. Using .snp instead." #72
Comments
|
Hi @AMCalejandro, could you send your Session Info? There's some fixes in the dev branch that I haven't merged yet. |
|
Here: r$> sessionInfo() R version 4.0.3 (2020-10-10) Platform: x86_64-conda-linux-gnu (64-bit) Running under: Ubuntu 20.04.3 LTS Matrix products: default BLAS/LAPACK: /home/acarrasco/.conda/envs/echoR/lib/libopenblasp-r0.3.12.so locale: [1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C LC_TIME=en_GB.UTF-8 LC_COLLATE=en_GB.UTF-8 LC_MONETARY=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8 LC_PAPER=en_GB.UTF-8 [8] LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] data.table_1.14.2 forcats_0.5.1 stringr_1.4.0 dplyr_1.0.7 purrr_0.3.4 readr_2.1.2 tidyr_1.2.0 tibble_3.1.6 ggplot2_3.3.5 tidyverse_1.3.1 echolocatoR_0.1.2 loaded via a namespace (and not attached): [1] readxl_1.3.1 backports_1.4.1 Hmisc_4.5-0 BiocFileCache_1.14.0 plyr_1.8.6 lazyeval_0.2.2 splines_4.0.3 [8] BiocParallel_1.24.1 GenomeInfoDb_1.26.7 digest_0.6.29 ensembldb_2.14.1 htmltools_0.5.2 fansi_1.0.2 magrittr_2.0.2 [15] checkmate_2.0.0 memoise_2.0.1 BSgenome_1.58.0 cluster_2.1.1 tzdb_0.2.0 Biostrings_2.58.0 modelr_0.1.8 [22] matrixStats_0.61.0 ggbio_1.38.0 askpass_1.1 prettyunits_1.1.1 jpeg_0.1-8.1 colorspace_2.0-2 rvest_1.0.2 [29] blob_1.2.2 rappdirs_0.3.3 haven_2.4.3 xfun_0.29 crayon_1.4.2 RCurl_1.98-1.5 jsonlite_1.7.3 [36] graph_1.68.0 Exact_2.1 survival_3.2-11 VariantAnnotation_1.36.0 glue_1.6.1 gtable_0.3.0 zlibbioc_1.36.0 [43] XVector_0.30.0 DelayedArray_0.16.3 BiocGenerics_0.36.1 scales_1.1.1 mvtnorm_1.1-1 DBI_1.1.2 GGally_2.1.1 [50] Rcpp_1.0.8 progress_1.2.2 htmlTable_2.1.0 foreign_0.8-81 bit_4.0.4 proxy_0.4-25 OrganismDbi_1.32.0 [57] Formula_1.2-4 stats4_4.0.3 htmlwidgets_1.5.3 httr_1.4.2 RColorBrewer_1.1-2 ellipsis_0.3.2 pkgconfig_2.0.3 [64] reshape_0.8.8 XML_3.99-0.6 nnet_7.3-15 dbplyr_2.1.1 utf8_1.2.2 tidyselect_1.1.1 rlang_1.0.1 [71] reshape2_1.4.4 AnnotationDbi_1.52.0 munsell_0.5.0 cellranger_1.1.0 tools_4.0.3 cachem_1.0.6 cli_3.1.1 [78] generics_0.1.2 RSQLite_2.2.9 broom_0.7.12 fastmap_1.1.0 fs_1.5.2 knitr_1.37 bit64_4.0.5 [85] AnnotationFilter_1.14.0 rootSolve_1.8.2.1 RBGL_1.66.0 xml2_1.3.3 biomaRt_2.46.3 compiler_4.0.3 rstudioapi_0.13 [92] curl_4.3 png_0.1-7 e1071_1.7-6 reprex_2.0.1 DescTools_0.99.41 stringi_1.7.6 GenomicFeatures_1.42.3 [99] lattice_0.20-41 ProtGenerics_1.22.0 Matrix_1.3-2 vctrs_0.3.8 pillar_1.7.0 lifecycle_1.0.1 BiocManager_1.30.12 [106] bitops_1.0-7 lmom_2.8 rtracklayer_1.50.0 GenomicRanges_1.42.0 R6_2.5.1 latticeExtra_0.6-29 gridExtra_2.3 [113] IRanges_2.24.1 gld_2.6.2 dichromat_2.0-0 boot_1.3-27 MASS_7.3-53.1 assertthat_0.2.1 SummarizedExperiment_1.20.0 [120] openssl_1.4.6 withr_2.4.3 GenomicAlignments_1.26.0 Rsamtools_2.6.0 S4Vectors_0.28.1 GenomeInfoDbData_1.2.4 expm_0.999-6 [127] parallel_4.0.3 hms_1.1.1 grid_4.0.3 rpart_4.1-15 class_7.3-18 MatrixGenerics_1.2.1 biovizBase_1.38.0 [134] Biobase_2.50.0 lubridate_1.8.0 base64enc_0.1-3 r$> |
|
I am having a similar error at the exact same step, and believe I have found the issue (or at least, an issue). When trying to load the cred file for merging into the multi-finemap file, I see the following error, which I think may be related to the error above: "[1] "+ FINEMAP:: Importing conditional probabilities (.cred)..." The issue here, from what I can gather, has to do with calling fread to load the .cred file. For some reason, despite the comment character leading the first line of the cred file, fread does not seem to register it as such and assumes these are the column names: "# Post-Pr(# of causal SNPs is 1) = 1 This may be throwing a resultant error assuming that the cred file "doesn't exist", when in fact it is just that fread is assuming the first commented line (# Post-Pr(# of causal SNPs is 1) = 1) is actually the header. Changing this line to another function may resolve this? |
|
Thanks @dklarin7 Did you try to modify the function on the echolocatoR package and rerun? I wonder if @bschilder already had a look at this. Thank you guys! |
|
Hi @AMCalejandro Thanks, |
|
Ok, so I was finally able to dig into this and resolve the root issues. FixesI've made some patches to the master branch to address these issues ( 1. Reading .cred files from FINEMAPThe crux of the issue here is that file formats changed across FINEMAP versions, which I hadn't accounted for. I've made some edits to ensure that the .cred files can be read properly regardless of version (though I'll keep an eye out for any further formatting changes in the future). 2. Merged fine-mapping resultsThis is a weird one, and I still don't fully understand why this was happening, but for some reason merging fine-mapping results from each method on the "SNP" column wasn't working with echoverseWhile I made just these patches on the master branch, I've rewritten much of remotes::install_github("RajLabMSSM/echolocatoR@echoverse") |
|
Hey @bschilder, I tried installing echoverse suite with the above mentioned command: remotes::install_github("RajLabMSSM/echolocatoR/echoverse"), but it was not available for R versions 4.1.0 or 3.5.3. Any tips you may have for this? Thanks a lot! |
|
@mkoromina apologies, that was a typo, should have been an @ before echoverse (not a "/"). Just corrected it |
@bschilder If data.table is behaving unexpectedly inside your package functions, try defining |
I'd never heard of this approach before! Thanks so much @jdblischak, I'll try this out, perhaps by adding to a .zzz file. |
|
Hi, Apparently the weird issue still persist. Maybe I should switch to echoverse..... Just copying the output when finemap fails vvvvv-- ABF --vvvvv ✅ All required columns present. ✅ All suggested columns present. vvvvv-- FINEMAP --vvvvv ✅ All required columns present. ✅ All suggested columns present. vvvvv-- SUSIE --vvvvv ✅ All required columns present. ✅ All suggested columns present. vvvvv-- POLYFUN_SUSIE --vvvvv ✅ All required columns present. ✅ All suggested columns present. +++ Multi-finemap:: ABF +++ +++ Multi-finemap:: FINEMAP +++ [1] "++ FINEMAP:: Constructing master file." [1] "++ FINEMAP:: Constructing data.z file." [1] "++ FINEMAP:: Constructing data.ld file." [1] "+ Using FINEMAP v1.4" |--------------------------------------| | Welcome to FINEMAP v1.4 | | | | (c) 2015-2020 University of Helsinki | | | | Help : | | - ./finemap --help | | - www.finemap.me | | - www.christianbenner.com | | | | Contact : | | - christian.benner@helsinki.fi | | - matti.pirinen@helsinki.fi | |--------------------------------------| -------- SETTINGS -------- - dataset : all - corr-config : 0.95 - n-causal-snps : 5 - n-configs-top : 50000 - n-conv-sss : 100 - n-iter : 100000 - n-threads : 1 - prior-k0 : 0 - prior-std : 0.05 - prob-conv-sss-tol : 0.001 - prob-cred-set : 0.95 ------------ FINE-MAPPING (1/1) ------------ - GWAS summary stats : FINEMAP/data.z - SNP correlations : FINEMAP/data.ld - Causal SNP stats : FINEMAP/data.snp - Causal configurations : FINEMAP/data.config - Credible sets : FINEMAP/data.cred - Log file : FINEMAP/data.log_sss - Reading input : done! - Number of GWAS samples : 3572 - Number of SNPs : 4057 - Prior-Pr(# of causal SNPs is k) : (0 -> 0) 1 -> 0.583 2 -> 0.291 3 -> 0.0971 4 -> 0.0243 5 -> 0.00485 - 100430 configurations evaluated (0.113/100%) : converged after 113 iterations - Computing causal SNP statistics : done! - Regional SNP heritability : 93.1 (SD: 0.374 ; 95% CI: [92.3,93.8]) - Log10-BF of >= one causal SNP : 109 - Post-expected # of causal SNPs : 5 - Post-Pr(# of causal SNPs is k) : (0 -> 0) 1 -> 0 2 -> 0 3 -> 1.92e-44 4 -> 3.85e-32 5 -> 1 - Computing credible sets : done! - Writing output : done! - Run time : 0 hours, 0 minutes, 30 seconds [1] ".cred not detected. Using .snp instead." [1] "+ FINEMAP:: Importing prob (.snp)..." Error in `[.data.table`(x, r, vars, with = FALSE) : column(s) not found: prob In addition: Warning message: In sdY.est(d$varbeta, d$MAF, d$N) : estimating sdY from maf and varbeta, please directly supply sdY if known r$> sessionInfo() R version 4.0.3 (2020-10-10) Platform: x86_64-conda-linux-gnu (64-bit) Running under: Ubuntu 20.04.3 LTS Matrix products: default BLAS/LAPACK: /home/acarrasco/.conda/envs/echoR/lib/libopenblasp-r0.3.12.so locale: [1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C LC_TIME=en_GB.UTF-8 LC_COLLATE=en_GB.UTF-8 LC_MONETARY=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8 LC_PAPER=en_GB.UTF-8 [8] LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] echolocatoR_0.2.3 loaded via a namespace (and not attached): [1] magrittr_2.0.2 tidyselect_1.1.2 munsell_0.5.0 colorspace_2.0-3 R6_2.5.1 rlang_1.0.2 fansi_1.0.3 dplyr_1.0.8 grid_4.0.3 data.table_1.14.2 gtable_0.3.0 [12] utf8_1.2.2 cli_3.2.0 DBI_1.1.2 ellipsis_0.3.2 assertthat_0.2.1 tibble_3.1.6 lifecycle_1.0.1 crayon_1.5.0 purrr_0.3.4 ggplot2_3.3.5 vctrs_0.3.8 [23] glue_1.6.2 compiler_4.0.3 pillar_1.7.0 generics_0.1.2 scales_1.1.1 pkgconfig_2.0.3 |
|
@AMCalejandro hey, so upon running this, do you not get any results from my understanding? |
|
I do get the plots with the regional plots, transcripts, epigenomic map based on the specified inputted data.
However, the fine mapping tools are not working properly.
…________________________________
From: Maria Koromina ***@***.***>
Sent: 25 March 2022 18:24
To: RajLabMSSM/echolocatoR ***@***.***>
Cc: Martinez Carrasco, Alejandro ***@***.***>; Mention ***@***.***>
Subject: Re: [RajLabMSSM/echolocatoR] [1] ".cred not detected. Using .snp instead." (Issue #72)
⚠ Caution: External sender
@AMCalejandro<https://github.com/AMCalejandro> hey, so upon running this, do you not get any results from my understanding?
—
Reply to this email directly, view it on GitHub<#72 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUDOQD2J4LBKY4ZHNPYXZK3VBYAF3ANCNFSM5NGZKKHQ>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
I was running FINEMAP and I believe that FINEMAP.check_files() fails to find an existing .cred file
First, I am copying the output of my FINEMAP run:
) ) ) ))))))}}}}}}}} {{{{{{{{{(((((( ( ( ( AC108211.1 (1 / 10) ) ) ) ))))))}}}}}}}} {{{{{{{{{(((((( ( ( ( [1] "+ CONDA:: Identified bgzip executable in echoR env." [1] "Determining chrom type from file header" [1] "LD:: Standardizing summary statistics subset." [1] "++ Preparing Gene col" [1] "++ Preparing A1,A1 cols" [1] "++ Preparing MAF,Freq cols" [1] "++ Inferring MAF from frequency column..." [1] "++ Preparing N_cases,N_controls cols" [1] "++ Preparing `proportion_cases` col" [1] "++ 'proportion_cases' not included in data subset." [1] "++ Preparing N col" [1] "+ Preparing sample_size (N) column" [1] "++ Using `sample_size = 3572` TRUE" [1] "++ Preparing t-stat col" [1] "+ Calculating t-statistic from Effect and StdErr..." [1] "++ Assigning lead SNP" [1] "++ Ensuring Effect, StdErr, P are numeric" [1] "++ Ensuring 1 SNP per row" [1] "++ Removing extra whitespace" [1] "++ Saving subset ==> /mnt/rreal/RDS/acarrasco/ANALYSES_WORKSPACE/EARLY_PD/POST_GWAS/ECHOLOCATOR/RESULTS_2//mixedmodels_GWAS/earlymotorPD_axial/AC108211.1/AC108211.1_earlymotorPD_axial_subset.tsv.gz" [1] "LD:: Using UK Biobank LD reference panel." [1] "+ UKB LD file name: chr4_156000001_159000001" [1] "+ LD:: Downloading full .gz/.npz UKB files and saving to disk." [1] "+ Overwriting pre-existing file." [1] "+ CONDA:: Identified axel executable in echoR env." [1] "+ Overwriting pre-existing file." [1] "+ CONDA:: Identified axel executable in echoR env." [1] "+ LD:: load_ld() python function input: /mnt/rreal/RDS/acarrasco/ANALYSES_WORKSPACE/EARLY_PD/POST_GWAS/ECHOLOCATOR/RESULTS_2//mixedmodels_GWAS/earlymotorPD_axial/AC108211.1/LD/chr4_156000001_159000001" [1] "+ LD:: Reading LD matrix into memory. This could take some time..." /mnt/rreal/RDS/acarrasco/ANALYSES_WORKSPACE/EARLY_PD/POST_GWAS/ECHOLOCATOR/RESULTS_2//mixedmodels_GWAS/earlymotorPD_axial/AC108211.1/LD/chr4_156000001_159000001.gz /mnt/rreal/RDS/acarrasco/ANALYSES_WORKSPACE/EARLY_PD/POST_GWAS/ECHOLOCATOR/RESULTS_2//mixedmodels_GWAS/earlymotorPD_axial/AC108211.1/LD/chr4_156000001_159000001.npz Processed URL: /mnt/rreal/RDS/acarrasco/ANALYSES_WORKSPACE/EARLY_PD/POST_GWAS/ECHOLOCATOR/RESULTS_2//mixedmodels_GWAS/earlymotorPD_axial/AC108211.1/LD/chr4_156000001_159000001 Some other message at the end [1] "+ Full UKB LD matrix: 21897 x 21897" [1] "+ Full UKB LD SNP data.table: 21897 x 5" [1] "+ LD:: Saving LD matrix ==> /mnt/rreal/RDS/acarrasco/ANALYSES_WORKSPACE/EARLY_PD/POST_GWAS/ECHOLOCATOR/RESULTS_2//mixedmodels_GWAS/earlymotorPD_axial/AC108211.1/LD/AC108211.1.UKB_LD.RDS" [1] "5340 x 5340 LD_matrix (sparse)" vvvvv-- ABF --vvvvv ✅ All required columns present. ✅ All suggested columns present. vvvvv-- FINEMAP --vvvvv ✅ All required columns present. ✅ All suggested columns present. vvvvv-- SUSIE --vvvvv ✅ All required columns present. ✅ All suggested columns present. vvvvv-- POLYFUN_SUSIE --vvvvv ✅ All required columns present. ✅ All suggested columns present. +++ Multi-finemap:: ABF +++ +++ Multi-finemap:: FINEMAP +++ [1] "++ FINEMAP:: Constructing master file." [1] "++ FINEMAP:: Constructing data.z file." [1] "++ FINEMAP:: Constructing data.ld file." [1] "+ Using FINEMAP v1.4" |--------------------------------------| | Welcome to FINEMAP v1.4 | | | | (c) 2015-2020 University of Helsinki | | | | Help : | | - ./finemap --help | | - www.finemap.me | | - www.christianbenner.com | | | | Contact : | | - christian.benner@helsinki.fi | | - matti.pirinen@helsinki.fi | |--------------------------------------| -------- SETTINGS -------- - dataset : all - corr-config : 0.95 - n-causal-snps : 5 - n-configs-top : 50000 - n-conv-sss : 100 - n-iter : 100000 - n-threads : 1 - prior-k0 : 0 - prior-std : 0.05 - prob-conv-sss-tol : 0.001 - prob-cred-set : 0.95 ------------ FINE-MAPPING (1/1) ------------ - GWAS summary stats : FINEMAP/data.z - SNP correlations : FINEMAP/data.ld - Causal SNP stats : FINEMAP/data.snp - Causal configurations : FINEMAP/data.config - Credible sets : FINEMAP/data.cred - Log file : FINEMAP/data.log_sss - Reading input : done! - Number of GWAS samples : 3572 - Number of SNPs : 5338 - Prior-Pr(# of causal SNPs is k) : (0 -> 0) 1 -> 0.583 2 -> 0.291 3 -> 0.0971 4 -> 0.0243 5 -> 0.00485 - 367552 configurations evaluated (0.654/100%) : converged after 654 iterations - Computing causal SNP statistics : done! - Regional SNP heritability : -2.08e-05 (SD: 0.00215 ; 95% CI: [0.00461,0.0124]) - Log10-BF of >= one causal SNP : 36.6 - Post-expected # of causal SNPs : 5 - Post-Pr(# of causal SNPs is k) : (0 -> 0) 1 -> 0 2 -> 0 3 -> 0 4 -> 9.65e-08 5 -> 1 - Computing credible sets : done! - Writing output : done! - Run time : 0 hours, 0 minutes, 54 seconds [1] ".cred not detected. Using .snp instead." [1] "+ FINEMAP:: Importing prob (.snp)..." Error in `[[<-.data.frame`(`*tmp*`, col, value = 1) : replacement has 1 row, data has 0 In addition: Warning messages: 1: Python '/home/acarrasco/.conda/envs/echoR/bin/python' was requested but '/tools/anaconda3/bin/python' was loaded instead (see reticulate::py_config() for more information) 2: In sdY.est(d$varbeta, d$MAF, d$N) : estimating sdY from maf and varbeta, please directly supply sdY if knownI then checked the FINEMAP.check_files() function
FINEMAP.check_files <- function(locus_dir,
results_file){
# locus_dir="/Users/schilder/Desktop/echolocatoR/results/GWAS/Nalls23andMe_2019/BST1"
### In FINEMAP v1.3, only one .cred file are produced.
### In FINEMAP v1.4, multiple FINEMAP files with # suffixes are produced.
.cred_files <- list.files(file.path(locus_dir,"FINEMAP"), "data.cred", full.names = T)
.cred_exists <- length(.cred_files)>0
.snp_exists <- file.exists(file.path(locus_dir,"FINEMAP/data.snp"))
.config_exists <- file.exists(file.path(locus_dir,"FINEMAP/data.config"))
file_options <- c(".cred",".snp",".config")[c(.cred_exists,.snp_exists,.config_exists)]
if(!results_file %in% file_options){
printer(results_file,"not detected.",
"Using",file_options[1],"instead.")
return(file_options[1])
}else { return(results_file)}
}
results_file <- FINEMAP.check_files(locus_dir, results_file)
Looking at the output of my run, it says that [1] ".cred not detected. Using .snp instead.", although this file certainly is on my locus dir
And if we manually run FINEMAP.check_files():
Even though the tool does not get to use .cred file, we still could use the .snp file present as well but then the error message arises
I understand the error is becuase there is no row with a prob value higher than the credset_threshold. Maybe worth adding a message
data.snp[data.snp[[prob_col]] > credset_thresh,] %>% plyr::mutate(CS=1) %>% dplyr::rename(PP=dply r::all_of(prob_col)) Error in `[[<-.data.frame`(`*tmp*`, col, value = 1) : replacement has 1 row, data has 0I understand there is no causal SNP on this specific locus by looking at the PIP. Maybe worth adding a message when that happens.
However, there is an issue getting to use cred and snp files that I could not figure out
The text was updated successfully, but these errors were encountered: