`complementSumstats.R`: merge failure

[espenhgn]

This print output defined on this line is incorrect:

containers/usecases/LDpred2/fun.R

Line 59 in ff12bb0

if (nrow(res) != nrRows) stop('The merge resulted in ', nrow(res), ' but input contained ', nrow(res))

in case the comparison fails, I got:

[deepchocolate]

That's fixed in PR #150

[espenhgn]

Ok, this issue should be closed by merging #150

[espenhgn]

Just a question; it's clear that there are SNPs in the sumstats file I'm using that are not part of the default GRCh37 build file, causing the crash. I replace the stop() to a print(paste(c(...))) statement just in order to forcefully add the missing columns. Is this safe enough? Would there be other genomic builds that may contain this missing information? There are about 10k SNPs out of 850k that are missing. I could perhaps drop these rows altogether.

[deepchocolate]

Just a question; it's clear that there are SNPs in the sumstats file I'm using that are not part of the default GRCh37 build file, causing the crash. I replace the stop() to a print(paste(c(...))) statement just in order to forcefully add the missing columns. Is this safe enough? Would there be other genomic builds that may contain this missing information? There are about 10k SNPs out of 850k that are missing. I could perhaps drop these rows altogether.

It's not a problem that there's missingness in the merged file. LDpred2::snp_match will filter out snps that are not matched or duplicated. The intention here was to stop if the merge resulted in a duplicated dataset, because that should indicate that there are duplicates in one of the datasets you're merging. But it seems that several people have had problems with this stop mark, so it might be better to replace it with a warning.

[deepchocolate]

Btw, I think PRSICE2 will stop and complain if SNPs are duplicated. If I remember correctly, you'll get a list of SNPs that are duplicated (might be a list of SNPS that are ok)...

[espenhgn]

Just a question; it's clear that there are SNPs in the sumstats file I'm using that are not part of the default GRCh37 build file, causing the crash. I replace the stop() to a print(paste(c(...))) statement just in order to forcefully add the missing columns. Is this safe enough? Would there be other genomic builds that may contain this missing information? There are about 10k SNPs out of 850k that are missing. I could perhaps drop these rows altogether.

It's not a problem that there's missingness in the merged file. LDpred2::snp_match will filter out snps that are not matched or duplicated. The intention here was to stop if the merge resulted in a duplicated dataset, because that should indicate that there are duplicates in one of the datasets you're merging. But it seems that several people have had problems with this stop mark, so it might be better to replace it with a warning.

I think printing a warning is a good compromise. If it works with LDpred2 (our immediate concern), I'm happy. We can of course also drop duplicates in this procedure, akin to

res <- res[!duplicated(res$SNP),]

Btw, I think PRSICE2 will stop and complain if SNPs are duplicated. If I remember correctly, you'll get a list of SNPs that are duplicated (might be a list of SNPS that are ok)...

The PRSice2 tutorial indeed suggests removing duplicates (https://choishingwan.github.io/PRS-Tutorial/prsice/#required-data).

[deepchocolate]

@espenhgn Great, but I'll change to a warning in #150 prior to merging. In my mind there shouldn't be any duplicates in the files this has been tested with so far, and I haven't encountered it with the sumstat files I've used with the MoBa data, but it would be good to know the cause prior to modifying the output data.