Peer certificate cannot be authenticated with given CA certificates

[hussius]

I get the following error when trying to open a remote file:

>>> bw = pyBigWig.open("https://data.broadinstitute.org/compbio1/PhyloCSFtracks/hg19/latest/PhyloCSF+0.bw")
[urlOpen] curl_easy_perform received an error: Peer certificate cannot be authenticated with given CA certificates

I've tried to install new CA certificates on my Mac (El Capitan 10.11.6) by following instructions I found online, but they don't seem to fix the issue so far. Would be grateful for suggestions.

[ghuls]

Does the command line version of curl work for you?

curl -O "https://data.broadinstitute.org/compbio1/PhyloCSFtracks/hg19/latest/PhyloCSF+0.bw"

[dpryan79]

@hussius As @ghuls mentioned, if you can get curl on the command line to work then pyBigWig should work too.

I should note that I've been meaning to have open() default to ignoring SSL certificate issues, since I don't expect regular end users (e.g., all the deepTools users) to have to deal with that.

[hussius]

It does work for me on the command line.

130-229-4-255-dhcp:~ mikaelhuss$ curl -O "https://data.broadinstitute.org/compbio1/PhyloCSFtracks/hg19/latest/PhyloCSF+0.bw"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0 3019M    0 4622k    0     0  1189k      0  0:43:19  0:00:03  0:43:16 1189k

[dpryan79]

My only guess is that whichever libcurl pyBigWig is getting linked against is using a different CAPATH. I really just need to adjust pyBigWig to ignore such things by default. In the interim, you can add the following to libBigWig/io.c after line 234:

if(curl_easy_setopt(URL->x.curl, CURLOPT_SSL_VERIFYPEER, 0) != CURLE_OK) {
    fprintf(stderr, "[urlOpen] Couldn't set CURLOPT_SSL_VERIFYPEER to 0!\n");
    goto error;
}

That won't solve the root problem, but it'll probably get things running.

[hussius]

Go figure - it works for me now in a fresh conda environment. I did not try the @dpryan79 addition because with my conda-installed package on Mac, pyBigWig.__file__ points to a .so file and I can't find any C source files. Cheers!

[dpryan79]

Ah, I'm not sure conda was using the system-installed certificates (there have also been on-going issues with libcurl on OSX in conda, I used to have a separate curl inside bioconda for this reason). Anyway, I'll leave this open as a reminder to myself to just have these sorts of errors ignored.

[dpryan79]

Starting in version 0.3.5 certificate issues should generally be ignored. I considered making certificate checks optional, but I suspect no one would ever actually bother to use that and adding it in would be a bit annoying, so I'll hold off unless someone explicitly asks for it.