Trouble with files on Amazon S3

[dzerbino]

Hello @dpryan79 ,

It would appear that libBigWig struggles with remote files on S3. e.g, with the following code (compiled to toto):

#include "bigWig.h"

void main(int c, char ** argv) {
bwOpen(argv[1], NULL, "r");
}

I get:

./toto https://encode-public.s3.amazonaws.com/2017/10/03/ad2c0f17-0824-4647-a749-74276daca7da/ENCFF278CUB.bigWig
Segmentation fault
wget https://encode-public.s3.amazonaws.com/2017/10/03/ad2c0f17-0824-4647-a749-74276daca7da/ENCFF278CUB.bigWig
[...]
./toto ENCFF278CUB.bigWig

Conversely, the Kent library has no issue with these remote bigWig files via CURL.

Would it be possible for libBigWig to handle these remote files on AWS?

Thank you,

@dzerbino

[diekhans]

note, kent libraries don't use CURL...

[dzerbino]

A-ha, what is used then?

[diekhans]

Kent code it yourself in C approach.

[dpryan79]

@dzerbino You forgot to run bwInit(some buffer size); first to setup curl:

#include "bigWig.h"
int main(int argc, char* argv[]) {
    int rv = bwInit(128000);
    bigWigFile_t *bw = bwOpen(argv[1], NULL, "r");
    bwClose(bw);
    bwCleanup();
    return 0;
}

128000 is the buffer size I used in pyBigWig, you can change it.

[dzerbino]

Thanks @dpryan79 ! I did indeed omit bwInit, both in the example here and in my code.