Invalid file path or buffer object type: <class 'deepcpg.data.utils.GzipFile'>

[dnaase]

Hi,
Thanks for making this software. After I download the example data files (by script in deepcpg/examples/setup.sh), when I am running the example script:
python deepcpg_path/dcpg_data.py --cpg_profiles ../data/cpg/BS27_1_SER.tsv.gz ../data/cpg/BS27_3_SER.tsv.gz ../data/cpg/BS27_5_SER.tsv.gz ../data/cpg/BS27_6_SER.tsv.gz ../data/cpg/BS27_8_SER.tsv.gz --dna_files ../data/dna/mm10 --out_dir ./data --dna_wlen 1001 --cpg_wlen 50 --nb_sample 1000

I got the problem of this error. It seems something is wrong about reading the file, but i have checked that all of the files path are correct in my system...

Traceback (most recent call last):
File "/home/yaping/software/deepcpg/deepcpg/scripts/dcpg_data.py", line 629, in
app.run(sys.argv)
File "/home/yaping/software/deepcpg/deepcpg/scripts/dcpg_data.py", line 260, in run
return self.main(name, opts)
File "/home/yaping/software/deepcpg/deepcpg/scripts/dcpg_data.py", line 413, in main
log=log.info)
File "/home/yaping/software/deepcpg/deepcpg/scripts/dcpg_data.py", line 114, in read_cpg_profiles
cpg_profile = dat.read_cpg_profile(cpg_file, sort=True, *args, **kwargs)
File "build/bdist.linux-x86_64/egg/deepcpg/data/utils.py", line 254, in read_cpg_profile
File "/home/yaping/software/anaconda/anaconda2/envs/ccimpute/lib/python2.7/site-packages/pandas/io/parsers.py", line 655, in parser_f
return _read(filepath_or_buffer, kwds)
File "/home/yaping/software/anaconda/anaconda2/envs/ccimpute/lib/python2.7/site-packages/pandas/io/parsers.py", line 392, in _read
filepath_or_buffer, encoding, compression)
File "/home/yaping/software/anaconda/anaconda2/envs/ccimpute/lib/python2.7/site-packages/pandas/io/common.py", line 210, in get_filepath_or_buffer
raise ValueError(msg.format(_type=type(filepath_or_buffer)))
ValueError: Invalid file path or buffer object type: <class 'deepcpg.data.utils.GzipFile'>

[cangermueller]

@dnaase, can you send me the output of pip list? Pandas seems to cause the problem.

[dnaase]

Package Version

---

backports-abc 0.5
backports.shutil-get-terminal-size 1.0.0
backports.ssl-match-hostname 3.4.0.2
cycler 0.10.0
decorator 4.0.11
deepcpg 1.0.4
enum34 1.1.6
funcsigs 1.0.2
functools32 3.2.3.post2
h5py 2.7.0
ipykernel 4.6.1
ipython 5.3.0
ipython-genutils 0.2.0
jupyter-client 5.0.1
jupyter-core 4.3.0
Keras 1.2.2
matplotlib 2.0.2
mock 2.0.0
multitables 1.1.0
mxnet 0.9.4
numexpr 2.6.2
numpy 1.12.1
pandas 0.20.1
pathlib2 2.2.1
pbr 3.0.1
pexpect 4.2.1
pickleshare 0.7.4
pip 9.0.1
prompt-toolkit 1.0.14
protobuf 3.3.0
ptyprocess 0.5.1
py 1.4.33
Pygments 2.2.0
pyparsing 2.2.0
pytest 3.0.7
python-dateutil 2.6.0
pytz 2017.2
PyYAML 3.12
pyzmq 16.0.2
scandir 1.5
scikit-learn 0.18.1
scipy 0.19.0
seaborn 0.7.1
setuptools 27.2.0
simplegeneric 0.8.1
singledispatch 3.4.0.3
six 1.10.0
sklearn 0.0
subprocess32 3.2.7
tables 3.4.2
tensorflow 1.1.0
tensorflow-gpu 1.1.0
tftables 1.1.1
Theano 0.9.0
tornado 4.5.1
traitlets 4.3.2
wcwidth 0.1.7
Werkzeug 0.12.2
wheel 0.29.0

[cangermueller]

pandas 0.20.1 caused the problem. Checkout the latest master version, then it should work.

[dnaase]

pandas 0.20.1 is the newest released version. Do you mean I should re-compile and use the newest github master version of pandas?

[dnaase]

I also tried the older version of pandas, it does not work..

[cangermueller]

Please run pip install -U pandas and git pull in your DeepCpG directory. Then try it again and send me the error message if it still does not work.

[almarzhm]

Thanks for the package. I'm getting a similar error using pandas 0.20.3

Traceback (most recent call last):
File "/home/hasan/anaconda3/bin/dcpg_data.py", line 630, in
app.run(sys.argv)
File "/home/hasan/anaconda3/bin/dcpg_data.py", line 261, in run
return self.main(name, opts)
File "/home/hasan/anaconda3/bin/dcpg_data.py", line 414, in main
log=log.info)
File "/home/hasan/anaconda3/bin/dcpg_data.py", line 115, in read_cpg_profiles
cpg_profile = dat.read_cpg_profile(cpg_file, sort=True, *args, **kwargs)
File "/home/hasan/anaconda3/lib/python3.6/site-packages/deepcpg/data/utils.py", line 182, in read_cpg_profile
usecols=usecols, dtype=dtype, skiprows=skiprows)
File "/home/hasan/anaconda3/lib/python3.6/site-packages/pandas/io/parsers.py", line 655, in parser_f
return _read(filepath_or_buffer, kwds)
File "/home/hasan/anaconda3/lib/python3.6/site-packages/pandas/io/parsers.py", line 392, in _read
filepath_or_buffer, encoding, compression)
File "/home/hasan/anaconda3/lib/python3.6/site-packages/pandas/io/common.py", line 210, in get_filepath_or_buffer
raise ValueError(msg.format(_type=type(filepath_or_buffer)))
ValueError: Invalid file path or buffer object type: <class 'deepcpg.data.utils.GzipFile'>

[almarzhm]

Installing pandas 0.19.2 should fix this issue:
pip install pandas==0.19.2

[AgentLjc]

pandas 0.19.2 indeed fix it

[cangermueller]

Sorry for the late reply, I am quite busy with my new job ;-). Thanks @dnaase and @almarzhm for solving this issue!