Bug in _uncompress_file

[SylvainTakerkart]

Hi!
(...still nilearn-sprinting in the train back to Marseille ;) )

I found a bug when trying to download .tar.gz archive files from osf.

I got an error when using _fetch_files:

from nilearn.datasets.utils import _fetch_files
opts = {'uncompress': True}
data_dir = '~/nilearn_data'
url = 'https://osf.io/vym9k/download'
_fetch_files(data_dir, [('sphere_right.gii', url, opts)])

After digging, I realized the problem is in _uncompress_file: the temporary downloaded file is actually called 'download' on disk (because of the url that's given by osf), and _uncompress_file does not extract the files if the archive was a .tar.gz one (it does not return an error, and it erases the 'download' file)

Note that:

• if the archive is a zip file, everything works... (as in the fetch_icbm152_2009 function); the temporary file is also called 'download' on disk, but _uncompress_file handles it properly
• if the file on disk is actually called 'whatever.tar.gz' and you run _uncompress_file on it, it also handles it properly

Soooooooooo, to reproduce the problem, the easiest is that:

1. you type https://osf.io/vym9k/download in your browser to get the file
2. you rename the file manually in your unix shell

> mv sphere_mesh.tar.gz download

3. you try to use _uncompress_file on it

from nilearn.datasets.utils import _uncompress_file
_uncompress_file('download')

It will: 1. not extract the files in the archive, 2. erase the 'download' file, 3. not return an error.

I hope this is clear enough...

Sylvain

[kchawla-pi]

@SylvainTakerkart asked me to replace on OSF the two .tar.gz files with the .zip files and send the two new links.
On it.

[kchawla-pi]

InterTVA: https://osf.io/aekp7/download
sphere mesh: https://osf.io/b79fy/download

[SylvainTakerkart]

Thanks! I can then move forward with writing my fetchers (just created a PR for the sphere meshes)...

But this bug will still need to be looked at at some point...

[kchawla-pi]

File an issue, with all the details.

Edit: Sorry I thought this was a PR.

[AlexandreAbraham]

Hello,

Indeed, there was a bug in the function. I created a PR to fix this:
#2126

Although, this kind of thing has been taken care of at _fetch_files level. Indeed, there is an opt to move a file after downloading (it's a bit of a hidden feature because it's a bit hacky). If you use

opts = {'uncompress': True, 'move': 'download.tgz'}

Then you should be fine because fetch_files with download your file, move it from download to download.gz and then _uncompress_file should work ok.

I am the unfaithful author of this evilish machinery, do not hesitate to mail me if you have that kind of issue.

[SylvainTakerkart]

Hi Alexandre, thanks for your message!
I have not contributed much to nilearn until now, but with the small things I've done that needed to setup some new data fetchers, I have to admit I have been struggling with using this machinery (I'm happy you self-caracterize it as evilish, this was my feeling also, haha ;) ), sometimes even trying to copy-paste its behavior from existing code... Is there any kind of documentation or tutorial available? That would be super useful!
Thanks,
Sylvain