Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Staging input files in a subfolder changes their names when they are symlinks #292

Closed
mfoll opened this issue Feb 23, 2017 · 4 comments
Closed

Comments

@mfoll
Copy link

mfoll commented Feb 23, 2017

When input files are symlinks (like titi.txt in the following example) pointing to a file with a different name (tata.txt in the example) AND we ask nextflow to stage this file as an input in a subfolder (TXT/ here), the new link created by nextflow in this folder in work is renamed with the name of the symlink target (tata.txt here).
Here is a replicable toy example of the bug. My current environment:

[follm@hn test_nf]$ ls -l
total 0
lrwxrwxrwx 1 follm lsu 20 Feb 22 17:08 titi.txt -> /data/follm/tata.txt
-rw-r--r-- 1 follm lsu  0 Feb 22 17:07 toto.txt

Note here titi.txt -> /data/follm/tata.txt. The nextflow script causing the bug:

txt = Channel.fromPath( '*.txt' ).toList()

process printHello {

   input:
   file 'TXT/*' from txt  

   output: 
   stdout into result

   shell:
   '''
   ls -l *
   '''
}	

result.println()

The output when running the script:

[follm@hn test_nf]$ nextflow run ~/nf_scripts/test.nf 
N E X T F L O W  ~  version 0.23.3
Launching `/home/follm/nf_scripts/test.nf` [grave_pike] - revision: a5b30c5813
[warm up] executor > lsf
[97/3b9225] Submitted process > printHello (1)
total 0
lrwxrwxrwx 1 follm lsu 26 Feb 23 09:47 tata.txt -> /mnt/beegfs/follm/tata.txt
lrwxrwxrwx 1 follm lsu 34 Feb 23 09:47 toto.txt -> /mnt/beegfs/follm/test_nf/toto.txt

Note that here tata.txt -> /mnt/beegfs/follm/tata.txt while the file was originally named titi.txt (note that in my system /data -> /mnt/beegfs, so this is correctly resolved).

Simply changing file 'TXT/*' from txt to file txt or even file '*' from txt returns the correct output (at least what I am expecting...):

[follm@hn test_nf]$ nextflow run ~/nf_scripts/test.nf 
N E X T F L O W  ~  version 0.23.3
Launching `/home/follm/nf_scripts/test.nf` [gloomy_noether] - revision: 31557eca14
[warm up] executor > lsf
[95/be833c] Submitted process > printHello (1)
lrwxrwxrwx 1 follm lsu 26 Feb 23 10:02 titi.txt -> /mnt/beegfs/follm/tata.txt
lrwxrwxrwx 1 follm lsu 34 Feb 23 10:02 toto.txt -> /mnt/beegfs/follm/test_nf/toto.txt

Note here titi.txt -> /mnt/beegfs/follm/tata.txt.

@mfoll
Copy link
Author

mfoll commented Feb 23, 2017

It seems to be related to #222, as the output is correct with a version before this has been fixed (but symlinks are not resolved of course):

[follm@hn test_nf]$ export NXF_VER=0.22.0
[follm@hn test_nf]$ nextflow run ~/nf_scripts/test.nf 
N E X T F L O W  ~  version 0.22.0
Launching `/home/follm/nf_scripts/test.nf` [lonely_mahavira] - revision: a5b30c5813
[warm up] executor > lsf
[6c/baf532] Submitted process > printHello (1)
total 0
lrwxrwxrwx 1 follm lsu 34 Feb 23 10:09 titi.txt -> /mnt/beegfs/follm/test_nf/titi.txt
lrwxrwxrwx 1 follm lsu 34 Feb 23 10:09 toto.txt -> /mnt/beegfs/follm/test_nf/toto.txt

@pditommaso
Copy link
Member

Good shot! I've just uploaded 0.23.4-SNAPSHOT, please confirm that fixes the issue on your side.

@mfoll
Copy link
Author

mfoll commented Feb 24, 2017

I just checked and it is working as expected. Thanks for the quick fix!

@pditommaso
Copy link
Member

Released with version 0.23.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants