inconsistent referencing of $TMPDIR ?

[kwongj]

Hi Andrew,
Thanks very much for your work on Roary - it's fantastic and we use it all the time.
Just recently (in the last week), I've been getting an error:

jkwong1@trillian:$ roary -a -p 16 -v gff/2014-*

Please cite Roary if you use any of the results it produces:
    Andrew J. Page, Carla A. Cummins, Martin Hunt, Vanessa K. Wong, Sandra Reuter, Matthew T. G. Holden, Maria Fookes, Daniel Falush, Jacqueline A. Keane, Julian Parkhill,
    "Roary: Rapid large-scale prokaryote pan genome analysis", Bioinformatics, 2015 Nov 15;31(22):3691-3693
    doi: http://doi.org/10.1093/bioinformatics/btv421
    Pubmed: 26198102

2016/10/25 11:16:50 Looking for 'Rscript' - found /home/linuxbrew/.linuxbrew/bin/Rscript
2016/10/25 11:16:50 Determined Rscript version is 3.3
2016/10/25 11:16:50 Looking for 'awk' - found /usr/bin/awk
2016/10/25 11:16:50 Looking for 'bedtools' - found /home/linuxbrew/.linuxbrew/bin/bedtools
2016/10/25 11:16:50 Determined bedtools version is 2.26
2016/10/25 11:16:50 Looking for 'blastp' - found /home/linuxbrew/.linuxbrew/bin/blastp
2016/10/25 11:16:50 Determined blastp version is 2.5.0
2016/10/25 11:16:50 Looking for 'grep' - found /usr/bin/grep
2016/10/25 11:16:50 Looking for 'kraken' - found /home/linuxbrew/.linuxbrew/bin/kraken
Use of uninitialized value in concatenation (.) or string at /bio/perl5/lib/perl5/Bio/Roary/External/CheckTools.pm line 129.
2016/10/25 11:16:50 Determined kraken version is 
2016/10/25 11:16:50 Looking for 'kraken-report' - found /home/linuxbrew/.linuxbrew/bin/kraken-report
Use of uninitialized value in concatenation (.) or string at /bio/perl5/lib/perl5/Bio/Roary/External/CheckTools.pm line 129.
2016/10/25 11:16:50 Determined kraken-report version is 
2016/10/25 11:16:50 Looking for 'mafft' - found /home/linuxbrew/.linuxbrew/bin/mafft
Use of uninitialized value in concatenation (.) or string at /bio/perl5/lib/perl5/Bio/Roary/External/CheckTools.pm line 129.
2016/10/25 11:16:50 Determined mafft version is 
2016/10/25 11:16:50 Looking for 'makeblastdb' - found /home/linuxbrew/.linuxbrew/bin/makeblastdb
2016/10/25 11:16:50 Determined makeblastdb version is 2.5.0
2016/10/25 11:16:50 Looking for 'mcl' - found /home/linuxbrew/.linuxbrew/bin/mcl
2016/10/25 11:16:50 Determined mcl version is 14-137
2016/10/25 11:16:50 Looking for 'parallel' - found /home/linuxbrew/.linuxbrew/bin/parallel
2016/10/25 11:16:50 Determined parallel version is 20160922
2016/10/25 11:16:50 Looking for 'prank' - found /home/linuxbrew/.linuxbrew/bin/prank
2016/10/25 11:16:50 Looking for 'sed' - found /home/linuxbrew/.linuxbrew/bin/sed
2016/10/25 11:16:50 Looking for 'cd-hit' - found /home/linuxbrew/.linuxbrew/bin/cd-hit
2016/10/25 11:16:50 Determined cd-hit version is 4.6
2016/10/25 11:16:50 Looking for 'FastTree' - found /home/linuxbrew/.linuxbrew/bin/FastTree
2016/10/25 11:16:50 Determined FastTree version is 2.1
2016/10/25 11:16:50 Roary version 3.7.0
2016/10/25 11:16:50 Fixing input GFF files
2016/10/25 11:16:52 Extracting proteins from GFF files
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-00353/prokka/2014-00353.gff
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-11670/prokka/2014-11670.gff
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-14728/prokka/2014-14728.gff
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-14798/prokka/2014-14798.gff
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-19680/prokka/2014-19680.gff
sh: /tmp/STUDENTjkwong1/uQWglUdx_8: No such file or directory
Combine proteins into a single file
Cant open file: /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/roary/5mEEBPBGKX/2014-00353.gff.proteome.faa

I wonder if it is related to inconsistent referencing of $TMPDIR ?
In the output above:

sh: /tmp/STUDENTjkwong1/uQWglUdx_8: No such file or directory

Unfortunately, the way that our user accounts are set up, our usernames have a \ in the middle which is quite annoying.

$ echo $USER
STUDENT\jkwong1
$ echo $TMPDIR
/tmp/STUDENT\jkwong1
$ ls /tmp/STUDENT\\jkwong1/
uQWglUdx_8
$ ls /tmp/STUDENTjkwong1/
$ 

[andrewjpage]

Hi Jason,
For temp directories I use the a standard Perl module File::Temp, which
respects all the standard conventions. Its very widely used. My guess is
that your administrators have changed something (perhaps a new version of
Perl or File::Temp etc...)?
http://perldoc.perl.org/File/Temp.html

Can you manually set $TMPDIR to a subdirectory of your current working
directory and see does it fix it?
Regards,
Andrew

On 25 October 2016 at 10:19, Jason Kwong notifications@github.com wrote:

Hi Andrew,
Thanks very much for your work on Roary - it's fantastic and we use it
all the time.
Just recently (in the last week), I've been getting an error:

jkwong1@trillian:$ roary -a -p 16 -v gff/2014-*

Please cite Roary if you use any of the results it produces:
Andrew J. Page, Carla A. Cummins, Martin Hunt, Vanessa K. Wong, Sandra Reuter, Matthew T. G. Holden, Maria Fookes, Daniel Falush, Jacqueline A. Keane, Julian Parkhill,
"Roary: Rapid large-scale prokaryote pan genome analysis", Bioinformatics, 2015 Nov 15;31(22):3691-3693
doi: http://doi.org/10.1093/bioinformatics/btv421
Pubmed: 26198102

2016/10/25 11:16:50 Looking for 'Rscript' - found /home/linuxbrew/.linuxbrew/bin/Rscript
2016/10/25 11:16:50 Determined Rscript version is 3.3
2016/10/25 11:16:50 Looking for 'awk' - found /usr/bin/awk
2016/10/25 11:16:50 Looking for 'bedtools' - found /home/linuxbrew/.linuxbrew/bin/bedtools
2016/10/25 11:16:50 Determined bedtools version is 2.26
2016/10/25 11:16:50 Looking for 'blastp' - found /home/linuxbrew/.linuxbrew/bin/blastp
2016/10/25 11:16:50 Determined blastp version is 2.5.0
2016/10/25 11:16:50 Looking for 'grep' - found /usr/bin/grep
2016/10/25 11:16:50 Looking for 'kraken' - found /home/linuxbrew/.linuxbrew/bin/kraken
Use of uninitialized value in concatenation (.) or string at /bio/perl5/lib/perl5/Bio/Roary/External/CheckTools.pm line 129.
2016/10/25 11:16:50 Determined kraken version is
2016/10/25 11:16:50 Looking for 'kraken-report' - found /home/linuxbrew/.linuxbrew/bin/kraken-report
Use of uninitialized value in concatenation (.) or string at /bio/perl5/lib/perl5/Bio/Roary/External/CheckTools.pm line 129.
2016/10/25 11:16:50 Determined kraken-report version is
2016/10/25 11:16:50 Looking for 'mafft' - found /home/linuxbrew/.linuxbrew/bin/mafft
Use of uninitialized value in concatenation (.) or string at /bio/perl5/lib/perl5/Bio/Roary/External/CheckTools.pm line 129.
2016/10/25 11:16:50 Determined mafft version is
2016/10/25 11:16:50 Looking for 'makeblastdb' - found /home/linuxbrew/.linuxbrew/bin/makeblastdb
2016/10/25 11:16:50 Determined makeblastdb version is 2.5.0
2016/10/25 11:16:50 Looking for 'mcl' - found /home/linuxbrew/.linuxbrew/bin/mcl
2016/10/25 11:16:50 Determined mcl version is 14-137
2016/10/25 11:16:50 Looking for 'parallel' - found /home/linuxbrew/.linuxbrew/bin/parallel
2016/10/25 11:16:50 Determined parallel version is 20160922
2016/10/25 11:16:50 Looking for 'prank' - found /home/linuxbrew/.linuxbrew/bin/prank
2016/10/25 11:16:50 Looking for 'sed' - found /home/linuxbrew/.linuxbrew/bin/sed
2016/10/25 11:16:50 Looking for 'cd-hit' - found /home/linuxbrew/.linuxbrew/bin/cd-hit
2016/10/25 11:16:50 Determined cd-hit version is 4.6
2016/10/25 11:16:50 Looking for 'FastTree' - found /home/linuxbrew/.linuxbrew/bin/FastTree
2016/10/25 11:16:50 Determined FastTree version is 2.1
2016/10/25 11:16:50 Roary version 3.7.0
2016/10/25 11:16:50 Fixing input GFF files
2016/10/25 11:16:52 Extracting proteins from GFF files
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-00353/prokka/2014-00353.gff
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-11670/prokka/2014-11670.gff
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-14728/prokka/2014-14728.gff
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-14798/prokka/2014-14798.gff
Extracting proteins from /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/2014-19680/prokka/2014-19680.gff
sh: /tmp/STUDENTjkwong1/uQWglUdx_8: No such file or directory
Combine proteins into a single file
Cant open file: /home/jkwong1/Nmeningo/MenW/research/nullarbor.MenW-ALL.20161021/roary/5mEEBPBGKX/2014-00353.gff.proteome.faa

I wonder if it is related to inconsistent referencing of $TMPDIR ? In the
output above:

sh: /tmp/STUDENTjkwong1/uQWglUdx_8: No such file or directory

Unfortunately, the way that our user accounts are set up, our usernames
have a \ in the middle which is quite annoying.

$ echo $USER
STUDENT\jkwong1
$ echo $TMPDIR
/tmp/STUDENT\jkwong1
$ ls /tmp/STUDENT\jkwong1/
uQWglUdx_8
$ ls /tmp/STUDENTjkwong1/
$

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#287, or mute the thread
https://github.com/notifications/unsubscribe-auth/AABeV3I8x-KCttIlu3fhXfjYX-OZJJMtks5q3cmQgaJpZM4KfvtC
.

[tseemann]

The weird thing is this didn't use to fail. But it did since I upgraded to Roary 3.7.0 two days ago. Does cpan Bio::Roary force updating of File::Temp ? I don't think it would unless you changed minimum requirements?

The issue is that our logins are managed by ActiveDirectory and staff are in the default domain and students are in the STUDENT domain, so their getuid() and $USER are set to STUDENT\username. But the \ gets interpreted as an escape code when Roary (or File::Temp) use it in double-quotes like "$ENV{$TMPDIR}/foo" instead of $ENV{$TMPDIR}."/foo".

The simplest solution is for me to change my setting of TMPDIR in the login scripts; I will get it to remove the STUDENT\ prefix so this doesn't cause other problems.

[andrewjpage]

Any idea what version you were on previously? Or when you might have last updated? It will help narrow things down, because there have been only very small changes over the last few months, and none of them related to temp directories. Theres no version of File::Temp set in CPAN.One thing to note though is that File::Temp is part of core Perl since 5.20.2, before that it would have come from CPAN, so just double check nobody has done an update to Perl?

[JasonRafeMiller]

Hello Andrew. Was this File::Temp issue ever resolved? I may be having the same problem though I'm on CentOS. My roary version is the latest ("roary -w" says 1.007001). I always get the error, "BLAST Database error: No alias or index file found for protein database". Looking for the cause, I see that the blast database does not exist and it was makeblastdb that failed upstream. I logged the blast related commands and it looks like each command is using a different temp directory name. I see valid, non-empty protein fasta files getting created in one temp directory, but not the temp directory where makeblastdb runs. And, I don't know if this is cause or effect, but I see that my _clustered file is empty, and that is the file that roary passes to "makeblastdb -in". Thanks for your help.

[andrewjpage]

Hi Jason, Torsten has found that linuxbrew has a non functional version of blast 2.6.0 with makeblastdb segment faulting. The solution is to install blast through a different route. Andrew

…

On 16 Mar 2017 18:07, "JasonRafeMiller" ***@***.***> wrote: Hello Andrew. Was this File::Temp issue ever resolved? I may be having the same problem though I'm on CentOS. My roary version is the latest ("roary -w" says 1.007001). I always get the error, "BLAST Database error: No alias or index file found for protein database". Looking for the cause, I see that the blast database does not exist and it was makeblastdb that failed upstream. I logged the blast related commands and it looks like each command is using a different temp directory name. I see valid, non-empty protein fasta files getting created in one temp directory, but not the temp directory where makeblastdb runs. And, I don't know if this is cause or effect, but I see that my _clustered file is empty, and that is the file that roary passes to "makeblastdb -in". Thanks for your help. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#287 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABeV3xe4jToNpctoT7B_Ipl79uWpij7ks5rmXpogaJpZM4KfvtC> .

[tseemann]

@andrewjpage I have changed our login scripts to avoid our \-in-username problem but you can still fix it in Roary by changing "$ENV{$TMPDIR}/foo" to $ENV{$TMPDIR}."/foo".