Mac compatibility

[aassie]

Finally getting the time to test your tools 😉
Here are a bunch of notes about issues I encountered trying to run gapseq on my Mojave 10.14.6

Mac users need to have coreutils and gnu binutils installed otherwise you will have the following errors:
readlink: illegal option -- f
stat: illegal option -- c

Both can be installed with homebrew and symlinked with:

ln -s /usr/local/bin/gstat /usr/local/bin/stat
and
ln -s /usr/local/bin/greadlink /usr/local/bin/readlink

[aassie]

Same thing with grep
ln -s /usr/local/bin/ggrep /usr/local/bin/grep

[aassie]

Maybe it would be good to have your mother script having set -e to stop the whole process on the first error occurring ;)

[aassie]

Mac tutorial is missing the install link for exonerate
Can be done with conda
conda install -c bioconda exonerate

[jotech]

Hi Adrien, cool this is great help indeed! Haven't so much Mac contact myself so so these hints are highly appreciated :)
I found another hint here. I'm wondering if just setting the $PATH variable by
export PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"
can also works? It avoids setting symlinks which would override OSX versions of stat, readlink and grep. What do you think?
Best wishes johannes

[aassie]

The issue is that when you install coreutils, grep and binutils, brew installs the command with the prefix "g".

So I had all of them installed already but to call them I am now used to ggrep, gsed etc..
That's why I used the symlink to have to script call the right command.

I am far to be an expert and I don't really know the best solution :/

[jotech]

I agree! As far as I got it, the problem is that the GNU version of the tools are not fully compatible with the BSD versions which are used in by OSX since it originates from BSD Unix.
In the /usr/local/opt/grep/libexec/gnubin:$PATH directory there should be the files with the usual name (without "g") and by setting the $PATH you add temporary higher priority to this folder.
The symlink would set priority to the GNU versions permanently. Maybe not all users agree with it?
I will try to get hands on a OSX machine and try to come out with a fix :)

[feiranl]

Met the same problem using MAC, error shows here:
readlink: illegal option -- f usage: readlink [-n] [file ...] readlink: illegal option -- f usage: readlink [-n] [file ...] readlink: illegal option -- f usage: readlink [-n] [file ...] Invalid file: toy/myb71.fna.gz

[feiranl]

Even though I followed the above instructions, and when I run the : gapseq find-transport toy/myb71.fna.gz, I still encounter the error, adn the result file is empty.

Found transporter for:
   1 

Found transporter and import reactions for:


No transport reactions found in database for:

No transporter found for compounds (existing transporter/exchanges should be removed?):
grep: -: No such file or directory

Running time: 306 s

[jotech]

hi @feiranl, thanks for trying gapseq on a mac! You are right, installation is a bit bumpy on ios, it's because we ourselves are linux user and unfortunately some command line programs used by gapseq differ under ios.

Have you tried the solution proposed by aassie?
brew install coreutils binutils

[feiranl]

Hi, @jotech,

That's what I have done for setting up.

brew install git glpk blast bedtools r brewsci/bio/barrnap
R -e 'install.packages(c("data.table", "stringr", "sybil", "getopt", "reshape2", "doParallel", "foreach", "R.utils", "stringi", "glpkAPI"), repos=c("http://cran.us.r-project.org","http://cran.rstudio.com"))'
R -e 'if (!requireNamespace("BiocManager", quietly = TRUE)) install.packages("BiocManager", repos=c("http://cran.us.r-project.org","http://cran.rstudio.com")); BiocManager::install("Biostrings")'
export
brew install coreutils 
brew install binutils
brew install wget
conda install -c bioconda exonerate
ln -s /usr/local/bin/greadlink /usr/local/bin/readlink
ln -s /usr/local/bin/gstat /usr/local/bin/stat
ln -s /usr/local/bin/ggrep /usr/local/bin/grep
export PATH="[mygapseq-dir]:$PATH" 

[jotech]

hm .. I remember we had some problems with transporter prediction and different awk versions in the past e8da324
Unfortunately, I don't have access to a ios system. Do you mind trying:
brew install awk
(and maybe also: ln -s /usr/local/bin/gawk /usr/local/bin/awk or something similar)

[feiranl]

Hi,

I tried, but this doesn't work out.

The message is the same as before: and the outfile is empty, is it related to the grep, because in the message it said :grep: -: No such file or directory. May I know which example can I also run to test for?

Found transporter for:
   1 

Found transporter and import reactions for:


No transport reactions found in database for:

No transporter found for compounds (existing transporter/exchanges should be removed?):
grep: -: No such file or directory

Running time: 306 s

But I tested grep using the code below:

echo 'dog eat dog' > dog
grep 'd' dog

it returned: dog eat dog , which suggests that grep is ok?

[jotech]

Thanks for this hint! If it is grep (which differs under mac as I just read) then you should already be able to use the linux grep command because you have installed the coreutils.
Do you mind trying if symlinking, as proposed before by aassie, could help?
ln -s /usr/local/bin/ggrep /usr/local/bin/grep

[feiranl]

Thanks! But I have already ran this before. Rerun it again:
ln -s /usr/local/bin/ggrep /usr/local/bin/grep
output: ln: /usr/local/bin/grep: File exists

And rerun the transporter annotation code, it still errors.

[jotech]

strange! Could you please report the output of whereis grep and run /path/to/grep -V with all grep version?

[feiranl]

whereis grep outputs /usr/bin/grep but /path/to/grep -V outputs -bash: /path/to/grep: No such file or directory

[jotech]

Can you please try /usr/bin/grep -V?

[feiranl]

yes, the output is grep (BSD grep) 2.5.1-FreeBSD

[jotech]

ok thanks for this, I think we are getting closer :)
I'm wondering, you also wrote in a former post that /usr/local/bin/grep already exists, right?
What is the output of /usr/local/bin/grep -V and echo $PATH?

[feiranl]

Thanks! The output for /usr/local/bin/grep -V is -bash: /usr/local/bin/grep: No such file or directory, and the output for echo $PATH is /Users/feiranl/Documents/GitHub/gapseq:/opt/anaconda3/bin:/opt/anaconda3/condabin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/ncbi/blast/bin:/usr/local/munki

[jotech]

the /usr/local/bin/grep: No such file or directory error is a bit strange because you wrote that ln -s /usr/local/bin/ggrep /usr/local/bin/grep was giving the error ln: /usr/local/bin/grep: File exists. Could you please double check, which one is right?

[feiranl]

Sorry for the trouble. I double checked that the grep exists in the path /usr/local/bin/grep. Below is the code that I cd to the /usr/local/bin/ to check whether grep exists or not.
(base) dhcp-141-14:bin feiranl$ grep

usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
	[-e pattern] [-f file] [--binary-files=value] [--color=when]
	[--context[=num]] [--directories=action] [--label] [--line-buffered]
	[--null] [pattern] [file ...]

(base) dhcp-141-14:bin feiranl$ grep -V
grep (BSD grep) 2.5.1-FreeBSD
(base) dhcp-141-14:bin feiranl$ pwd
/usr/local/bin

[jotech]

I'm afraid they way you used 'cd' to get to /usr/local/bin and call grep will use the default grep from /usr/bin/grep.

I think we could try two things:

• if /usr/local/bin/grep doesn't exist, as you said, then ln -s /usr/local/bin/ggrep /usr/local/bin/grep should work
• your $PATH variable prioritizes /Users/feiranl/Documents/GitHub/gapseq , thus you could also simply create a grep symlink in the gapseq folder itself: ln -s /usr/local/bin/ggrep /Users/feiranl/Documents/GitHub/gapseq/grep

I'm sorry if it is a bit confusing, but I would like to find a quick solution first and look for a more user-friendly alternative afterwards, when I know it can work in principle.

[feiranl]

Hi, I managed to get the grep version by setting the symlink using the code as below:

(base) dhcp-141-14:bin feiranl$ ln -s /usr/bin/grep /usr/local/bin/grep
(base) dhcp-141-14:bin feiranl$ /usr/local/bin/grep -V

output: grep (BSD grep) 2.5.1-FreeBSD

I tried to run the transporter annotation again, but is still failed. BTW: is this normal for outputting below message in the process for transporter annotation?

** FATAL ERROR **: Unrecognised symbol 'J' (ascii:74) file:[all.fasta] seq:[gnl|TC-DB|KXJ16754.1|1.C.112.1.6] pos:[16]
exiting ..

[jotech]

/usr/bin/grep is the wrong source, you need the one which was installed by the coreutils package:

rm /usr/local/bin/grep
ln -s /usr/local/bin/ggrep /usr/local/bin/grep

It should work when grep -V is giving you something like grep (GNU grep) 3.4

[feiranl]

But there is not a ggrep in the location of /usr/local/bin. Is this should be installed by coreutils package? I do have the coreutils installed.

 Warning: coreutils 8.32 is already installed and up-to-date
To reinstall 8.32, run `brew reinstall coreutils`
Warning: binutils 2.35.1 is already installed and up-to-date
To reinstall 2.35.1, run `brew reinstall binutils`

[jotech]

oh I see, this explains the ongoing misunderstanding ^^, sorry for the confusion!
found a hint here:

brew install grep

should do the trick and afterwards the symlink should work :)

[feiranl]

Hi, now it works fine. Thanks!

I summarized steps that I have done to install gapseq in MAC as below.
BTW: repos repos=c("http://cran.us.r-project.org","http://cran.rstudio.com")) for installing R packages are also needed, in my case without that, an error was reported.

brew install git glpk blast bedtools r brewsci/bio/barrnap coreutils binutils wget grep
R -e 'install.packages(c("data.table", "stringr", "sybil", "getopt", "reshape2", "doParallel", "foreach", "R.utils", "stringi", "glpkAPI"), repos=c("http://cran.us.r-project.org","http://cran.rstudio.com"))'
R -e 'if (!requireNamespace("BiocManager", quietly = TRUE)) install.packages("BiocManager", repos=c("http://cran.us.r-project.org","http://cran.rstudio.com")); BiocManager::install("Biostrings")'
conda install -c bioconda exonerate
ln -s /usr/local/bin/greadlink /usr/local/bin/readlink
ln -s /usr/local/bin/gstat /usr/local/bin/stat
ln -s /usr/local/bin/ggrep /usr/local/bin/grep
git clone https://github.com/jotech/gapseq && cd gapseq
export PATH="[gapseq-dir]:$PATH" 

[jotech]

Hi @feiranl , i'm glad it worked!
in future version, we will try to integrate the different locations of the gnu on mac so that the symlink workaround will not be needed anymore.

[feiranl]

Looking forward! Thanks!

[Waschina]

Dear MacOS user,

together with students we have tested the installation of gapseq on different MacOS-Systems. We realized, that the easiest solution is to install gapseq in a separate conda environment. We provide step-by-step instructions how to do that here: https://gapseq.readthedocs.io/en/latest/install.html

Best wishes
Silvio