Error connecting to external database with mysqlimport

[wkpalan]

I am currently setting up Annoscript to run on SGE cluster at my university, I have to use an external server as mysql server to connect to Annocript.
Line 1585 from db_creater.pm encounters an error when connecting to external server.

    if ( try_exec_command ("mysqlimport -u$mySqlUser -p$mySqlPass $database $path_to_table --use-threads=$threads --local --fields-terminated-by='\\t' --lines-terminated-by='\\n'" ) < 1){

The error occurs because the host parameter is not passed to connect to external server. I have modified as follows to get it to connect to an external server. Notice addition of -h$host

 if ( try_exec_command ("mysqlimport -u$mySqlUser -p$mySqlPass -h$host $database $path_to_table --use-threads=$threads --local --fields-terminated-by='\\t' --lines-terminated-by='\\n'" ) < 1){

I have restarted the script and it will take some time before I can confirm if this corrected the issue. I will update asap.

[frankMusacchia]

Dear Gokul,

at this moment Annocript cannot be run on SGE clusters. Even if you will be
able to send the job, there is no qsub call in the code hence it will use a
single node.

Anyway, we never tried to run it on that, so let me know how it works.

Regards,

Francesco

2016-02-08 23:45 GMT+01:00 Gokul Wimalanathan notifications@github.com:

I am currently setting up Annoscript to run on SGE cluster at my
university, I have to use an external server as mysql server to connect to
Annocript
Line 1585 from db_createrpm encounters an error when connecting to
external server

if ( try_exec_command ("mysqlimport -u$mySqlUser -p$mySqlPass $database $path_to_table --use-threads=$threads --local --fields-terminated-by='\\t' --lines-terminated-by='\\n'" ) < 1){

The error occurs because the host parameter is not passed to connect to
external server I have modified as follows to get it to connect to an
external server Notice addition of -h$host

if ( try_exec_command ("mysqlimport -u$mySqlUser -p$mySqlPass -h$host $database $path_to_table --use-threads=$threads --local --fields-terminated-by='\t' --lines-terminated-by='\n'" ) < 1){

I have restarted the script and it will take some time before I can
confirm if this corrected the issue I will update asap

—
Reply to this email directly or view it on GitHub
#5.

Francesco Musacchia - Ph.D.
Computational Biology - Bioinformatics Core
Telethon Institute of Genetics and Medicine
Via Campi Flegrei 34, 80078 Pozzuoli (NA), Italy.
Tel. +39 081 19230692
Mobile: +39 349 6396351

[wkpalan]

I misspoke, we have a torque based cluster with high-memory/high-core nodes in the cluster, and I am trying to run this only on a single node, but with qsub. I think setting this up for large SGE cluster would take some effort, and thankfully I don't have to deal with that.

Do you have anyway that we can set it up so that we don't need an interactive qsub session to run it?

[frankMusacchia]

Dear Gokul,
This is something we Are planning for the future version of Annocript but you cannot do it right now. So far, We are constricted with the interactive version.

Regards,

Francesco

Il giorno 09 feb 2016, alle ore 16:40, Gokul Wimalanathan notifications@github.com ha scritto:

I misspoke, we have a torque based cluster with high-memory/high-core nodes in the cluster, and I am trying to run this only on a single node, but with qsub. I think setting this up for large SGE cluster would take some effort, and thankfully I don't have to deal with that.

Do you have anyway that we can set it up so that we don't need an interactive qsub session to run it?

—
Reply to this email directly or view it on GitHub.

[frankMusacchia]

Dear Gokul,

I was thinking that if you want you can try to change the behaviour of
Annocript to use the cluster by changing the line 273 (Annocript1.1.2). I
do not have time to try that right now but I will do ASAP.

Anyway at that point the configuration is completely checked and all the
modules will be started wth a system call. The parameters that I use
there could be passed using* -v parameter of *qsub for environment
variables and in the annocript_executor.pl http://annocript_executor.pl
script you can pick and use those.

I saw you are expert user, hence I thought to inform you about this.

Let me know if you need...

Best,

Francesco

2016-02-09 20:13 GMT+01:00 francescomusacchia@gmail.com:

Dear Gokul,
This is something we Are planning for the future version of Annocript but
you cannot do it right now. So far, We are constricted with the
interactive version.

Regards,

Francesco

Il giorno 09 feb 2016, alle ore 16:40, Gokul Wimalanathan <
notifications@github.com> ha scritto:

I misspoke, we have a torque based cluster with high-memory/high-core
nodes in the cluster, and I am trying to run this only on a single node,
but with qsub. I think setting this up for large SGE cluster would take
some effort, and thankfully I don't have to deal with that.

Do you have anyway that we can set it up so that we don't need an
interactive qsub session to run it?

—
Reply to this email directly or view it on GitHub
#5 (comment)
.

Francesco Musacchia - Ph.D.
Computational Biology - Bioinformatics Core
Telethon Institute of Genetics and Medicine
Via Campi Flegrei 34, 80078 Pozzuoli (NA), Italy.
Tel. +39 081 19230692
Mobile: +39 349 6396351

[wkpalan]

Hi Francesco,

I started by setting up the directory structure and everything using the interactive session, and while it was running blast I quit annocript.

I then used annocript_executor.pl script with the command line parameters necessary to get annocript running to completion using qsub and it worked well.

Thanks for pointing out the line. It also took a lot of hacking to get the scripts working with the environment outside the installation folder using environment modules. I have not kept track of what changes I made, but I should be able figure them out by diff if that will be useful for someone else.

Thanks

Gokul

[frankMusacchia]

Sounds really amazing: Annocript running on an HPC cluster just by hacking!

You are a pioneer... congratulations!

Whenever you have the time you can post this in the forum:
https://groups.google.com/forum/#!forum/annocript

Thanks a lot and Enjoy Annocript!

Francesco

2016-02-15 16:10 GMT+01:00 Gokul Wimalanathan notifications@github.com:

Hi Francesco,

I started by setting up the directory structure and everything using the
interactive session, and while it was running blast I quit annocript.

I then used annocript_executor.pl script with the command line parameters
necessary to get annocript running to completion using qsub and it worked
well.

Thanks for pointing out the line. It also took a lot of hacking to get the
scripts working with the environment outside the installation folder using
environment modules. I have not kept track of what changes I made, but I
should be able figure them out by diff if that will be useful for someone
else.

Thanks

Gokul

—
Reply to this email directly or view it on GitHub
#5 (comment)
.

Francesco Musacchia - Ph.D.
Computational Biology - Bioinformatics Core
Telethon Institute of Genetics and Medicine
Via Campi Flegrei 34, 80078 Pozzuoli (NA), Italy.
Tel. +39 081 19230692
Mobile: +39 349 6396351

[aburzynski]

Dear Gokul,
I would be very much interested in the details of your setup, I am trying to use similar configuration and would like to avoid obvious pitfalls... Any info will be appreciated,

j131

[wkpalan]

I am stretched for time right now. Will do it sometime over this week. I have to dig for my notes on this.

[AmaliT]

Hi @wkpalan

I am quite keen to hear how you manage to setup annocript on the server too. I was wondering if there is a location you put up your notes on this?

Cheers
A

[jsabinop]

Hi all,

For the past coule of years I have been working with Annocript on a desktop linux without any issues.
Recently I moved to another university and I no longer have the option to have a desktop and have to work from the university cluster.
After a couple of months of trouble shooting to make Annocript work on a cluster it finally works (runs to the end without giving any errors just like would in the previous desktop), however, when I check the folder of my job (where the log file says the output should be) there is no output, the folders Annocript creates are there, but there is nothing inside.

As anyone ever experienced such an issue? I know running from a cluster is in itself a source of issues, but I was wondering if anyone had this happen to them before.

Thanks in advace,

Cheers,
Joana