Updating script that loads organisms and gene info into the database, as well as requirements file, to include latest version of django-organisms and django-genes #8

rzelayafavila · 2017-11-08T22:21:02Z

No description provided.

…h', to make the name clearer - also formatting and cleaning up this file, adding commands to load gene history, and removing commands for loading GO, KEGG and DO sets, as those are processed by the annotation-refinery now. Also removing update_tribe.sh file, since it was unnecessary.

…ase.txt.

…nstallation process.

dongbohu

I reviewed the PR and left a few comments. Using "for" loop will definitely reduce the number of lines and make the code more compact. I think it will be a good idea in the long run. If you want to close this PR as soon as possible, I strongly recommend that you add an issue to fix it later.

dongbohu · 2017-11-09T18:55:31Z

load_organisms_and_genes_to_db.sh

+python manage.py organisms_create_or_update --taxonomy_id=208964 \
+    --scientific_name="Pseudomonas aeruginosa" --common_name="Pseudomonas aeruginosa"
+
+#CREATE XRDBS


Did you use "#CREATE XRDBS" (w/o a space char after #) on purpose? (Same format is found in other comments lines below too.) If not, you may want to add the space char to make the comment style consistent.

No - that was actually legacy stuff from the previous files, I used a better format in the new comments, but I will change this.

dongbohu · 2017-11-09T18:58:34Z

load_organisms_and_genes_to_db.sh

@@ -0,0 +1,244 @@
+# Create organism records


If you are sure that this script will be run with sh or bash, a line like this at the beginning will be helpful:
#!/bin/sh
or

#!/bin/bash

Yep, good idea, I will add that

My bad. The second line of code should be #!/bin/bash. I have corrected it.

Ah good call, thanks!

dongbohu · 2017-11-09T19:02:03Z

load_organisms_and_genes_to_db.sh

@@ -0,0 +1,244 @@
+# Create organism records
+python manage.py organisms_create_or_update --taxonomy_id=9606 \


Not sure whether it will be better to write a for loop to deal with these commands. What do you think?

dongbohu · 2017-11-09T19:03:23Z

load_organisms_and_genes_to_db.sh

+    --scientific_name="Pseudomonas aeruginosa" --common_name="Pseudomonas aeruginosa"
+
+#CREATE XRDBS
+python manage.py genes_add_xrdb --name=Ensembl \


Same question: What about a "for" loop?

Actually for this one, would it really make sense to create a for loop? According to the link you posted, I would need to declare an associative array for each of the xrdbs, which would take as many lines as just calling these commands, no?

dongbohu · 2017-11-09T19:05:52Z

load_organisms_and_genes_to_db.sh

+# database for each organism
+
+#HUMAN
+wget -P data/ -N \


"for" loop?

dongbohu

I forgot to tell you before: when I deployed tribe on my desktop, I had to change line 28 of requirements/base.txt from:
psycopg2==2.6
to:
psycopg2==2.7
@mhuyck reported this issue some time ago when he deployed adage-server.

dongbohu · 2017-11-09T19:30:21Z

I did some research on how easy it is in bash to define an array of dictionaries to be used in for loop. Bash 4.0 (released in 2009) introduced associative array, which makes the implementation easy. See this example:
https://stackoverflow.com/questions/1494178/how-to-define-hash-tables-in-bash

…rganisms_and_genes_to_db.sh file, as per code review comments.

rzelayafavila · 2017-11-10T16:47:29Z

@dongbohu - I started to implement the for-loops with the associative array as you suggested, but I feel like it is a more involved process (with logic that is not as trivial), and I don't know how much it would help at this point, so I think I will open an issue for it. Is that ok with you?

rzelayafavila · 2017-11-10T19:20:15Z

Ok @dongbohu, I think I have addressed everything. I opened up issue #9 to address the converting of commands into for-loop as per our conversations here and in person. Let me know how it looks, thanks!

dongbohu · 2017-11-10T19:27:51Z

Did you see my comment on updating psycopg2 from 2.6 to 2.7 in requirements/base.txt?

dongbohu · 2017-11-10T19:29:22Z

About using for loop in bash, I agree its syntax seems a little weird. Let me think about it. You don't need to do it now.

rzelayafavila · 2017-11-10T19:29:46Z

Ah yes, I did, and I made the change to the psycopg2 version, I had just forgotten to commit it and push it, good catch! Just pushed it.

dongbohu · 2017-11-10T19:35:55Z

When I was a Unix system administrator (~2002), sometimes I wrote C++ programs to generate shell script that had a lot of boilerplate lines, because sh/bash/csh/tcsh was not powerful enough. It was more like a workaround, not a good long-term solution.

dongbohu

Looks good.

rzelayafavila · 2017-11-10T19:41:24Z

Thanks! I agree with your comment about not writing other programs to generate the boilerplate shell commands, as it is not a long-term solution.

rzelayafavila added 3 commits November 8, 2017 17:12

Updating django-organisms and django-genes versions in requirements/b…

4db9b29

…ase.txt.

Removing 'json' line from '.bowerrc' file, since it broke the bower i…

6245c1b

…nstallation process.

rzelayafavila requested review from mhuyck and dongbohu November 8, 2017 22:21

ghost assigned rzelayafavila Nov 8, 2017

ghost added the code review label Nov 8, 2017

dongbohu reviewed Nov 9, 2017

View reviewed changes

rzelayafavila added 2 commits November 9, 2017 16:28

Correct formatting, and adding a few lines at the beginning of load_o…

6a7f078

…rganisms_and_genes_to_db.sh file, as per code review comments.

Make load_organisms_and_genes_to_db.sh script executable.

bdfc6c9

rzelayafavila mentioned this pull request Nov 10, 2017

Turn commands in 'load_ortanisms_and_genes_to_db.sh' script into for-loops, to make it shorter and more maintainable. #9

Open

Updating version of psycopg2 in requirements/base.txt

f02ca8d

dongbohu approved these changes Nov 10, 2017

View reviewed changes

rzelayafavila merged commit 8b8c194 into greenelab:master Nov 10, 2017

ghost removed the code review label Nov 10, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating script that loads organisms and gene info into the database, as well as requirements file, to include latest version of django-organisms and django-genes #8

Updating script that loads organisms and gene info into the database, as well as requirements file, to include latest version of django-organisms and django-genes #8

rzelayafavila commented Nov 8, 2017

dongbohu left a comment

dongbohu Nov 9, 2017

rzelayafavila Nov 9, 2017

dongbohu Nov 9, 2017 •

edited

Loading

rzelayafavila Nov 9, 2017

dongbohu Nov 9, 2017

rzelayafavila Nov 9, 2017

dongbohu Nov 9, 2017

dongbohu Nov 9, 2017

rzelayafavila Nov 10, 2017

dongbohu Nov 9, 2017

dongbohu left a comment

dongbohu commented Nov 9, 2017

rzelayafavila commented Nov 10, 2017

rzelayafavila commented Nov 10, 2017

dongbohu commented Nov 10, 2017

dongbohu commented Nov 10, 2017

rzelayafavila commented Nov 10, 2017 •

edited

Loading

dongbohu commented Nov 10, 2017

dongbohu left a comment

rzelayafavila commented Nov 10, 2017

		@@ -0,0 +1,244 @@
		# Create organism records
		python manage.py organisms_create_or_update --taxonomy_id=9606 \

Updating script that loads organisms and gene info into the database, as well as requirements file, to include latest version of django-organisms and django-genes #8

Updating script that loads organisms and gene info into the database, as well as requirements file, to include latest version of django-organisms and django-genes #8

Conversation

rzelayafavila commented Nov 8, 2017

dongbohu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dongbohu Nov 9, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dongbohu left a comment

Choose a reason for hiding this comment

dongbohu commented Nov 9, 2017

rzelayafavila commented Nov 10, 2017

rzelayafavila commented Nov 10, 2017

dongbohu commented Nov 10, 2017

dongbohu commented Nov 10, 2017

rzelayafavila commented Nov 10, 2017 • edited Loading

dongbohu commented Nov 10, 2017

dongbohu left a comment

Choose a reason for hiding this comment

rzelayafavila commented Nov 10, 2017

dongbohu Nov 9, 2017 •

edited

Loading

rzelayafavila commented Nov 10, 2017 •

edited

Loading