Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

need to create a new queue for frontend modules that must be returned right away #132

Closed
kevinkle opened this issue Jun 30, 2017 · 3 comments

Comments

@kevinkle
Copy link
Member

this is a problem as modules like the database status views also use rq multiples which wont return anytime soon if there are long-running analyses also occurring

issue overlaps with #115

the new workers will likely become:

  1. priority > multiples > backlog
  2. priority > singles
  3. blazegraph
  4. priority
@kevinkle
Copy link
Member Author

I'm adding the priority queue to backlog and singles, but also creating a separate container with some workers solely handling priority (similar to the way blazegraph queue is setup)

@kevinkle
Copy link
Member Author

kevinkle commented Jul 4, 2017

Getting and error:

modules.beautify.beautify.beautify({'disable_amr': False, 'i': '/datastore/2017-06-30-21-53-27-595283-GCA_000023365.1_ASM2336v1_genomic.fna', 'disable_vf': False, 'disable_serotype': False, 'pi': 90, 'options': {'pi': 90, 'amr': False, 'serotype': True, 'vf': True}}, '/datastore/2017-06-30-21-53-27-595283-GCA_000023365.1_ASM2336v1_genomic.fna_rgi.tsv_rgi.p') from multiples
b50e9cb9-8f34-43dd-82c8-f43282147c67
Failed 3 days ago
Traceback (most recent call last):
  File "/opt/conda/envs/backend/lib/python2.7/site-packages/rq/worker.py", line 700, in perform_job
    rv = job.perform()
  File "/opt/conda/envs/backend/lib/python2.7/site-packages/rq/job.py", line 500, in perform
    self._result = self.func(*self.args, **self.kwargs)
  File "./modules/beautify/beautify.py", line 124, in beautify
    json_r = check_alleles(json_r)
  File "./modules/beautify/find_widest.py", line 126, in check_alleles
    raise Exception('The Panadas DF from gene_dict is empty.')
Exception: The Panadas DF from gene_dict is empty.

The same error happened in the past when rgi wasnt run before beautifying. However, it looks like the pickle dump /datastore/2017-06-30-21-53-27-595283-GCA_000023365.1_ASM2336v1_genomic.fna_rgi.tsv_rgi.p isnt empty.

@kevinkle
Copy link
Member Author

kevinkle commented Jul 4, 2017

Added in #137

@kevinkle kevinkle closed this as completed Jul 4, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant