schedule task to build genome index if one is not available #49
Conversation
| @shared_task | ||
| def build_genome_fragment_indices(genome_id): | ||
| genome = Genome.objects.get(pk=genome_id) | ||
| genome.indexed_genome() |
There was a problem hiding this comment.
Cool. So this is different from the build blastdb for the genome, right? also where is the index stored?
There was a problem hiding this comment.
The indices are per-fragment, and maps sequence bps to a bp numeric number. It's stored in the database.
| @@ -330,6 +330,10 @@ def on_get(self, request, genome_id): | |||
| args = q_parser.parse_args(request) | |||
| field = args['field'] | |||
|
|
|||
| if not genome.has_location_index: | |||
There was a problem hiding this comment.
What causes the genome index not built when it was first created? random failures?
There was a problem hiding this comment.
The indices are not required. In fact, because the indices are not shared between genomes, if we build one per fragment all the time, it would remove some of the storage advantage of Edge for engineered genomes. Hence, we don't actually create one when a new genome is created, only when someone go hit the genome on the website, or hit an API that requires it. The issue is that with RO server, if you hit there, you will never end up building the indices. So the solution here is to have building the indices be done via celery, on RO APIs. If a RO server receives an API that's meant for updating, then that's a caller error.
This allows a GET request to the RO server to trigger a celery task to build genome index, if one does not exist. Currently if there's no genome index, the GET call fails, and the index is not built, so subsequent calls also fail.