Unify task_get_pag_by_qc_faster and task_get_pag_by_qc

[SamStudio8]

-- 20210128
Discussion below

[SamStudio8]

Bumped to next as this blocks COG-UK/dipi-group#17

[SamStudio8]

So, we have:

• task_get_pag_by_qc - the original endpoint
• task_get_pag_by_qc_v3 - the v3 implementation for GET PAG (only used for secure PHA queries currently)
• task_get_pag_by_qc_faster - the faster implementation for fetching pagfiles only
• task_api_get_pags_to_publish - the fastest implementation for fetching explicit information about PAGs

The goal(s) I think are:

• Deprecate and remove the original task_get_pag_by_qc endpoint
  • Check whether this endpoint is relied on by Elan and friends
  • Check the fields are all included in at least one of the other interfaces
  • Update any URLs or old code that may need to direct to a newer task implementation
  • Remove the function
• Keep the recently introduced "mode" to the GET PAG interface and use this to:
  • Offer a pagfiles mode in task_api_get_pags_to_publish and deprecate the task_get_pag_by_qc_faster interface
  • Generisize the task_api_get_pags_to_publish interface for all QC matrix options

[SamStudio8]

• Deprecate or tidy task_get_pag_by_qc_faster c28bd7a
• Remove hard coded publishing requirements from task_api_get_pags_to_publish d7d9877
• Deprecate task_get_pag_by_qc e442b32, 2aad90c
• Ensure all fields from task_get_pag_by_qc are available in the task_api_get_pags_to_publish superset 2aad90c
• Rename task_get_pag_by_qc_faster to something sensible: task_get_pagfiles beba3d7
• Remove the "second" PAG GET endpoint beba3d7
• Ensure all references to task_get_pag_by_qc correctly forward tasks to task_api_get_pags_to_publish
• Rename task_api_get_pags_to_publish to something like task_get_pag_v2 d221cdf
• Update Ocarina to handle appropriate changes (SamStudio8/ocarina2@4e935a4)
• Solidify the GET PAG API and document it
• Remove code to filter PAGs and place them in the calling view, not the task. Task should accept list of PAG ids.
• Deprecate the serpy PAGQCSerializer and PAGSerializer. Keep these for now.

Simultaneous changes to implement in production:

• Push to Majora prod (Friday AM)
• Update Ocarina anywhere that fetches PAGs (eg. samstudio8 conda control env) (Friday AM)
• Switch get pagfiles call in Elan post-publish (https://github.com/SamStudio8/elan-nextflow/blob/61ecaa5b0e9e08b89af84f9e47dc068d7a147d5c/bin/control/cog-publish.sh#L19) (Friday AM - compared to Thursday PM)
• ocarina-get-gisaid.sh, fix broken column names (below) (before Monday)
• ocarina-get-ena.sh ocarina-get-ena-assembly.sh (before Monday)
• Likely affects plans for Asklepian so be aware (NA)