[20.09] Fix mule job messaging

[mvdbeek]

Handler assignment via mule messaging passes around the job id, so we need to flush before we can assign.
Should fix #11146.

[AjitPS]

Ah, I tried updating my Galaxy from 19.01 to 20.09 and all works but jobs stuck in "new" state... we use Mule handlers too so hopefully pulling this will fix it. thanks

[mvdbeek]

@nsoranzo the circle-ci setup breaks when changing the base. Let me see if a rebase fixes that

[AjitPS]

would be good to have this fix. thanks!

[AjitPS]

awaiting this to be merged. would be great to have Mule job messaging and job submission resolved

[AjitPS]

Thanks both.
FYI, I am trying to update our institute's Galaxy (since mid- last week) from v19.01 to v20.09. I pulled the release_20.09 code branch on Jan. 13th, and got it up and running on our end. Our galaxy 19.01 used to submit jobs via Slurm to our HPC and this job submission now stalls (stuck in "new" state).

I contacted @nsoranzo who suggested pulling this fix #11148 (pulled this commit only which has 2 files changed) but same issues persist.

I also noticed in .logs that 3 existing shed_tools (plink, vcftools-filter, hmmer_hmmalign) throwing continuous errors of missing datatypes and code exceptions while the tool panel is being populated. perhaps as they are not python3.

also get:
galaxy.tools.parameters.basic WARNING 2021-01-19 09:06:06,812 Datatype class not found for extension 'text', which is used in the 'format' attribute of parameter 'input_keep_ids'

Not sure how to proceed as my galaxy.yml, job_conf.xml etc look good and have restarted galaxy since pulling the fix too but my test jobs (very simple tools like: count lines in file, etc) are still stuck in "new" state. main Galaxy log shows job_id but handlers empty... not sure what I'm missing.

[AjitPS]

e.g., on launching a test job:

galaxy_web_0.log:galaxy.web_stack.transport DEBUG 2021-01-19 09:57:07,036 [p:32189,w:1,m:0] [uWSGIWorker1Core0] Sending message to farm job-handlers: {"target": "job_handler", "params": {"task": "setup", "job_id": 175843}, "__classname__": "JobHandlerMessage"}

but handler logs empty and jobs_directory has no new dir.

[mvdbeek]

@AjitPS please open an issue and describe your setup, including python version used and your job handler setup. The message your seeing here means the PR works.