409 response to PUT /worker/task even when the worker is ready to accept more tasks.

[rosesyrett]

Overview

Currently through the cli there is no way of running more than one plan without restarting the server each time, because if you try to do this you will get a 409 response, indicating the worker is still busy.

Looking at the worker logs however it is not busy; there is a bug somewhere.

Steps to recreate

checkout main and type, in two terminals:

[user@terminal1] blueapi serve
[user@terminal2] blueapi controller run sleep '{"time": 1.0}'
{'task_id': '...'}
<some seconds later...>
[user@terminal2] blueapi controller run sleep '{"time": 1.0}'
Traceback (most recent call last):
  File "/venv/bin/blueapi", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/venv/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rose/Documents/projects/blueapi/src/blueapi/cli/cli.py", line 64, in wrapper
    func(*args, **kwargs)
  File "/venv/lib/python3.11/site-packages/click/decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rose/Documents/projects/blueapi/src/blueapi/cli/cli.py", line 99, in run_plan
    updated = client.update_worker_task(WorkerTask(task_id=task_id))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rose/Documents/projects/blueapi/src/blueapi/cli/rest.py", line 60, in update_worker_task
    return self._request_and_deserialize(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rose/Documents/projects/blueapi/src/blueapi/cli/rest.py", line 77, in _request_and_deserialize
    raise BlueskyRemoteError(str(response))
blueapi.cli.amq.BlueskyRemoteError: <Response [409]>

[rosesyrett]

I've found out why;

the PUT endpoint for 'worker/task' has the following logic:

if handler.worker.get_active_task() is not None:
    raise HTTPException(status_code=409, detail="Worker already active")
elif task.task_id is not None:
    handler.worker.begin_task(task.task_id)

This bug indicates the HTTP exception is raised even when the worker logs indicate it is active. The method for handler.worker.get_active_task just returns self._current, which is an Optional[TrackableTask[Task]] - i.e. if self._current is not None, this rest endpoint will raise this exception.

self._current is set to None upon initialisation but it is not unset after it is actually done in self._cycle. So this is a one line change.