feat(TaskProcessingApi): Add endpoint for getting the next task

[provokateurin]

Summary

The ExApps need a way to get the next task.
@marcelklehr I think there is a bug as the tasks are not marked as running when the getNextScheduledTask method of the service is called (or maybe it was also intentional and the endpoint should do it manually).

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)

[marcelklehr]

Ah, I missed this, yes.

@marcelklehr I think there is a bug as the tasks are not marked as running when the getNextScheduledTask method of the service is called (or maybe it was also intentional and the endpoint should do it manually).

Yeah, let's do this manually directly in the endpoint

[bigcat88]

We need endpoint for getting nextTask by specifying multiple task_types.
One provider can process multiple task types and we do not want to spam NC with one nextTask request for each task_type.

Or just to change string $taskTypeId to array of strings.

[julien-nc]

• I agree with @marcelklehr that it makes more sense to change the task status to "running" in the controller method.
  The Manager::getNextScheduledTask method might be used in some context where we don't want to change to task status.

• Also, how about having a setTaskRunning parameter (with true as default value) in TaskProcessingApiController::getNextScheduledTask ? Just in case one wants to get the next scheduled task without actually processing it.

• About requesting multiple task types in getNextScheduledTask, much needed indeed. @marcelklehr Would you go with another endpoint or change the current one?

[provokateurin]

I don't see a reason why you would ever need to know the next task without also running it at the same time, but I can change it if you really want 🤷‍♀️

[marcelklehr]

I don't see a reason why you would ever need to know the next task without also running it at the same time

The synchronous background job will re-schedule itself based on the fact whether there is another job waiting to be executed.

[marcelklehr]

Would you go with another endpoint or change the current one?

We can add this to the current endpoint.

[bigcat88]

ExApp needs to call these APIs to process tasks:

getNextScheduledTask
getFileContents
setProgress
setResult

more precisely, there should be two versions of getFileContents - one for the UI with userId and other one for ExApp with [ExAppRequired] without checking for userId

cause only having those 4 endpoint ExApp can have no "system" flag to process tasks from what I see

[nickvergessen]

The security middleware looks good now

[bigcat88]

I didn't change the logic inside getNextScheduledTask - but it doesn't seem right to me for these reasons:

1. $task->setStatus(Task::STATUS_RUNNING); won't write anything to the database, right?
2. Reading and writing to the database must be atomic within “getNextScheduledTask” (so that two requests for processing a task in parallel do not receive the same task)
3. I didn’t understand the logic at all why $providerIds is there as input and what it should do, that’s why I didn’t change the entry to atomic inside “getNextScheduledTask”

[marcelklehr]

Ah, yes, good point @bigcat88, this needs to be persisted

[marcelklehr]

Suggested change

	*
	*
	* It could be that a provider is looking for new tasks but the task type of the next scheduled task is actually configured to be run with a different provider

[marcelklehr]

Imagine the task queue is as follows

1. free_prompt
2. summarize

and the configuration for the preferred providers says

free_prompt: context_chat
summarize: llm2-llama-3

Then llm2 calls this endpoint with getNextScheduledTask([free_prompt, summarize], [llm2-llama-3])

What will happen is:

• getNextScheduledTask will return the free_prompt task
• But that is configured to be run by context_chat
• So, the endpoint returns NotFoundResponse, even though there is the summarize task waiting that fulfills the requirements of the request