Stuck on Thinking... when request send

[oboeglen]

Which version of assistant are you using?

2.1.1

Which version of Nextcloud are you using?

30.0.1

Which browser are you using? In case you are using the phone App, specify the Android or iOS version and device please.

Chrome 129.0.6668.101

Describe the Bug

Hello,

As soon as a request is sent via the Nextcloud assistant, it remains stuck in processing, and I get no response from my request. However, on my OpenAI account, I can see that tokens are indeed being used, as shown in the screenshot.

Thank you in advance for your help.

Expected Behavior

A response

To Reproduce

Open Nextcloud Assistant and use Chat with AI or Text Generation

[cheesynator]

Watch at [Bug]: No answer from KI in Assistent #142

The answer is always given every 5 minutes e.g. 13.05, 13.10
What surprises me is that the bug has been reported for several weeks but there have been no concrete answers regarding a fix.

[julien-nc]

Hi, this is because since NC 30, tasks are running in a background job. So there is a delay, you wait until the next cron.php execution.
This delay can be avoided by running an occ command that executes a specific background job type ASAP. More info at https://docs.nextcloud.com/server/latest/admin_manual/ai/overview.html#ai-overview-improve-ai-task-pickup-speed

[oboeglen]

Hi, this is because since NC 30, tasks are running in a background job. So there is a delay, you wait until the next cron.php execution. This delay can be avoided by running an occ command that executes a specific background job type ASAP. More info at https://docs.nextcloud.com/server/latest/admin_manual/ai/overview.html#ai-overview-improve-ai-task-pickup-speed

Thank you for the information.

Could you clarify if this service implementation affects the integration generally, or is it only relevant in the case of a LocalAI installation? I am encountering this issue specifically with OpenAI, not in a local setup.

[oboeglen]

Indeed, with the implementation of the service, OpenAI is working again! Thank you very much for this information!

[julien-nc]

There are 2 types of AI providers in NC. The asynchronous and synchronous ones.
The async ones are in charge of asking NC regularly for a new task to consume.
The sync ones are executed in background jobs.

Usually, providers that are implemented in a classic Php NC app are synchronous (integration_openai, integration_replicate for example). So you need the occ background-job:worker "OC\TaskProcessing\SynchronousBackgroundJob" running if you want to avoid the cron.php delay.

In your case, it does not matter if you configured integration_openai to use LocalAI or OpenAI's API. The app works the same, it is a synchronous provider.

[oboeglen]

There are 2 types of AI providers in NC. The asynchronous and synchronous ones. The async ones are in charge of asking NC regularly for a new task to consume. The sync ones are executed in background jobs.

Usually, providers that are implemented in a classic Php NC app are synchronous (integration_openai, integration_replicate for example). So you need the occ background-job:worker "OC\TaskProcessing\SynchronousBackgroundJob" running if you want to avoid the cron.php delay.

In your case, it does not matter if you configured integration_openai to use LocalAI or OpenAI's API. The app works the same, it is a synchronous provider.

One last question, regarding Docker with the Nextcloud image—how can I set up this service? I'm specifying that it's not the AIO image, but the community image.

[julien-nc]

You can run the occ command with docker run or docker compose run in the service script.

[oboeglen]

You can run the occ command with docker run or docker compose run in the service script.

Ok thanks, does the service start indefinitely with the docker run command? Sorry if I don't understand well.

[julien-nc]

This occ command will not exit immediately but rather continue living and execute any background job when it sees a new one appear.

There's a -t param to specify a max execution time to be able to restart this process from time to time. Restarting it is necessary because inside this process, any change to the NC setting values won't be seen since these values are only loaded at the start of the occ Php process.

So the recommended way to run this command is to launch it in a systemd (or equivalent) service with a timeout (stop after param) of a minute or so and to configure the service to restart itself when it stops. So the service restart mechanism is the loop. The service runs the occ command once. The occ command dies after some time, the service restarts etc...

Is that a bit clearer?

[oboeglen]

This occ command will not exit immediately but rather continue living and execute any background job when it sees a new one appear.

There's a -t param to specify a max execution time to be able to restart this process from time to time. Restarting it is necessary because inside this process, any change to the NC setting values won't be seen since these values are only loaded at the start of the occ Php process.

So the recommended way to run this command is to launch it in a systemd (or equivalent) service with a timeout (stop after param) of a minute or so and to configure the service to restart itself when it stops. So the service restart mechanism is the loop. The service runs the occ command once. The occ command dies after some time, the service restarts etc...

Is that a bit clearer?

Thanks for the infos, very helpfull !