Kolibri shutdown blocked by running cancellable task #10249
Comments
|
@bjester I would like to work on this issue |
|
Hi @photon0205, I have assigned you. First step would be to replicate this error (it looks like @bjester was importing resources for a channel). The next step would be to look at the worker implementation and see how it handles shutdown to see why his might be happening. |
|
We run into this all the time if there's a worker downloading content from a task. In the log it reports that the job is being cancelled, but nothing actually happens. First, the task is received and it starts downloading the items: Then I interrupt the server to stop it and it says it's canceling the task: But the downloads keep running. Finally when they're done the server shuts down: |
|
@rtibbles what do you think is going on here? Does the |
|
Oh, Which resulted in: This stack overflow post has some more details about it. Presumably this is because you can't cancel a thread. You could workaround it by using a process pool, but you'd have to have a custom executor to kill the running processes. I guess to do this generically in Kolibri there would have to be a separate mechanism to signal a running task that it needs to stop. Possibly how you'd do it is that you pass an |
dbnicholson
added a commit
to endlessm/kolibri-installer-android
that referenced
this issue
Aug 17, 2023
Running Kolibri tasks cannot be cancelled[1] and shutting down the process bus running them will block until the running tasks complete. This means the Kolibri server can't be stopped quickly or reliably during the activity's `onActivityStopped` hook. To workaround that, move the task workers spawned from the `ServicesPlugin` into a separate process bus. This bus will only be stopped when the activity is destroyed (or killed). Besides making the server lifecycle much more reliable, it allows tasks to continue running and starting in the background as long as the activity process is alive. Ideally the server and workers would be separate services outside of the main webview activity, but python-for-android makes that really hard to accomplish. 1. learningequality/kolibri#10249 Helps: endlessm/kolibri-explore-plugin#740
dbnicholson
added a commit
to endlessm/kolibri-installer-android
that referenced
this issue
Aug 17, 2023
Running Kolibri tasks cannot be cancelled[1] and shutting down the process bus running them will block until the running tasks complete. This means the Kolibri server can't be stopped quickly or reliably during the activity's `onActivityStopped` hook. To workaround that, move the task workers spawned from the `ServicesPlugin` into a separate process bus. This bus will only be stopped when the activity is destroyed (or killed). Besides making the server lifecycle much more reliable, it allows tasks to continue running and starting in the background as long as the activity process is alive. Ideally the server and workers would be separate services outside of the main webview activity, but python-for-android makes that really hard to accomplish. 1. learningequality/kolibri#10249 Helps: endlessm/kolibri-explore-plugin#740
dbnicholson
added a commit
to endlessm/kolibri-installer-android
that referenced
this issue
Aug 17, 2023
Running Kolibri tasks cannot be cancelled[1] and shutting down the process bus running them will block until the running tasks complete. This means the Kolibri server can't be stopped quickly or reliably during the activity's `onActivityStopped` hook. To workaround that, move the task workers spawned from the `ServicesPlugin` into a separate process bus. This bus will only be stopped when the activity is destroyed (or killed). Besides making the server lifecycle much more reliable, it allows tasks to continue running and starting in the background as long as the activity process is alive. Ideally the server and workers would be separate services outside of the main webview activity, but python-for-android makes that really hard to accomplish. 1. learningequality/kolibri#10249 Helps: endlessm/kolibri-explore-plugin#740
dylanmccall
pushed a commit
to endlessm/kolibri-installer-android
that referenced
this issue
Aug 17, 2023
Running Kolibri tasks cannot be cancelled[1] and shutting down the process bus running them will block until the running tasks complete. This means the Kolibri server can't be stopped quickly or reliably during the activity's `onActivityStopped` hook. To workaround that, move the task workers spawned from the `ServicesPlugin` into a separate process bus. This bus will only be stopped when the activity is destroyed (or killed). Besides making the server lifecycle much more reliable, it allows tasks to continue running and starting in the background as long as the activity process is alive. Ideally the server and workers would be separate services outside of the main webview activity, but python-for-android makes that really hard to accomplish. 1. learningequality/kolibri#10249 Helps: endlessm/kolibri-explore-plugin#740
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Observed behavior
Kolibri wouldn't shutdown while a content import task was running. Observe the timestamps around
^Cin the logs below-- above it was also already during shutdown.Errors and logs
Expected behavior
It seems that this should not have blocked Kolibri shutdown, since the task is cancellable. So for some reason the cancellation of the job, as captured by the logs, didn't actually stop the content import task.
User-facing consequences
In ability to shutdown Kolibri until tasks are complete
Steps to reproduce
Context
Ubuntu 22.04
Kolibri develop pex https://github.com/learningequality/kolibri/suites/11465479013/artifacts/591721049
Chrome
The text was updated successfully, but these errors were encountered: