Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up threaded RP interaction #334

Open
eirrgang opened this issue Apr 12, 2023 · 0 comments
Open

Clean up threaded RP interaction #334

eirrgang opened this issue Apr 12, 2023 · 0 comments
Labels
hardening robustness and sustainability runtime optimization scalems.radical Connectivity to RADICAL Pilot threading
Milestone

Comments

@eirrgang
Copy link
Contributor

In several cases, we avoid blocking the asyncio event loop by wrapping rp calls in ThreadPoolExecutor threads. However, even if the wrapping asyncio Tasks time out, a running Thread cannot generally be cancelled. If the rp call blocks forever, the thread pool could fill up.

We need to make sure that all wrapped rp calls have some sort of guarantee of completion, or we need to appropriately treat timeouts of thread futures as potentially critical internal disruptions of the program state.

In some cases, we can look for alternative coding constructs that allow for periodic polling or for an Exception to be injected that would interrupt the Thread.

Near-term steps in addressing this issue include enumerating the non-trivial occurrences of potentially-hanging wrapped blocking calls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hardening robustness and sustainability runtime optimization scalems.radical Connectivity to RADICAL Pilot threading
Projects
None yet
Development

No branches or pull requests

1 participant