Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The worker is not freed if the call is interrupted. #97

Open
randy3k opened this issue Dec 6, 2021 · 1 comment
Open

The worker is not freed if the call is interrupted. #97

randy3k opened this issue Dec 6, 2021 · 1 comment

Comments

@randy3k
Copy link

randy3k commented Dec 6, 2021

Describe the bug

The discussion originates from HenrikBengtsson/future.callr#20 where I discovered that
the worker is not freed if the call to future_lapply is interrupted. The issue is not only applied to future.callr, multisession has similar issues.

Reproduce example

future::plan(future.callr::callr, workers = 5)
R.utils::withTimeout(
    future.apply::future_lapply(1:5, function(.) Sys.sleep(1000)), 
    timeout = 2,
    onTimeout = "silent")
#> NULL
for (i in 1:10) gc()
future::nbrOfFreeWorkers()
#> [1] 0

Created on 2021-12-06 by the reprex package (v2.0.1)

Expected behavior

Expect that future::nbrOfFreeWorkers should recover.

Session information

r$> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur 10.16

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_4.1.2
@randy3k
Copy link
Author

randy3k commented Dec 6, 2021

In the context of future.apply, I think that we should install an interrupt handler to cleanup the unresolved futures when the code is interrupted.

As described in HenrikBengtsson/future.callr#20, a more general issue is that we should free up the workers when the future loses its reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants