Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error handling for non-parallelizable learners #546

Closed
RaphaelS1 opened this issue Sep 3, 2020 · 3 comments
Closed

Error handling for non-parallelizable learners #546

RaphaelS1 opened this issue Sep 3, 2020 · 3 comments

Comments

@RaphaelS1
Copy link
Contributor

I am unsure how easy this is to do, and it may be that it requires manual checks for every learner, but it would be useful to have some sort of error handling or checks to prevent conflicting parallelisation methods. This is related to #501 and possible #542.

For example, I ran a large benchmark experiment that kept crashing without me realising, the first time because a learner was being used when OpenMP was automatically enabled by its package, the second time because the learner was interfaced via reticulate, which future specifically says cannot be parallelised.

Some ideas for how we could deal with this, in increasingly complexity:

  1. Add a line in the @details section of docs - but people don't read this and then may not realise why the experiment never ends
  2. Write manual checks for each learner to detect which future::plan is used - e.g. in train
  3. Same as 2. but if conflict detected then call future::plan("sequential")
@mllg
Copy link
Sponsor Member

mllg commented Feb 4, 2021

Some ideas for how we could deal with this, in increasingly complexity:

1. Add a line in the `@details` section of docs - but people don't read this and then may not realise why the experiment never ends

2. Write manual checks for each learner to detect which `future::plan` is used - e.g. in `train`

3. Same as 2. but if conflict detected then call `future::plan("sequential")`

Some remote backends work nicely with nested parallelization (e.g., batchtools + basically everything else), others do not. I can implement (2) and (3) only if we come up with a list of combinations which are known to not work together, or is it only multicore + OpenMP?

@RaphaelS1
Copy link
Contributor Author

I guess this is now closed in #605 and mlr-org/mlr3learners#174 ?

@mllg
Copy link
Sponsor Member

mllg commented Mar 1, 2021

Yes, sort of. If you actively set the threads, you should know what you are doing.

@mllg mllg closed this as completed Mar 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants