Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R-package] limit number of threads used in tests and examples (fixes #5987) #5988

Merged
merged 7 commits into from Jul 19, 2023

Conversation

jameslamb
Copy link
Collaborator

@jameslamb jameslamb commented Jul 18, 2023

Fixes #5987.

This PR fixes the two issues that blocked the release of {lightgbm} v4.0.0 to CRAN a few days ago, documented in #5987.

  • ensures that at most 2 threads are used in all examples and most tests
  • modifies 2 internal functions with .-separated names to instead use _

Notes for Reviewers

Once this is approved, I'll push a release containing this PR to CRAN, as v4.0.0.1 (since a new submission requires a new release number, and to avoid creating a new actual LightGBM release of all other components).

Why is this touching so many lines?

I tried many variations of the idea from #5102 (comment), using a global option (specified in R-package/tests/testthat.R or at the tops of test files) to pass through a number of threads set in Booster$new(). Unfortunately, they all resulted in segmentation faults (😬) from testthat. Maybe related to some interplay between the fact that {testthat} runs tests in sub-processes and the fact that session options are initialized when packages are loaded?

I'm not sure. For now, to try to get a release up before the CRAN summer break, I think we should just hard-code num_threads to 2 in as many places as possible... can work out a more elegant solution later, when there isn't time pressure.

Other references I consulted while trying to find a better solution

@jameslamb jameslamb changed the title WIP: [R-package] limit number of threads used in tests and examples (fixes #5102, fixes #5987) WIP: [R-package] limit number of threads used in tests and examples (fixes #5987) Jul 19, 2023
@jameslamb jameslamb changed the title WIP: [R-package] limit number of threads used in tests and examples (fixes #5987) [R-package] limit number of threads used in tests and examples (fixes #5987) Jul 19, 2023
@jameslamb jameslamb marked this pull request as ready for review July 19, 2023 05:32
@jameslamb jameslamb requested a review from jmoralez as a code owner July 19, 2023 05:32
@guolinke
Copy link
Collaborator

We will revert this PR when the "global option" works?

@jameslamb
Copy link
Collaborator Author

The parts of it about limiting number of threads in tests and examples, yes.

The renaming of those two internal functions should be permanent, now that CRAN is complaining about them.

@jameslamb
Copy link
Collaborator Author

Thanks @shiyu1994 ! I'll merge this and submit a new version to CRAN right now. Will @ you here when I do.

@jameslamb jameslamb merged commit 7dcbb8c into master Jul 19, 2023
41 checks passed
@jameslamb jameslamb deleted the fix/cran-issues branch July 19, 2023 14:22
@jameslamb
Copy link
Collaborator Author

Ok @shiyu1994 I just submitted on CRAN as v4.0.0.1. Please check your inbox. Thanks for your help!

image

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 18, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[R-package] v4.0.0 CRAN submission issues
3 participants