Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RAM consumption becoming bottleneck when #feature are large and RAM are intensive #4

Closed
guolinke opened this issue Oct 17, 2016 · 2 comments
Closed

RAM consumption becoming bottleneck when #feature are large and RAM are intensive #4

guolinke opened this issue Oct 17, 2016 · 2 comments
Assignees
@chivee @guolinke

Comments

@guolinke
Copy link
Collaborator

LightGBM will allocate a cache to store historical histograms. This will cost about #leaf x #feature x #bin. May need to design a LRU liked cache to reduce this memory cost.
@chivee chivee changed the title Use many momories when #feature is large Use many momories when #feature is large and memory is intensive Oct 17, 2016
@chivee
Copy link
Collaborator

chivee commented Oct 17, 2016 •
edited
Loading

yes, when RAM is intensive for your task, LightGBM will waste lots of time on paging.

@chivee chivee changed the title Use many momories when #feature is large and memory is intensive RAM consumption becoming bottleneck when #feature are large and RAM are intensive Oct 17, 2016
@guolinke guolinke mentioned this issue Oct 22, 2016
Merged
@guolinke guolinke added this to the 0.1 milestone Oct 23, 2016
chivee added a commit that referenced this issue Oct 23, 2016
@chivee
Merge pull request #29 from guolinke/master
70f7d60 
add lru pool to reduce memory usage when #feature is large(#4)
@chivee chivee closed this as completed Oct 23, 2016
@huanzhang12 huanzhang12 mentioned this issue Apr 13, 2017
Merged
@gugatr0n1c gugatr0n1c mentioned this issue Apr 21, 2017
Closed
@hlee13 hlee13 mentioned this issue Jun 7, 2017
Closed
@fenqingr fenqingr mentioned this issue Jul 30, 2017
Closed
@weidong8405347 weidong8405347 mentioned this issue Nov 30, 2017
Closed
@meetdave06 meetdave06 mentioned this issue Mar 29, 2018
Closed
eisber pushed a commit to eisber/LightGBM that referenced this issue Mar 15, 2019
@elibarzilay
Merge pull request microsoft#4 from elibarzilay/readme-tweaks
121b599 
Minor readme tweaks
@fangaohz fangaohz mentioned this issue Jul 10, 2019
Closed
@rsdmse rsdmse mentioned this issue Aug 8, 2020
Closed
@wellswei wellswei mentioned this issue Jul 2, 2022
Open
shiyu1994 pushed a commit that referenced this issue Jul 29, 2022
@robert-howley-zocdoc
Merge pull request #4 from Zocdoc/rh_impl_phase_1
d3347d2 
feat: rough migration of acbull's source
@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@chivee chivee

@guolinke guolinke

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chivee @guolinke