Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

limit maximum amount of memory used by worker #1121

Closed
4 tasks done
joachimmetz opened this issue Jan 10, 2017 · 2 comments
Closed
4 tasks done

limit maximum amount of memory used by worker #1121

joachimmetz opened this issue Jan 10, 2017 · 2 comments
Assignees
@joachimmetz
Labels
core Issues relating to Plaso's core - processing, file access etc. enhancement New or improved functionality
Milestone
2017 October rele...

Comments

@joachimmetz
Copy link
Member

joachimmetz commented Jan 10, 2017 •
edited

Prevent a grinding slow down before OOO killer by having the foreman monitor and limit the maximum amount of memory used by worker.

For a distributed worker we likely want to adopt a nanny approach similar to that of GRR
@joachimmetz joachimmetz added core Issues relating to Plaso's core - processing, file access etc. enhancement New or improved functionality labels Jan 10, 2017
@joachimmetz joachimmetz added this to the 1.5.2 release milestone Jan 10, 2017
@joachimmetz joachimmetz self-assigned this Jan 10, 2017
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 10, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
ca519cb
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 11, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
61e4a4c
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 11, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
730afbe
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 11, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
6cc637a
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 11, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
7521d25
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 14, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
710a971
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 14, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
f77e453
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 14, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
632e405
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 14, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
54948fe
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 14, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
d5ea113
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 14, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
ca8651b
@joachimmetz joachimmetz mentioned this issue Jan 15, 2017
Closed
4 tasks
@joachimmetz joachimmetz added blocked Work cannot progress until another issue is resolved and removed blocked Work cannot progress until another issue is resolved labels Jan 15, 2017
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 16, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
8096d49
joachimmetz added a commit to joachimmetz/plaso that referenced this issue Jan 19, 2017
@joachimmetz
Limited maximum amount of memory used by worker log2timeline#1121
1df4d58
joachimmetz added a commit that referenced this issue Jan 19, 2017
@joachimmetz
Code review: 318260043: Limited maximum amount of memory used by worker
2ed652b 
#1121
@joachimmetz joachimmetz modified the milestones: 1.6.0 release, 1.5.2 release Apr 2, 2017
@joachimmetz joachimmetz mentioned this issue Apr 3, 2017
Closed
@joachimmetz joachimmetz mentioned this issue Jun 28, 2017
Closed
@Onager
Copy link
Contributor

Onager commented Jul 19, 2017

I've tested this on Linux, and can confirm the worker is killed when it uses too much memory. Regarding #1203, we now catch this error and retry starting a replacement worker, as of #1301.

@joachimmetz
Copy link
Member Author

Closing issue did some rough checks on Windows and macOS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@joachimmetz joachimmetz

Labels
core Issues relating to Plaso's core - processing, file access etc. enhancement New or improved functionality
Projects
None yet
Milestone
2017 October release - Heimdall
Development

No branches or pull requests
2 participants
@Onager @joachimmetz