Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve handling of large files #288

Open
alewitt2 opened this issue Jan 29, 2022 · 0 comments
Open

Improve handling of large files #288

alewitt2 opened this issue Jan 29, 2022 · 0 comments
Labels
bug Something isn't working enhancement New feature or request

Comments

@alewitt2
Copy link
Member

Is your feature request related to a problem? Please describe.
when trying to apply a large file (one real world example they were trying to apply a 2.7MB file), the remoteresource-controller container kept getting OOMKilled.

Describe the solution you'd like
dont download the entire file, these large files are usually large because they are --- separated multipart yamls. You should be able to download or stream chunks and process them accordingly.

Describe alternatives you've considered
use smaller files. divide large multipart yamls into smaller files with less yamls.

Additional context
real world sample of container getting killed

remoteresource-controller:
    Image:          registry.ng.bluemix.net/armada-master/remoteresource:1.0.3_b3def35
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Fri, 28 Jan 2022 21:10:31 +0000
      Finished:     Fri, 28 Jan 2022 21:13:12 +0000
    Ready:          False
    Restart Count:  41
    Limits:
      cpu:     100m
      memory:  200Mi
    Requests:
      cpu:     40m
      memory:  75Mi
@alewitt2 alewitt2 added enhancement New feature or request bug Something isn't working labels Jan 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant