Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add data streaming where beneficial to reduce memory usage #3365

Merged
merged 5 commits into from Apr 24, 2022

Commits on Apr 2, 2022

  1. Updated attachment download responses to stream from filesystem

    This allows download of attachments that are larger than current memory
    limits, since we're not loading the entire file into memory any more.
    
    For inline file responses, we take a 1kb portion of the file to sniff
    before to check mime before we proceed.
    ssddanbrown committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    82e8b15 View commit details
    Browse the repository at this point in the history
  2. Fixed streamed outputs in more extreme scenarios

    Fixes hitting memory limits where downloaded file sizes are much greater
    than memory limit. Stopping and flushing output buffer seemed to stop
    limits causing issues when fpassthru is used.
    Tested with 24M memory limit and 734M file
    ssddanbrown committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    6749faa View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    cb770c5 View commit details
    Browse the repository at this point in the history
  4. Added streaming support to API attachment read responses

    Required some special handling due to the content being base64-encoded
    within a JSON response.
    ssddanbrown committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    08a8c00 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2022

  1. Fixed tests from streaming changes

    - Added testing check to buffer stop/clear on streaming output due to
      interference during tests.
    - Made content-disposition header a little safer in download responses.
    - Also aligned how we check for testing environment.
    ssddanbrown committed Apr 3, 2022
    Configuration menu
    Copy the full SHA
    59d1fb2 View commit details
    Browse the repository at this point in the history