Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve memory handling for remote COPY #2698

Merged
merged 1 commit into from
Dec 2, 2020

Commits on Dec 2, 2020

  1. Improve memory handling for remote COPY

    This change improves memory usage in the `COPY` code used for
    distributed hypertables. The following issues have been addressed:
    
    * `PGresult` objects were not cleared, leading to memory leaks.
    * The caching of chunk connections didn't work since the lookup
      compared ephemeral chunk pointers instead of chunk IDs. The effect
      was that cached chunk connection state was reallocated every time
      instead of being reused. This likely also caused worse performance.
    
    To address these issues, the following changes are made:
    
    * All `PGresult` objects are now cleared with `PQclear`.
    * Lookup for chunk connections now compares chunk IDs instead of chunk
      pointers.
    * The per-tuple memory context is moved the to the outer processing
      loop to ensure that everything in the loop is allocated on the
      per-tuple memory context, which is also reset at every iteration of
      the loop.
    * The use of memory contexts is also simplified to have only one
      memory context for state that should survive across resets of the
      per-tuple memory context.
    
    Fixes timescale#2677
    erimatnor committed Dec 2, 2020
    Configuration menu
    Copy the full SHA
    b4cd8be View commit details
    Browse the repository at this point in the history