Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce memory bloating in FileMappedDict when reading metrics. #160

Merged
merged 1 commit into from
Oct 23, 2019
Merged

Reduce memory bloating in FileMappedDict when reading metrics. #160

merged 1 commit into from
Oct 23, 2019

Commits on Oct 14, 2019

  1. Reduce memory bloating in FileMappedDict when reading metrics. 

    During the read path (metrics export) we open all existing metric files
and read through them in order to aggregate metrics across different
processes. When reading non-empty files we end up parsing file content
twice: first during FileMappedDict initialisation, then again in the
caller site (`all_values`).

This commit refactors FileMappedDict so that while the `@positions` map
is populated at creation time, the actual metric values are read only
when explicitly requested. This avoids the memory bloat of unpacking
file content twice.

Signed-off-by: Cristian Greco <cristian@gocardless.com>
    Cristian Greco committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    c0260ff View commit details
    Browse the repository at this point in the history