Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql_series: optimize memory use for range time series #61

Merged
merged 2 commits into from Jun 5, 2018

Conversation

Projects
None yet
2 participants
@vavrusa
Copy link
Contributor

commented Jun 5, 2018

Refactored the series parsing code to avoid intermediate object allocations and mutations, which make longer time series too expensive. The series is now constructed in single pass, backfilling null values when needed.

For example, here's a pageload of 24h graph with ~100 series.

Before - 115s to load, 4602MB of memory:
screen shot 2018-06-04 at 21 56 47

After - 8s to load, 158MB of memory:
screen shot 2018-06-04 at 22 00 52

vavrusa added some commits Jun 5, 2018

sql_series: optimize memory use for range time series
Refactored the series parsing code to avoid intermediate object allocations
and mutations, which make longer time series too expensive. The series
is now constructed in single pass, backfilling null values when needed.
sql_query: apply ad-hoc filters on inner subqueries
The ad-hoc filters were previously applied on outer query, not on
inner query over an actual table.

@vavrusa vavrusa force-pushed the vavrusa:master branch from aba55c7 to 0f0fae0 Jun 5, 2018

@hagen1778

This comment has been minimized.

Copy link
Collaborator

commented Jun 5, 2018

Hi @vavrusa

Great work! Thank you for valuable improvements!

@hagen1778 hagen1778 merged commit 5ff2d16 into Vertamedia:master Jun 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.