Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removed redundant memory allocations in parsers #5311

Merged
merged 1 commit into from
Feb 3, 2022

Conversation

vlad-diachenko
Copy link
Contributor

What this PR does / why we need it:
This implementation reuses errors instead of creating a new one for each line that is not a valid JSON in JSONParser and Unpack parser. These errors are used as boolean flags, we just check if an error exists and if so, we add an additional label to the line with another error message.

Which issue(s) this PR fixes:
Fixes #5310

Special notes for your reviewer:

benchstat old new                                                                                              
name                                             old time/op    new time/op    delta
_Parser/json/no_labels_hints-16                    3.83µs ± 1%    3.87µs ± 6%      ~     (p=0.587 n=5+5)
_Parser/json/labels_hints-16                       2.87µs ± 1%    2.87µs ± 2%      ~     (p=0.690 n=5+5)
_Parser/not_json_line/no_labels_hints-16            247ns ± 1%      38ns ±22%   -84.70%  (p=0.008 n=5+5)
_Parser/not_json_line/labels_hints-16               244ns ± 1%      40ns ± 5%   -83.48%  (p=0.008 n=5+5)
_Parser/unpack/no_labels_hints-16                  1.01µs ± 2%    1.10µs ± 5%    +8.12%  (p=0.008 n=5+5)
_Parser/unpack/labels_hints-16                     1.02µs ± 2%    1.08µs ± 2%    +5.52%  (p=0.008 n=5+5)
_Parser/not_json_line_unpack/no_labels_hints-16     250ns ± 5%      34ns ± 1%   -86.31%  (p=0.016 n=5+4)
_Parser/not_json_line_unpack/labels_hints-16        248ns ± 2%      32ns ± 3%   -87.13%  (p=0.008 n=5+5)
_Parser/logfmt/no_labels_hints-16                  2.11µs ± 4%    2.07µs ± 1%      ~     (p=0.206 n=5+5)
_Parser/logfmt/labels_hints-16                     2.09µs ± 3%    2.10µs ± 1%      ~     (p=0.548 n=5+5)
_Parser/regex_greedy/no_labels_hints-16            4.97µs ± 1%    4.54µs ± 3%    -8.70%  (p=0.008 n=5+5)
_Parser/regex_greedy/labels_hints-16               4.64µs ± 3%    4.47µs ± 2%    -3.61%  (p=0.008 n=5+5)
_Parser/regex_status_digits/no_labels_hints-16      331ns ± 1%     332ns ± 2%      ~     (p=1.000 n=5+5)
_Parser/regex_status_digits/labels_hints-16         337ns ± 3%     327ns ± 2%      ~     (p=0.095 n=5+5)
_Parser/pattern/no_labels_hints-16                  192ns ± 1%     186ns ± 4%      ~     (p=0.151 n=5+5)
_Parser/pattern/labels_hints-16                     161ns ± 1%     153ns ± 1%    -4.61%  (p=0.008 n=5+5)

name                                             old alloc/op   new alloc/op   delta
_Parser/json/no_labels_hints-16                      656B ± 0%      656B ± 0%      ~     (all equal)
_Parser/json/labels_hints-16                         512B ± 0%      512B ± 0%      ~     (all equal)
_Parser/not_json_line/no_labels_hints-16            48.0B ± 0%      0.0B       -100.00%  (p=0.008 n=5+5)
_Parser/not_json_line/labels_hints-16               48.0B ± 0%      0.0B       -100.00%  (p=0.008 n=5+5)
_Parser/unpack/no_labels_hints-16                    537B ± 0%      537B ± 0%      ~     (all equal)
_Parser/unpack/labels_hints-16                       537B ± 0%      537B ± 0%      ~     (all equal)
_Parser/not_json_line_unpack/no_labels_hints-16     48.0B ± 0%      0.0B       -100.00%  (p=0.008 n=5+5)
_Parser/not_json_line_unpack/labels_hints-16        48.0B ± 0%      0.0B       -100.00%  (p=0.008 n=5+5)
_Parser/logfmt/no_labels_hints-16                    336B ± 0%      336B ± 0%      ~     (all equal)
_Parser/logfmt/labels_hints-16                       336B ± 0%      336B ± 0%      ~     (all equal)
_Parser/regex_greedy/no_labels_hints-16              194B ± 0%      193B ± 0%      ~     (p=0.167 n=5+5)
_Parser/regex_greedy/labels_hints-16                 194B ± 0%      194B ± 0%      ~     (all equal)
_Parser/regex_status_digits/no_labels_hints-16      52.0B ± 0%     52.0B ± 0%      ~     (all equal)
_Parser/regex_status_digits/labels_hints-16         52.0B ± 0%     52.0B ± 0%      ~     (all equal)
_Parser/pattern/no_labels_hints-16                  35.0B ± 0%     35.0B ± 0%      ~     (all equal)
_Parser/pattern/labels_hints-16                     32.0B ± 0%     32.0B ± 0%      ~     (all equal)

name                                             old allocs/op  new allocs/op  delta
_Parser/json/no_labels_hints-16                      46.0 ± 0%      46.0 ± 0%      ~     (all equal)
_Parser/json/labels_hints-16                         39.0 ± 0%      39.0 ± 0%      ~     (all equal)
_Parser/not_json_line/no_labels_hints-16             2.00 ± 0%      0.00       -100.00%  (p=0.008 n=5+5)
_Parser/not_json_line/labels_hints-16                2.00 ± 0%      0.00       -100.00%  (p=0.008 n=5+5)
_Parser/unpack/no_labels_hints-16                    17.0 ± 0%      17.0 ± 0%      ~     (all equal)
_Parser/unpack/labels_hints-16                       17.0 ± 0%      17.0 ± 0%      ~     (all equal)
_Parser/not_json_line_unpack/no_labels_hints-16      2.00 ± 0%      0.00       -100.00%  (p=0.008 n=5+5)
_Parser/not_json_line_unpack/labels_hints-16         2.00 ± 0%      0.00       -100.00%  (p=0.008 n=5+5)
_Parser/logfmt/no_labels_hints-16                    16.0 ± 0%      16.0 ± 0%      ~     (all equal)
_Parser/logfmt/labels_hints-16                       16.0 ± 0%      16.0 ± 0%      ~     (all equal)
_Parser/regex_greedy/no_labels_hints-16              2.00 ± 0%      2.00 ± 0%      ~     (all equal)
_Parser/regex_greedy/labels_hints-16                 2.00 ± 0%      2.00 ± 0%      ~     (all equal)
_Parser/regex_status_digits/no_labels_hints-16       2.00 ± 0%      2.00 ± 0%      ~     (all equal)
_Parser/regex_status_digits/labels_hints-16          2.00 ± 0%      2.00 ± 0%      ~     (all equal)
_Parser/pattern/no_labels_hints-16                   2.00 ± 0%      2.00 ± 0%      ~     (all equal)
_Parser/pattern/labels_hints-16                      1.00 ± 0%      1.00 ± 0%      ~     (all equal)

Copy link
Contributor

@cyriltovena cyriltovena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vlad-diachenko vlad-diachenko changed the title added upgrade notes Removed redundant memory allocations in parsers Feb 3, 2022
@cyriltovena cyriltovena merged commit ea37f45 into main Feb 3, 2022
@cyriltovena cyriltovena deleted the parsers_memory_allocation_improvement branch February 3, 2022 11:16
KMiller-Grafana pushed a commit to KMiller-Grafana/loki that referenced this pull request Feb 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

JSON and Unpack parsers - redundant memory allocation
2 participants