pmwebd: improve graphite archive-cache performance w.r.t. syscalls #429
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It was reported that on a large collection of pcp archives, which included a number of corrupt (0-byte ones), the graphite metric-enumeration query took too long. One source of this was excessive effort on - frequently retrying opening corrupt archives, and - fstat'ing all files under -A $DIR We no longer do either. Corrupt archives are treated as though they were fresh at the moment of pmwebd startup, but containing no content. The -A directory's transitive contents are no longer routinely fstat()d, only readdir() enumerated. This costs us the ability to follow subdirectory symlinks, but it's a pretty big win otherwise. No QA impact, only performance.
|
ran baseline (master) QA for group 'pmwebapi' and 1090 and 1388 failed. Then built and installed with the patches from this PR and #428, and now 661 and 1042 fail. That's on the same machine, so maybe there is residual pmwebapi QA pollution between test runs or something. Needs more investigation. |
|
Earlier report mentioning ".. residual pmwebapi QA pollution between test runs" has proven correct - I've applied this patch and run QA on a different system, and it's fine now. So I'll be merging this PR shortly. |
goodwinos
added a commit
to goodwinos/pcp
that referenced
this pull request
Mar 7, 2018
…fche-merge Merged from PR performancecopilot#429 - pmwebd: improve graphite archive-cache performance w.r.t. syscalls
|
Merged into upstream master. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It was reported that on a large collection of pcp archives, which
included a number of corrupt (0-byte ones), the graphite
metric-enumeration query took too long. One source of this was
excessive effort on
We no longer do either. Corrupt archives are treated as though they
were fresh at the moment of pmwebd startup, but containing no content.
The -A directory's transitive contents are no longer routinely
fstat()d, only readdir() enumerated. This costs us the ability to
follow subdirectory symlinks, but it's a pretty big win otherwise.
No QA impact, only performance.