New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HPCC-24273 Improve speed of WUListQueries. #13880
HPCC-24273 Improve speed of WUListQueries. #13880
Conversation
https://track.hpccsystems.com/browse/HPCC-24273 |
Local testing with an environment with ~9000 deployed queries, this speeds up the scan from approx 30 seconds, to ~ 1.1 second. @ghalliday - please review. |
Automated Smoketest: ✅ Unit tests result:
Regression test result:
HPCC Stop: OK
|
Should this target 7.8.x ? |
@jakesmith code looks good. I think it could even be considered for 7.6.x given the impact v complexity. |
could do, didn't target a point build because it's an old issue and there are other pending related issues that may not make it before 7.10, and the environment where this was seen may want to wait until then (which is not too far off). OTH - this issue will slowdown at least Eclwatch in all environments where there's a decent number of deployed queries when using WUListQueries .. I don't know how comment it is to call from SOAPCALL .. @richardkchapman - shall I target 7.6.x ? |
@jakesmith Yes |
The cost of validating whether a query was active, by scanning Alias' for a match when there were 1000's of queries, was very expensive. Build a unordered_set 1st then use that to validate if queries are active. Signed-off-by: Jake Smith <jake.smith@lexisnexisrisk.com>
ff81d4f
to
4056e6d
Compare
@ghalliday @richardkchapman - rebased to 7.6.x |
Automated Smoketest: ✅ Rebuild: success Unit tests result:
Regression test result:
HPCC Stop: OK
|
The cost of validating whether a query was active, by scanning
Alias' for a match when there were 1000's of queries, was very
expensive.
Build a unordered_set 1st then use that to validate if queries
are active.
Signed-off-by: Jake Smith jake.smith@lexisnexisrisk.com
Type of change:
Checklist:
Smoketest:
Testing: