Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept some differing units for throttling #1103

Merged
merged 2 commits into from
Nov 3, 2020

Conversation

danielmitterdorfer
Copy link
Member

@danielmitterdorfer danielmitterdorfer commented Nov 3, 2020

With #1100 we introduced more flexible throttling which requires that
the unit in which requests are throttled and the unit in which they are
reported, are aligned. This causes issues at the moment with scroll
requests which are throttled in ops/s but reported in pages/s. Similarly we have
also throttled bulk requests based on ops/s sometimes but their throughput is
reported in docs/s.

With this commit we correct the unit mismatch for this special case in
order to stay backwards-compatible with older versions of Rally. We also
ensure that throttling is preserved in test mode so our integration
tests can spot such issues in the future.

Relates #1100

With elastic#1100 we introduced more flexible throttling which requires that
the unit in which requests are throttled and the unit in which they are
reported, are aligned. This causes issues at the moment with scroll
requests which are throttled in ops/s but reported in pages/s.

With this commit we correct the unit mismatch for this special case in
order to stay backwards-compatible with older versions of Rally. We also
ensure that throttling is preserved in test mode so our integration
tests can spot such issues in the future.

Relates elastic#1100
@danielmitterdorfer danielmitterdorfer added bug Something's wrong :Load Driver Changes that affect the core of the load driver such as scheduling, the measurement approach etc. labels Nov 3, 2020
@danielmitterdorfer danielmitterdorfer added this to the 2.0.3 milestone Nov 3, 2020
@danielmitterdorfer danielmitterdorfer self-assigned this Nov 3, 2020
Copy link
Contributor

@dliappis dliappis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I think we should change (in master) the corresponding scroll queries in geonames and pmc e.g. https://github.com/elastic/rally-tracks/blob/ea52c9f601f1b5fbe485941b081d2bba9e445c20/geonames/challenges/default.json#L104-L105 to have throughput specified as "20 pages/s".

@danielmitterdorfer
Copy link
Member Author

I think we should change (in master) the corresponding scroll queries in geonames and pmc e.g. https://github.com/elastic/rally-tracks/blob/ea52c9f601f1b5fbe485941b081d2bba9e445c20/geonames/challenges/default.json#L104-L105 to have throughput specified as "20 pages/s".

I agree; what we have here is only temporary for backwards-compatibility. After we've merged this, I'll raise a corresponding issue in rally-tracks. I think we could do the change in rally-tracks after we've released 2.1.0 so there is a long enough grace period for people to upgrade.

@danielmitterdorfer
Copy link
Member Author

@danielmitterdorfer
Copy link
Member Author

@elasticmachine test this please

@danielmitterdorfer danielmitterdorfer merged commit d10979f into elastic:master Nov 3, 2020
@danielmitterdorfer danielmitterdorfer deleted the scroll-fix branch November 3, 2020 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something's wrong :Load Driver Changes that affect the core of the load driver such as scheduling, the measurement approach etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants