-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve performance of GET requests #271
Open
moio
wants to merge
13
commits into
manicminer:main
Choose a base branch
from
moio:faster_get
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
Cascades to GetHttpRequestInput and related functions Signed-off-by: Silvio Moioli <silvio@moioli.net>
Use json.RawMessage to unmarshal OData values separately from the rest of the OData structure, and directly into the a desired result slice. Signed-off-by: Silvio Moioli <silvio@moioli.net>
Unmarshal errors separately from the rest so that unmarshaling with two different types can be avoided. Signed-off-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
moio
added a commit
to rancher/rancher
that referenced
this pull request
Jan 22, 2024
Replace the hamilton library with a 0.46 version with this PR merged: manicminer/hamilton#271 Signed-off-by: Silvio Moioli <silvio@moioli.net>
moio
added a commit
to rancher/rancher
that referenced
this pull request
Jan 23, 2024
Replace the hamilton library with a 0.46 version with this PR merged: manicminer/hamilton#271 Signed-off-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
Signed-off-by: Silvio Moioli <silvio@moioli.net>
… re-run if response middlewares are set Signed-off-by: Silvio Moioli <silvio@moioli.net>
@manicminer do you have any impressions to share about this work? |
moio
added a commit
to rancher/rancher
that referenced
this pull request
Mar 1, 2024
includes manicminer/hamilton#271 Signed-off-by: Silvio Moioli <silvio@moioli.net>
This was referenced May 21, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
👋 hamilton maintainers!
I am opening this to ask whether it would make sense to include some new code, or patch existing code, in order to improve resource consumption of the
Get
method - particularly memory usage but also CPU.I came to this problem as part of my day work at SUSE for Rancher, which uses hamilton to implement Entra ID (formerly Azure AD) integration. In some high-scale use tests we have users with principals in the thousands, groups in the millions and group assignments per principal in the hundreds - and enough traffic to need multiple (tens of) goroutines to periodically refresh all these objects in parallel. In extreme cases we have seen hundreds of MBs to GiBs worth of heap objects, which tend to cause out-of-memory situations.
Here is an example of a similar flame graph:
![image](https://private-user-images.githubusercontent.com/250541/298570405-fbab24d7-8efe-4b28-a21f-73ef0d68bb08.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjI4MDMwNDcsIm5iZiI6MTcyMjgwMjc0NywicGF0aCI6Ii8yNTA1NDEvMjk4NTcwNDA1LWZiYWIyNGQ3LThlZmUtNGIyOC1hMjFmLTczZWYwZDY4YmIwOC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODA0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgwNFQyMDE5MDdaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT05YmU2ZmJlYjU3MmVmOTIxYzU0MGQ3MmYzZWM2NmJjNWNlMGFmYzY1MTU3OWYwMGQwOWM4MjIxZDdlMWM0YTRhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.Af8P0z0iUf1r3xnjt1TZJisHPbvx5VB9IJ5VbwFWJ1E)
Of course our usage pattern isn't typical, and we are working at ways to change the way we interact with the library. Nevertheless according to my analysis
Get
(and code called by it) does quite a bit of needless computation:json.Unmarshal
calls are amplified by a factor of 2 due to the error handling inOData.UnmarshalJSON
FromResponse
calls, which driveOData.UnmarshalJSON
calls, are amplified by a factor of 2 byperformRequest
Get
needlessly re-unmarshal responses once afterFasterResponse
(again with the amplification factors above), and especiallyGet
un-marshals and subsequently re-marshals responses if they are paginated. This unmarshaling-marshaling extra round is repeated by the number of pagesThis PR cuts down unmarshaling to one pass only (divided in sub-passes via
json.RawMessage
) and eliminates remarshaling altogether.I could not find ways to achieve this without altering the
OData
struct and structs/methods which use it (ConsistencyFailureFunc
,ValidStatusFunc
,GetHttpRequestInput
), therefore, I opted for a completely new code path infasterclient.go
. The approach can be changed if API changes are acceptable.For now and for the sake of discussion, I only patched the one endpoint that really hurts my use case:
ListGroupMemberships
. Changes are minimal other than using theFasterGet
method in place ofGet
- ideally this could be extended to all other (internal) usages ofGet
.I am also contributing code to exercise
ListGroupMemberships
more thoroughly so that the effects of the patch are evident. Furthermore I added a utility method that will output CPU and memory usage during the test run.My results are as follows:
CPU load is ~36.14% of the original
Memory consumption is ~13.64% of the original
Is there any interest to merge such an improvement? What would be needed to have a similar PR accepted?
Thanks in advance, I hope this helps