Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proxy policy uses an unbounded map #13277

Closed
jvandemark opened this issue Jan 19, 2021 · 4 comments
Closed

Proxy policy uses an unbounded map #13277

jvandemark opened this issue Jan 19, 2021 · 4 comments
Assignees
@jeremymeng
Labels
Azure.Core bug This issue requires a change to an existing behavior in the product in order to be resolved. Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization.
Milestone
[2021] February

Comments

@jvandemark
Copy link

  • Package Name: @azure/storage-blob
  • Package Version: ~12.1.0
  • Operating system: all

Describe the bug
proxyPolicy contains an unbounded Map which will not scale in a production environment.

To Reproduce
Add a few hundred thousand containers/blobs to a storage account.
Then simply iterate over the containers/blobs.
Periodically generate heap dumps & review in chrome.
Memory consumption grows without bounds

Expected behavior
Memory should NOT grow without bounds.

Screenshots
heap dump
@ghost ghost added needs-triage This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jan 19, 2021
@jeremymeng
Copy link
Contributor

@jvandemark Thanks for reporting the issue! You are right. We should be caching based on host, not uri.

@ghost ghost removed the needs-triage This is a new issue that needs to be triaged to the appropriate team. label Jan 19, 2021
@jeremymeng jeremymeng added the bug This issue requires a change to an existing behavior in the product in order to be resolved. label Jan 19, 2021
@ghost ghost added the needs-team-attention This issue needs attention from Azure service team or SDK team label Jan 19, 2021
@jeremymeng jeremymeng added the Client This issue points to a problem in the data-plane of the library. label Jan 19, 2021
@jeremymeng jeremymeng self-assigned this Jan 19, 2021
@jeremymeng
Copy link
Contributor

/cc @bterlson so this is really a concern when thousands of different urls are used. Any small size cache as suggested in #7069 (comment) would miss majority of the requests. I think we have to re-parse the uri and cache the host part instead?

@bterlson
Copy link
Member

Sounds like a plan to me!

@ramya-rao-a ramya-rao-a added this to the [2021] February milestone Jan 19, 2021
@ramya-rao-a ramya-rao-a removed needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jan 19, 2021
@jeremymeng jeremymeng mentioned this issue Jan 21, 2021
Merged
@jeremymeng
Copy link
Contributor

Closing as #13329 has been merged and @azure/core-http v1.2.3 has been published.

@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jeremymeng jeremymeng

Labels
Azure.Core bug This issue requires a change to an existing behavior in the product in order to be resolved. Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization.
Projects
None yet
Milestone
[2021] February
Development

No branches or pull requests
4 participants
@bterlson @jvandemark @jeremymeng @ramya-rao-a