Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

object store Extract Connection Limiting Logic out Of S3 #2175

Closed
alamb opened this issue Jul 26, 2022 · 0 comments
Closed

object store Extract Connection Limiting Logic out Of S3 #2175

alamb opened this issue Jul 26, 2022 · 0 comments
Assignees
@tustvold
Labels
enhancement Any new improvement worthy of a entry in the changelog good first issue Good for newcomers help wanted object-store Object Store Interface

Comments

@alamb
Copy link
Contributor

alamb commented Jul 26, 2022

Ported from influxdata/object_store_rs#16 by @tustvold

Problem

The S3 implementation currently has a SemaphoreClient that limits concurrent requests, this is useful beyond just S3, confuses the implementation, and dates from a time when ObjectStore was not object-safe.

Proposal

I would like a LimitStore that wraps a <T: ObjectStore> much like ThrottledStore and provides a configurable concurrency limit.
@alamb alamb added enhancement Any new improvement worthy of a entry in the changelog object-store Object Store Interface labels Jul 26, 2022
@alamb alamb mentioned this issue Jul 26, 2022
Closed
@alamb alamb added good first issue Good for newcomers help wanted labels Jul 26, 2022
@alamb alamb changed the title Extract object store Connection Limiting Logic out Of S3 object store Extract Connection Limiting Logic out Of S3 Jul 26, 2022
@tustvold tustvold self-assigned this Jul 30, 2022
tustvold added a commit to tustvold/arrow-rs that referenced this issue Jul 30, 2022
@tustvold
Add LimitStore (apache#2175)
001b654
@tustvold tustvold mentioned this issue Jul 30, 2022
Merged
tustvold added a commit that referenced this issue Jul 31, 2022
@tustvold
Add LimitStore (#2175) (#2242)
6c3f9a2 
* Add LimitStore (#2175)

* Review feedback

* Fix test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tustvold tustvold

Labels
enhancement Any new improvement worthy of a entry in the changelog good first issue Good for newcomers help wanted object-store Object Store Interface
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alamb @tustvold