-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Empty KinesisEndpoint doesn't default to generated endpoint #5
Comments
I noticed the same thing. The DynamoDB checkpointer implementation does the empty string check and make sure to only override the
But the worker which reads from Kinesis doesn't do this
AWS specifies resolver to return
|
I managed to achieve this by doing workaround like this,
But ideally this code should be added to the part mentioned by pandar00 - https://github.com/vmware/vmware-go-kcl-v2/blob/main/clientlibrary/worker/worker.go#L165 |
This commit fixes vmware#5 by returning `aws.EndpointNotFoundError` from the endpoint resolver when no `KinesisEndpoint` is defined, which will resolve the default AWS endpoint. This is the same process used by the DynamoDB checkpointer to resolve the default endpoint. Signed-off-by: Caleb Stewart <caleb.stewart94@gmail.com>
* fixing infinite worker loop Signed-off-by: Mike Monaghan <mike_monaghan@live.ca> * Automatically resolve default KinesisEndpoint This commit fixes vmware#5 by returning `aws.EndpointNotFoundError` from the endpoint resolver when no `KinesisEndpoint` is defined, which will resolve the default AWS endpoint. This is the same process used by the DynamoDB checkpointer to resolve the default endpoint. Signed-off-by: Caleb Stewart <caleb.stewart94@gmail.com> * fix: catch DynamoDB Scan error when trying to scan nonexistent table/index in syncLeases() Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * chore: Adding periods to copyright comment to satisfy gofmt Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * feat: Sending renewed lease metric Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * fix: add DeleteMetricMillisBehindLatest for error case Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * Refactor in prep for testing rate limiting improvements Signed-off-by: John Calixto <jcalixto@vmware.com> * fix: add getRecords TPS rate limiting Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * fix: add hard cap maxRetries for getRecord errors Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * fix: add maxBytes per second getRecord check Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * chore: log RemoveLeaseOwner errors with debug instead of error Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * fix: add check for GetRecords error within callGetRecordsAPI Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * fix: use nanosecond precision in lease comparisons Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * chore: add info logs in sleep case for kinesis backoff errors Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * fix: Check token bucket corner cases correctly. Signed-off-by: John Calixto <jcalixto@vmware.com> * Bump github.com/prometheus/client_golang from 1.11.0 to 1.11.1 Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.11.0 to 1.11.1. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](prometheus/client_golang@v1.11.0...v1.11.1) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Bump golang.org/x/sys from 0.0.0-20211216021012-1d35b9e2eb4e to 0.1.0 Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.0.0-20211216021012-1d35b9e2eb4e to 0.1.0. - [Release notes](https://github.com/golang/sys/releases) - [Commits](https://github.com/golang/sys/commits/v0.1.0) --- updated-dependencies: - dependency-name: golang.org/x/sys dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * fix: add shutdown and leaseExpired error cases for checkpoint function Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * feat: make lease renewal async Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * fix: return err log in case of ErrLeaseNotAcquired Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * chore: Remove extraneous err check After checking the scan result above this line, checking err here no longer has any effect. Signed-off-by: John Calixto <jcalixto@vmware.com> * fix: pass in ctx with cancel for renewLease Signed-off-by: Shiva Pentakota <spentakota@vmware.com> * ran go mod tidy --------- Signed-off-by: Mike Monaghan <mike_monaghan@live.ca> Signed-off-by: Caleb Stewart <caleb.stewart94@gmail.com> Signed-off-by: Shiva Pentakota <spentakota@vmware.com> Signed-off-by: John Calixto <jcalixto@vmware.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Mike Monaghan <mike_monaghan@live.ca> Co-authored-by: Caleb Stewart <caleb.stewart94@gmail.com> Co-authored-by: Shiva Pentakota <spentakota@vmware.com> Co-authored-by: spentakota <120056013+spentakota@users.noreply.github.com> Co-authored-by: John Calixto <jcalixto@vmware.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: cmckelvey-vmware <85366153+cmckelvey-vmware@users.noreply.github.com> Co-authored-by: vmwjc <108959326+vmwjc@users.noreply.github.com>
Describe the bug
The comments for the AWS endpoint fields state that:
However, if they are empty then the AWS clients are configured with the endpoints
""
, rather than defaulting to the default endpoint resolver.Reproduction steps
Expected behavior
The default endpoint resolver is used.
Additional context
No response
The text was updated successfully, but these errors were encountered: