-
-
Notifications
You must be signed in to change notification settings - Fork 355
Add Spring Integration support with KCL #1496
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
* Provide a `KclMessageDrivenChannelAdapter` - the `MessageProducer` implementation to consume data from Kinesis stream using KCL API internally * Reorder `spring-cloud-aws-kinesis` dependencies, since KPL wants a higher `protobuf-java` version than KCL provides * Mark `KplKclIntegrationTests` as `@DisabledOnOs(OS.WINDOWS)` since KPL does not provide Windows native library anymore * Add `spring-cloud-aws-starter-integration-kinesis-client` * Document the feature
* Disable metrics for KCL in the `KplKclIntegrationTests`
* Remove all the `@AfterAll` from Kinesis integration test. It looks like KCL infrastructure is not done even after a graceful shutdown
* `setCredentialsRefreshDelay(300000L)` for KPL to avoid credentials refresh as much as possible
It fails with: ``` 17:24:43.079 [lam-thread] WARN com.amazonaws.services.dynamodbv2.AmazonDynamoDBLockClient - Could not acquire lock because of a client side failure in talking to DDB ``` Somehow KCL doesn't work well where there is KPL, too
|
OK. Now it is good for review. Thank you! |
MatejNedic
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Artem looks amazing!
|
Hello guys! No pressure, but this is the last PR to be merged before I officially archive this repo: https://github.com/spring-projects/spring-integration-aws. After this is merged I will be able to start migrating https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis over here. Thanks |
|
Thanks @artembilan ! We're a bit delayed but we're trying to keep up ;) |
KclMessageDrivenChannelAdapter- theMessageProducerimplementation to consume data from Kinesis stream using KCL API internallyspring-cloud-aws-kinesisdependencies, since KPL wants a higherprotobuf-javaversion than KCL providesKplKclIntegrationTestsas@DisabledOnOs(OS.WINDOWS)since KPL does not provide Windows native library anymorespring-cloud-aws-starter-integration-kinesis-client📢 Type of change
📜 Description
💡 Motivation and Context
💚 How did you test it?
📝 Checklist
🔮 Next steps