-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scala Stream Collector: configure the ability to use IP address as partition key #2331
Comments
In the end we decided to make this a configurable setting, defaulting to random partitioning. |
fblundun
changed the title
Decide whether to keep using IP address as partition key
Scala Stream Collector: configure the ability to use IP address as partition key
Jan 13, 2016
The default will be random sharding - but if you need events sharded by |
I'll add an optional boolean configuration setting called collector.sink.kinesis.useIpAddressAsPartitionKey defaulting to false. |
lukeindykiewicz
pushed a commit
to snowplow/stream-collector
that referenced
this issue
Jun 4, 2020
lukeindykiewicz
pushed a commit
to snowplow/stream-collector
that referenced
this issue
Jun 5, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The SSC doesn't guarantee that records will be sent to Kinesis in the same order they arrive, which means sending events from the same IP to the same shard doesn't do anything.
So we should either stop using the IP address as a partition key or start guaranteeing preservation of record order.
The text was updated successfully, but these errors were encountered: