Fix: Firehose: Multiple firehose delivery streams not receiving messages from kinesis event stream

[maxhoheiser]

Motivation

Multiple instances of kcl listening to the same Kinesis event stream - only the "first" will process events, due to all using the same dynamoDB lease table. This was raised in #9476

We are spinning up a mock Kinesis server (https://github.com/etspaceman/kinesis-mock) and use the java plication aws kinesis client library to connect to this mock server. For services queriing events from the mock kinesis server, we provide a connector function listen_to_kinesis in kinesis_connector

This method starts the kcl for each listener. Kcl starts one or more instances with a worker bound to one or more shards. Each worker processes messages from the shard(s) and sends received messages to a programmatically created Python script that sends the messages to a socket. A listener function running in a separate thread listens to incoming messages on this socket and handles further processing defined by the initiator e.g. Firehose

To track shard lease and assignment shards to worker processes as well as to track checkpoints (latest read message from the shard) kcl uses a dynamoDB “lease table”

Changes

Adding a unique id to the lease table name for each instance of kcl that is instantiated.
Adding an AWS validated snapshot test for multiple Firehose delivery streams subscribing to the same Kinesis event stream

This PR fixes #9476

[github-actions]

LocalStack Community integration with Pro

    2 files      2 suites   1h 21m 52s ⏱️
2 623 tests 2 374 ✅ 249 💤 0 ❌
2 625 runs  2 374 ✅ 251 💤 0 ❌

Results for commit c186595.

♻️ This comment has been updated with latest results.

[dfangl]

Would be nice if we could avoid that kinesis fixture change for now, to avoid more extensive testing on this :)

[dfangl]

Thanks for addressing my comments! Looks good to me!

[coveralls]

coverage: 83.842% (-0.02%) from 83.861%
when pulling c186595 on fix/firehose/9476-multiple-firehose-delivery-streams-not-receiving-messages-from-kinesis-stream
into 871bfb8 on master.