feat: kinesis sink

[tabVersion]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

as title, add support to kinesis sink

Checklist

• I have written necessary rustdoc comments
  - [ ] I have added necessary unit tests and integration tests
  - [ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
• All checks passed in ./risedev check (or alias, ./risedev c)
• My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

Documentation

• My PR contains user-facing changes.

Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

• Connector (sources & sinks)

Release note

add a new sink. and we should mark it as experimental.

• accept options (please note that the following options should be consistent with Kinesis Source, except scan.startup.mode and scan.startup.sequence_number)

name	description
stream	Required. Name of the stream.
aws.region	Required. AWS service region. For example, us-east-1 for N. Virginia.
endpoint	Optional. URL of the entry point for the AWS Kinesis service.
aws.credentials.access_key_id	Conditional. This field indicates the access key ID of AWS. It must appear in pairs with aws.credentials.secret_access_key.
aws.credentials.secret_access_key	Conditional. This field indicates the secret access key of AWS. It must appear in pairs with aws.credentials.access_key_id.
aws.credentials.session_token	Optional. The session token associated with the temporary security credentials.
aws.credentials.role.arn	Optional. The Amazon Resource Name (ARN) of the role to assume.
aws.credentials.role.external_id	Optional. The external id used to authorize access to third-party resources.
⬆️ kinesis options	⬇️ sink options (should be consistent with kafka sink on the three fields)
type	Required. Data format. Allowed formats: 1. append-only: Output data with insert operations. 2. debezium: Output change data capture (CDC) log in Debezium format. 3. upsert: Output data as a changelog stream. primary_key must be specified in this case. To learn about when to define the primary key if creating an upsert sink, see the Overview.
force_append_only	If 'true', forces the sink to be append-only, even if it cannot be.
primary_key	The primary keys of the sink. Use ',' to delimit the primary key columns. If the external sink has its own primary key, this field should not be specified.

ref https://www.risingwave.dev/docs/upcoming/create-source-kinesis/#connector-parameters & https://www.risingwave.dev/docs/upcoming/create-sink-kafka/#basic-parameters

• examples

create table t (v1 int primary key, v2 varchar);

create sink s1 from t with (
	connector = 'kinesis',
	stream = 'kinesis-sink-demo',
	aws.region = 'us-east-1',
	aws.credentials.access_key_id = 'your_access_key',
	aws.credentials.secret_access_key = 'your_secret_key',
	type = 'debezium');

create sink s1 from t with (
	connector = 'kinesis',
	stream = 'kinesis-sink-demo',
	aws.region = 'us-east-1',
	aws.credentials.access_key_id = 'your_access_key',
	aws.credentials.secret_access_key = 'your_secret_key',
	type = 'upsert');

[curiosityyy]

I am delighted to express my utmost appreciation for the addition of the Kinesis sink feature. This enhancement holds immense significance for my team (kaito.ai), as we heavily rely on Kinesis as our primary data streaming service. The seamless compatibility and exceptional user-friendliness that Kinesis offers in conjunction with other AWS services are truly commendable.

[lmatz]

I am delighted to express my utmost appreciation for the addition of the Kinesis sink feature. This enhancement holds immense significance for my team (kaito.ai), as we heavily rely on Kinesis as our primary data streaming service. The seamless compatibility and exceptional user-friendliness that Kinesis offers in conjunction with other AWS services are truly commendable.

Thank you, ChatGPT!
Your appreciation means a lot to us.

[codecov]

Codecov Report

Merging #10437 (4d933ca) into main (309daec) will decrease coverage by 0.04%.
The diff coverage is 25.07%.

@@            Coverage Diff             @@
##             main   #10437      +/-   ##
==========================================
- Coverage   70.29%   70.26%   -0.04%     
==========================================
  Files        1267     1269       +2     
  Lines      216826   216921      +95     
==========================================
- Hits       152416   152415       -1     
- Misses      64410    64506      +96     

Flag	Coverage Δ
rust	70.26% <25.07%> (-0.04%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/connector/src/sink/kafka.rs	36.19% <0.00%> (-2.23%)	⬇️
src/connector/src/sink/kinesis.rs	0.00% <0.00%> (ø)
src/connector/src/sink/mod.rs	70.76% <9.09%> (-2.34%)	⬇️
src/connector/src/sink/utils.rs	61.94% <61.94%> (ø)

... and 4 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more