Skip to content

Commit

Permalink
chore(msk-alpha): update KafkaVersion (#29440)
Browse files Browse the repository at this point in the history
### Issue # (if applicable)

Could not find any in the backlog

### Reason for this change

Update the CDK listed Kafka versions to match the current availability,
as well as add missing deprecated versions

### Description of changes

* Added latest version
* `3.6.0` also supports tiered storage, see
[docs](https://docs.aws.amazon.com/msk/latest/developerguide/msk-tiered-storage.html#msk-tiered-storage-requirements).
I replaced the prefix check with a list of supported versions, as I'm
not sure if say every reason after 3.6.0 will support it, and the
`.tiered` prefix isn't consistently applied anymore
* Added two unlisted, deprecated versions, as they are still returned by
the SDK. The goal is to remove any differences between the SDK and the
CDK to ease future automation

### Description of how you validated changes

I compared the current CDK versions to live SDK data, using the
`kafka:ListKafkaVersions` API results.

### Checklist
- [x] My code adheres to the [CONTRIBUTING
GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and
[DESIGN
GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache-2.0 license*

---------

Co-authored-by: GZ <yuanhaoz@amazon.com>
  • Loading branch information
nmussy and GavinZZ committed Mar 13, 2024
1 parent 00e8a7b commit 4582ac5
Show file tree
Hide file tree
Showing 3 changed files with 44 additions and 4 deletions.
5 changes: 3 additions & 2 deletions packages/@aws-cdk/aws-msk-alpha/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,15 +217,16 @@ You can configure an MSK cluster storage mode using the `storageMode` property.
Tiered storage is a low-cost storage tier for Amazon MSK that scales to virtually unlimited storage,
making it cost-effective to build streaming data applications.

> Visit [Tiered storage](https://docs.aws.amazon.com/msk/latest/developerguide/msk-tiered-storage.html) for more details.
> Visit [Tiered storage](https://docs.aws.amazon.com/msk/latest/developerguide/msk-tiered-storage.html)
to see the list of compatible Kafka versions and for more details.

```ts
declare const vpc: ec2.Vpc;
declare const bucket: s3.IBucket;

const cluster = new msk.Cluster(this, 'cluster', {
clusterName: 'myCluster',
kafkaVersion: msk.KafkaVersion.V2_8_2_TIERED,
kafkaVersion: msk.KafkaVersion.V3_6_0,
vpc,
storageMode: msk.StorageMode.TIERED,
});
Expand Down
35 changes: 34 additions & 1 deletion packages/@aws-cdk/aws-msk-alpha/lib/cluster-version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,15 @@ export class KafkaVersion {
*/
public static readonly V1_1_1 = KafkaVersion.of('1.1.1');

/**
* **Deprecated by Amazon MSK. You can't create a Kafka cluster with a deprecated version.**
*
* Kafka version 2.1.0
*
* @deprecated use the latest runtime instead
*/
public static readonly V2_1_0 = KafkaVersion.of('2.1.0');

/**
* Kafka version 2.2.1
*/
Expand All @@ -21,6 +30,15 @@ export class KafkaVersion {
*/
public static readonly V2_3_1 = KafkaVersion.of('2.3.1');

/**
* **Deprecated by Amazon MSK. You can't create a Kafka cluster with a deprecated version.**
*
* Kafka version 2.4.1
*
* @deprecated use the latest runtime instead
*/
public static readonly V2_4_1 = KafkaVersion.of('2.4.1');

/**
* Kafka version 2.4.1
*/
Expand Down Expand Up @@ -111,6 +129,11 @@ export class KafkaVersion {
*/
public static readonly V3_5_1 = KafkaVersion.of('3.5.1');

/**
* Kafka version 3.6.0
*/
public static readonly V3_6_0 = KafkaVersion.of('3.6.0');

/**
* Custom cluster version
* @param version custom version number
Expand All @@ -119,6 +142,16 @@ export class KafkaVersion {
return new KafkaVersion(version);
}

/**
* List of Kafka versions that support tiered storage
*
* @see https://docs.aws.amazon.com/msk/latest/developerguide/msk-tiered-storage.html#msk-tiered-storage-requirements
*/
private static readonly TIERED_STORAGE_COMPATIBLE_VERSIONS = [
KafkaVersion.V2_8_2_TIERED,
KafkaVersion.V3_6_0,
].map(({ version }) => version);

/**
*
* @param version cluster version number
Expand All @@ -129,6 +162,6 @@ export class KafkaVersion {
* Checks if the cluster version supports tiered storage mode.
*/
public isTieredStorageCompatible() {
return this.version.endsWith('.tiered');
return KafkaVersion.TIERED_STORAGE_COMPATIBLE_VERSIONS.includes(this.version);
};
}
8 changes: 7 additions & 1 deletion packages/@aws-cdk/aws-msk-alpha/test/cluster.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -790,14 +790,20 @@ describe('MSK Cluster', () => {

describe('created with storage mode', () => {
describe('with tiered storage mode', () => {
test('version.isTieredStorageCompatible', () => {
expect(msk.KafkaVersion.V2_8_2_TIERED.isTieredStorageCompatible()).toBeTruthy();
expect(msk.KafkaVersion.V3_5_1.isTieredStorageCompatible()).toBeFalsy();
expect(msk.KafkaVersion.V3_6_0.isTieredStorageCompatible()).toBeTruthy();
});

test('create a cluster with tiered storage mode', () => {
new msk.Cluster(stack, 'Cluster', {
clusterName: 'cluster',
instanceType: ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.LARGE),
kafkaVersion: msk.KafkaVersion.V2_8_2_TIERED,
vpc,
storageMode: msk.StorageMode.TIERED,
}),
});
Template.fromStack(stack).hasResourceProperties('AWS::MSK::Cluster', {
StorageMode: 'TIERED',
});
Expand Down

0 comments on commit 4582ac5

Please sign in to comment.