Skip to content
This repository was archived by the owner on Aug 7, 2025. It is now read-only.

Commit e21c620

Browse files
Managed Flink: Tagging support and CloudWatch Logs integration
1 parent b349481 commit e21c620

File tree

1 file changed

+82
-3
lines changed
  • content/en/user-guide/aws/kinesisanalyticsv2

1 file changed

+82
-3
lines changed

content/en/user-guide/aws/kinesisanalyticsv2/index.md

Lines changed: 82 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -153,14 +153,93 @@ You can verify this with:
153153
$ awslocal s3api list-objects --bucket sink-bucket
154154
{{< /command >}}
155155

156+
## CloudWatch Logging
157+
158+
LocalStack MSAF supports [CloudWatch Logs integration](https://docs.aws.amazon.com/managed-flink/latest/java/cloudwatch-logs.html) to help monitor the Flink cluster for application events or configuration problems.
159+
The logging option can be added at the time of creating the Flink application using the [CreateApplication](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_CreateApplication.html) operation.
160+
Logging options can also be managed at a later point using the [AddApplicationCloudWatchLoggingOption](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_AddApplicationCloudWatchLoggingOption.html) and [DeleteApplicationCloudWatchLoggingOption](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_DeleteApplicationCloudWatchLoggingOption.html) operations.
161+
162+
There are following prerequisites for CloudWatch Logs integration:
163+
- You must create the application's log group and log stream.
164+
Flink will not create it for you.
165+
- You must add the permissions your application needs to write to the log stream to the service execution role.
166+
Generally the following IAM actions are sufficient: `logs:DescribeLogGroups`, `logs:DescribeLogStreams` and `logs:PutLogEvents`
167+
168+
To add a logging option:
169+
170+
{{< command >}}
171+
$ awslocal kinesisanalyticsv2 add-application-cloud-watch-logging-option \
172+
--application-name msaf-app \
173+
--cloud-watch-logging-option '{"LogStreamARN": "arn:aws:logs:us-east-1:000000000000:log-group:msaf-log-group:log-stream:msaf-log-stream"}'
174+
{
175+
"ApplicationARN": "arn:aws:kinesisanalytics:us-east-1:000000000000:application/msaf-app",
176+
"ApplicationVersionId": 2,
177+
"CloudWatchLoggingOptionDescriptions": [
178+
{
179+
"CloudWatchLoggingOptionId": "1.1",
180+
"LogStreamARN": "arn:aws:logs:us-east-1:000000000000:log-group:msaf-log-group:log-stream:msaf-log-stream"
181+
}
182+
]
183+
}
184+
{{< /command >}}
185+
186+
Configured logging options can be retrieved using [DescribeApplication](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_DescribeApplication.html):
187+
188+
{{< command >}}
189+
$ awslocal kinesisanalyticsv2 describe-application --application-name msaf-app | jq .ApplicationDetail.CloudWatchLoggingOptionDescriptions
190+
[
191+
{
192+
"CloudWatchLoggingOptionId": "1.1",
193+
"LogStreamARN": "arn:aws:logs:us-east-1:000000000000:log-group:msaf-log-group:log-stream:msaf-log-stream"
194+
}
195+
]
196+
{{< /command >}}
197+
198+
Log events can be retrieved from CloudWatch Logs using the appropriate operation.
199+
To retrieve all events:
200+
201+
{{< command >}}
202+
$ awslocal logs get-log-events --log-group-name msaf-log-group --log-stream-name msaf-log-stream
203+
{{< /command >}}
204+
205+
{{< callout >}}
206+
Logs events are reported to CloudWatch every 10 seconds.
207+
{{< /callout >}}
208+
209+
## Resource Tagging
210+
211+
You can manage [resource tags](https://docs.aws.amazon.com/managed-flink/latest/java/how-tagging.html) using [TagResource](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_TagResource.html), [UntagResource](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_UntagResource.html) and [ListTagsForResource](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_ListTagsForResource.html).
212+
Tags can also be specified when creating the Flink application using the [CreateApplication](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_CreateApplication.html) operation.
213+
214+
{{< command >}}
215+
$ awslocal kinesisanalyticsv2 tag-resource \
216+
--resource-arn arn:aws:kinesisanalytics:us-east-1:000000000000:application/msaf-app \
217+
--tags Key=country,Value=SE
218+
219+
$ awslocal kinesisanalyticsv2 list-tags-for-resource \
220+
--resource-arn arn:aws:kinesisanalytics:us-east-1:000000000000:application/msaf-app
221+
{
222+
"Tags": [
223+
{
224+
"Key": "country",
225+
"Value": "SE"
226+
}
227+
]
228+
}
229+
230+
$ awslocal kinesisanalyticsv2 untag-resource \
231+
--resource-arn arn:aws:kinesisanalytics:us-east-1:000000000000:application/msaf-app \
232+
--tag-keys country
233+
{{< /command >}}
234+
156235
## Supported Flink Versions
157236

158237
| Flink version | Supported by LocalStack | Supported by Apache |
159238
|:---:|:---:|:---:|
160239
| 1.20.0 | yes | yes |
161240
| 1.19.1 | yes | yes |
162241
| 1.18.1 | yes | yes |
163-
| 1.15.2 | yes | yes |
242+
| 1.15.2 | yes | no |
164243
| 1.13.1 | yes | no |
165244

166245
## Limitations
@@ -169,7 +248,7 @@ $ awslocal s3api list-objects --bucket sink-bucket
169248
- Only S3 zipfile code is supported
170249
- Values of 20,000 ms for `execution.checkpointing.interval` and 5,000 ms for `execution.checkpointing.min-pause` are used for checkpointing.
171250
They can not be overridden.
172-
- [Tagging](https://docs.aws.amazon.com/managed-flink/latest/java/how-tagging.html) is not supported
173251
- In-place [version upgrades](https://docs.aws.amazon.com/managed-flink/latest/java/how-in-place-version-upgrades.html) and [roll-backs](https://docs.aws.amazon.com/managed-flink/latest/java/how-system-rollbacks.html) are not supported
174252
- [Snapshot/savepoint management](https://docs.aws.amazon.com/managed-flink/latest/java/how-snapshots.html) is not implemented
175-
- CloudWatch and CloudTrail integration is not implemented
253+
- CloudTrail integration and CloudWatch metrics is not implemented.
254+
The application logging level defaults to `INFO` and can not be overridden.

0 commit comments

Comments
 (0)