aws-ecs: No way to override aws-for-fluent-bit image

[mbonig]

Describe the feature

I'm currently using the aws firelens log router on an ECS container, like so:

new ApplicationLoadBalancedFargateService(this, 'Backend', {
      cluster,
      // ... removed for brevity
     taskImageOptions: {
        // ... removed
        logDriver: LogDrivers.firelens({
          options: {
          // .. removed for brevity  
          },
        }),

However, a recent issue with the container caused ECS services to recycle tasks non-stop:
aws/aws-for-fluent-bit#491

There doesn't appear to be any way to override the image tag that will be used when using this construct.

I also don't see a way to use an escape hatch or other method to choose a different SSM parameter for the log driver to use.

What is the recommended way to override the tag being used? Could an additional property be exposed on the LogDrivers.fireLens() to allow the setting of the tag?

Use Case

I'd like to be able to control the tag of the aws-for-fluent-bit image.

Proposed Solution

I don't have one.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.53.0

Environment details (OS name and version, etc.)

[nbutkowski-chub]

+1 - I'm also looking for this functionality.

[nathanalderson]

You can use https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.FirelensLogRouter.html like this:

new ecs.FirelensLogRouter(this, 'MyFirelensLogRouter', {
    firelensConfig: {
      type: ecs.FirelensLogRouterType.FLUENTBIT,
      options: {
        configFileType: ecs.FirelensConfigFileType.FILE,
        configFileValue: "/fluent-bit/configs/minimize-log-loss.conf",
        enableECSLogMetadata: true
      }
    },
    image: ecs.ContainerImage.fromRegistry("public.ecr.aws/aws-observability/aws-for-fluent-bit:2.28.4"),
    taskDefinition: this.taskDefinition,
    containerName: 'FirelensLogRouter'
})

This is from within our export class Application extends ecs_patterns.ApplicationLoadBalancedFargateService. Hopefully this helps.

[peterwoodworth]

We don't currently offer a way to directly configure this when using LogDrivers.firelens(). Here's the code that's ran when the containers are rendered:

aws-cdk/packages/@aws-cdk/aws-ecs/lib/base/task-definition.ts

Lines 787 to 804 in 7b7294c

	private renderContainers() {
	// add firelens log router container if any application container is using firelens log driver,
	// also check if already created log router container
	for (const container of this.containers) {
	if (container.logDriverConfig && container.logDriverConfig.logDriver === 'awsfirelens'
	&& !this.containers.find(x => x instanceof FirelensLogRouter)) {
	this.addFirelensLogRouter('log-router', {
	image: obtainDefaultFluentBitECRImage(this, container.logDriverConfig),
	firelensConfig: {
	type: FirelensLogRouterType.FLUENTBIT,
	},
	logging: new AwsLogDriver({ streamPrefix: 'firelens' }),
	memoryReservationMiB: 50,
	});
	break;
	}
	}

obtainDefaultFluentBitECRImage() method doesn't have any value for imageTag passed in. We could add a way to pass in a configurable imageTag prop here

For now, you can work around this using the FirelensLogRouter class and adding it to your task definition directly, where the image is already configurable. This will prevent the above code from running where the image is not configurable

[mbonig]

Would it make sense to update this to use 'stable'?

[peterwoodworth]

There is a feature request I reopened recently to default the whole thing to stable. Given that it broke again I think we should reconsider our default image.

I think this feature request is distinct from adjusting the overall default to use the stable image, and both are valid requests.