OPERATR on the AWS Container Marketplace.
These instructions and CloudFormation are provided for use with OPERATR on the AWS Container Marketplace.
Deploying to ECS or EKS.
OPERATR is a great fit for deployment to ECS or EKS because it a single docker container with zero dependencies.
OPERATR requires a minimum allocation of 2GB heap and 2GB disk, making it ideal for provisioning as a FARGATE task.
OPERATR is a single docker container that connects to your Kafka cluster with exactly the same configuration as a Kafka Producer or Consumer. If you have connected to you cluster, you know how to configure OPERATR.
This repository contains configuration for an OPERATR Cloudformation Stack.
This configuration is provided as a quick-start demonstration of provisioning and configuration options, and is for example purposes only. It does not include provisioning of infrastructure to access and loadbalance the OPERATR UI.
This configuration defines the following resources:
- AWS::ECS::TaskDefinition to run your OPERATR AWS Container Marketplace subscription.
- AWS::ECS::Service containing the Task, provisioned within a Subnet designated by you.
- AWS::EC2::SecurityGroup with permissive egress to ECR/Kafka and ingress on the UI port.
- AWS::IAM::Role providing IAM actions to ECS, Logs, and Marketplace:RegisterUsage.
This configuration sets the NOFILE ulimit to 100,000, applies minimum heap of 2GB and vpu of 1024, and provides access to the full set of OPERATR environment variables required to:
- Connect to un-authenticated clusters use operatr-no-auth.yml
- Connect to authenticated clusters use operatr-auth.yml
Once started the OPERATR task will attempt to connect to Kafka on the bootstrap URL you provided.
In order for your OPERATR task to run correctly you may need to alter your Kafka Cluster Security Group to allow ingress on the port of the bootstrap URL (eg. 9092, 9094 for MSK w/ SSL connections, etc).
Authentication & Encryption
OPERATR supports all standard Kafka client authentication options, configured via environment variables.
If you are connecting to a cluster configured with Mutual TLS authentication you will need to configure a keystore containing your certificates.
The container we provide in your OPERATR subscription does not contain your keystore, so you will need to build a derivative Docker container from your subscription that contains your keystore. You can then configure that store with environment variables as defined in operatr-auth.yml.
Further information on containers, certificates, and network security can be found here.
The OPERATR container is based on Amazon Corretto 11, providing Amazon CA certificates within its truststore. This means that encrypted connections to Amazon MSK are as simple as selecting 'SSL' for your security protocol.
To configure encrypted connections using your own self-signed certificates, or certificates whose CA certificate is not within the base truststore, you need to build a derivative Docker container as described in Mutual TLS (above) and then install and configure the truststore containing your CA certificate.
You may configure an encrypted connection and disable certificate validation by setting the SslEndpointIdentificationAlgorithm variable as empty. If you choose that option you will not need to configure a truststore.
If stack creation fails with a 'CannotPullContainer' error please either:
- Select the 'Auto-Assign-Public-IP' deployment option; or,
- Ensure the subenet you choose to deploy OPERATR into has access to a NAT.
This is required to pull your OPERATR subscription container image, see here for more information.