NIFI-11288 Add missing dependencies required by AWS AssumeRoleWithWeb…

[juldrixx]

…Identity method

Summary

NIFI-11288

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
• JDK 21

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

How to test

1- Create an EKS cluster and an S3 bucket

terraform {
  backend "s3" {
  }
}

provider "aws" {
  region = var.region
}

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.16"

  cluster_name    = var.cluster_name
  cluster_version = "1.28"

  vpc_id                         = module.vpc.vpc_id
  subnet_ids                     = module.vpc.private_subnets
  cluster_endpoint_public_access = true

  eks_managed_node_groups = {
    initial = {
      instance_types = ["m5.large"]

      min_size     = 1
      max_size     = 5
      desired_size = 4
    }
  }
}

# Filter out local zones, which are not currently supported 
# with managed node groups
data "aws_availability_zones" "available" {
  filter {
    name   = "opt-in-status"
    values = ["opt-in-not-required"]
  }
}

locals {
  azs = slice(data.aws_availability_zones.available.names, 0, 3)
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "${var.cluster_name}-vpc"

  cidr = var.vpc_cidr
  azs  = local.azs

  private_subnets = [for k, v in local.azs : cidrsubnet(var.vpc_cidr, 4, k)]
  public_subnets  = [for k, v in local.azs : cidrsubnet(var.vpc_cidr, 8, k + 48)]

  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_dns_hostnames = true

  public_subnet_tags = {
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
    "kubernetes.io/role/elb"                    = 1
  }

  private_subnet_tags = {
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
    "kubernetes.io/role/internal-elb"           = 1
  }
}

resource "aws_s3_bucket" "s3-bucket" {
  bucket = "my-s3-bucket-XXXXXXXXXXXX"

  tags = {
    Name        = "Bucket for NIFI"
  }
}

2- Create an AWS policy to give access to S3

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:*"
      ],
      "Resource": "*"
    }
  ]
}

3- Create an AWS role that allow the K8S service account to assume it and that has the previous policy attached

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/oidc.eks.eu-west-3.amazonaws.com/id/<IDENTITY_PROVIDER_ID>"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.eu-west-3.amazonaws.com/id/<IDENTITY_PROVIDER_ID>:aud": "sts.amazonaws.com",
          "oidc.eks.eu-west-3.amazonaws.com/id/<IDENTITY_PROVIDER_ID>:sub": "system:serviceaccount:nifi:nifi"
        }
      }
    }
  ]
}

4- Create a K8S service account that will asssume the role

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nifi
  namespace: nifi
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>

5- Create NiFi cluster with a Pod and a Service to expose it

apiVersion: v1
kind: Pod
metadata:
  name: nifi
  namespace: nifi
  labels:
    app: nifi
spec:
  serviceAccountName: nifi
  containers:
  - name: nifi
    image: <YOUR_UPDATED_IMAGE>
    ports:
    - containerPort: 8443
    env:
    - name: NIFI_WEB_HTTPS_PORT
      value: "8443"
    - name: NIFI_WEB_PROXY_HOST
      value: <YOUR_HOST>
    - name: SINGLE_USER_CREDENTIALS_USERNAME
      value: nifi
    - name: SINGLE_USER_CREDENTIALS_PASSWORD
      value: nifinifinifi
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-name: nifi
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-type: external
  labels:
    app: nifi
  name: nifi
  namespace: nifi
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 8443
  selector:
    app: nifi
  sessionAffinity: ClientIP
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 10800
  type: LoadBalancer

7- Try to push/pull data from your S3 bucket

[mh013370]

LGTM

[exceptionfactory]

Thanks for the improvement and thorough background @juldrixx, and thanks for the feedback @mh013370! +1 merging