OpenSearchIndexManager: Add cross-account assume-role support to AWS SigV4 authentication for OpenSearch Serverless

[bwright86]

Problem

When OpenMetadata is deployed on EKS in a different AWS account from the OpenSearch Serverless (AOSS) collection, the existing SigV4 authentication path fails with:

Unexpected OpensearchException while getting alias vector_search_index:
Request failed: [security_exception] authentication/authorization failure

The OpenSearchIndexManager (and the SigV4RequestSigningInterceptor it uses) signs requests using whatever credentials DefaultCredentialsProvider resolves — in an EKS deployment, this is the pod's IRSA role. AOSS data access policies only accept principals from the same account as the collection. A cross-account IAM role cannot be added as a principal in an AOSS data access policy, regardless of trust policy configuration.

The required credential chain for cross-account access is:

Pod IRSA role (EKS account A)
  → sts:AssumeRole → data access role (AOSS account B)
    → signs SigV4 AOSS requests ✓

There is currently no way to configure this in the OpenSearch connection settings.

Proposed Solution

Add an optional assumeRoleArn field to the OpenSearch connection configuration. When set, wrap the base credential provider with StsAssumeRoleCredentialsProvider before constructing the SigV4RequestSigningInterceptor.

Required dependency (likely already transitive via the AWS SDK BOM):

software.amazon.awssdk:sts

Implementation sketch:

private AwsCredentialsProvider buildCredentialsProvider(
        String awsRegion, String assumeRoleArn) {
    AwsCredentialsProvider base = DefaultCredentialsProvider.create();
    if (assumeRoleArn == null || assumeRoleArn.isBlank()) {
        return base;
    }
    return StsAssumeRoleCredentialsProvider.builder()
        .stsClient(StsClient.builder()
            .credentialsProvider(base)
            .region(Region.of(awsRegion))
            .build())
        .refreshRequest(AssumeRoleRequest.builder()
            .roleArn(assumeRoleArn)
            .roleSessionName("openmetadata-opensearch")
            .build())
        .build();
}

Configuration — extend the existing OpenSearch connection schema or Helm values:

# values.yaml
elasticsearch:
  searchType: opensearch
  host: <collection-id>.us-east-1.aoss.amazonaws.com
  port: 443
  scheme: https
  # New field:
  assumeRoleArn: "arn:aws:iam::123456789012:role/openmetadata-aoss-access"

This mirrors the same pattern proposed for RDS IAM cross-account auth (see related issue #27552 ).

Environment

• OpenMetadata: 1.12.4 / 1.12.5
• Search backend: AWS OpenSearch Serverless (AOSS)
• EKS cluster account: separate from the AOSS collection account
• AOSS constraint: data access policies only accept same-account principals

---

[Darshan3690]

Hi @bwright86, @harshach @pmbrull @PubChimps sir

I’ve already contributed by resolving 2 issues in this repository, and I understand the current limitation around SigV4 authentication when OpenMetadata is deployed cross-account with AOSS.

Proposed approach:

• Introduce an optional assumeRoleArn configuration in the OpenSearch connection settings.

• Extend the credential provider chain to:

  • Use DefaultCredentialsProvider as base (IRSA in EKS)
  • Wrap it with StsAssumeRoleCredentialsProvider when assumeRoleArn is provided

• Ensure backward compatibility (no impact when assumeRoleArn is not set)

• Add unit tests for both default and assume-role flows

This aligns well with the implementation sketch and mirrors the RDS IAM cross-account pattern already discussed.

Please assign this issue to me — I’d be happy to take it forward.

Thanks!

[harshach]

@Darshan3690 unless you can actually test this with real AWS infra, with these settings enabled, we won’t be able to accept the PR here

[bwright86]

I could probably help with this, as we are currently deploying this tool in my work environment, and we use multi-account setup with OpenSearch in an central account.
If there is a branch and some instructions on how to build the image, I can build and confirm that it works. If that helps here...

Is it sufficient to see that this addition doesn't cause any logic errors with/without these new settings?