[Feature] Opensearch Replication Plugin - Proxy mode connection setup

[Firdaus1]

Hello Opensearch community,

Is there a way to setup opensearch replication across 2 different kubernetes clusters?

Currently, the /_cluster/settings?pretty API accepts the following body:
{
"persistent": {
"cluster": {
"remote": {
"opensearch-cluster-alias": {
"seeds": ["0.0.0.0:9300"]
}
}
}
}
}
where “seeds” is the elastic discovery service name or IP address.
What I am trying to accomplish is use kubernetes ingress to route traffic to port 9300.

Any suggestions ?
Is replication plugin meant to be implemented across different kubernetes clusters ?

Thank you in advance,
Firdaus

[soosinha]

I have not tried setting up replication using kubernetes clusters.

Other than the remote cluster setup you mentioned, you can also setup a connection using a proxy address which forwards to the OpenSearch cluster LB.
The remote connection can be created using the proxy mode like below.

_cluster/settings?pretty -d '
{
  "persistent": {
    "cluster": {
      "remote": {
        "opensearch-cluster-alias": {
          "mode": "proxy",
          "proxy_address": "<proxy IP:proxy port>"
        }
      }
    }
  }
}'

Let us know if this works for you.

[peace-tree]

@soosinha I could not find remote connection creation using proxy mode in the documentation.
Is it officially supported? If yes, then are there plans to include it in documentation?

[maksym-sereda]

@Firdaus1 , @namrataa-p did you configure it? Please let me know how? Because the solution mentioned by @soosinha doesn't work for me

[draemi]

We do have also this use-case when we need to perform CCR between 2 k8s clusters.
When using proxy mode, CCR is not working even if remote connection is established.

Is there any limitation regarding using proxy mode? Couldn't find anything in the documentation.
Also is there any plan supporting this model?

[saikaranam-amazon]

Currently, We don't have support for replication with proxy mode for remote cluster connection.
I will let this issue remain open to track this feature request.

[mrudrego]

Hi,
We are facing the same issue in deploying CCR across Kubernetes cluster and got to know from the ticket that CCR is not currently supported across Kubernetes Clusters. Please let us know when this enhancement is planned.

Thanks,

[saikaranam-amazon]

We are tracking v2.5.0 for this issue.

[Gitwiththewind]

I noticed that this feature is not marked as part of the 2.7.0 release on the opensearch roadmap. Is this because this CCR capability is being classified as a bug and not a feature?

[mrudrego]

Hi,
Can you please confirm if this enhancement is on track for v2.7.0 release which is planned for April ?

Thanks,

[saikaranam-amazon]

@mrudrego we are finalizing the tasks under cross cluster replication for v2.7.0.
Yes, Currently, we are tracking for v2.7.0

[mrudrego]

@saikaranam-amazon, good to hear that and thanks for the response .

[mrudrego]

Hi,
Is CCR supported across 2 kubernetes cluster is 2.7 version of OpenSearch ?
If so, can you please share the configuration could not find anything specific to proxy based which i believe is required for configuring CCR across K8s cluster.

Thanks,

[ishankka]

Hi @mrudrego,
I havent tried CCR between 2 k8s, but we do support 2.7 with proxy mode connection.

Can you try this configuration
The remote connection can be created using the proxy mode like below.

_cluster/settings?pretty -d '
{
  "persistent": {
    "cluster": {
      "remote": {
        "opensearch-cluster-alias": {
          "mode": "proxy",
          "proxy_address": "<proxy IP:proxy port>"
        }
      }
    }
  }
}'

[monusingh-1]

Addressed here #786