Support for custom realms for Elasticsearch authentication.

[nkvoll]

By default we will use the file realm for our internal uses and let consumers of our deployment use the ES native realm. For larger deployments, custom realms become more important:

These resources might have to include a versioned component as well (not all of these are as important as the others):

• SAML
• LDAP
• Oauth / OpenID connect

In some cases, this would also entail installing a certificate (e.g ldaps) so we can support encryption.

[nkvoll]

Now supported through a mixture of podTemplate and `config.

[iahmad-khan]

Hi,

Any documentation how to enable LDAP user auth through ECK operator?

[thbkrkr]

Any documentation how to enable LDAP user auth through ECK operator?

There is no special thing to do to configure LDAP authentication for Elasticsearch when using ECK.

See the Elasticsearch documentation to configure LDAP: https://www.elastic.co/guide/en/elasticsearch/reference/current/ldap-realm.html#ldap-realm-configuration

See the ECK documentation to add settings to the Elasticsearch configuration:
https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-node-configuration.html

Note that in version 1.0.0-beta1, we introduced a regression that by default disables the native realm. If you want it, you need explicitely declare it (https://discuss.elastic.co/t/new-user-cant-login-kibana/204810/2).

[iahmad-khan]

Great, Thanks

[iahmad-khan]

@thbkrkr I tried this config but doesnt seem to take any effect , is there a way to chck where it is failing? I don't see any ldap config created by operator.

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1beta1
kind: Elasticsearch
metadata:
  name: livelogs
  namespace: livelogs
spec:
  version: 7.5.0
  nodeSets:
  - name: livelogs
    count: 3
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          image: docker.elastic.co/elasticsearch/elasticsearch:7.5.0
          resources:
            limits:
              memory: 24Gi
              cpu: 4
          env:
          - name: ES_JAVA_OPTS
            value: "-Xms16g -Xmx16g"
  http:
    service:
      spec:
        type: NodePort
    tls:
      selfSignedCertificate:
        disabled: true
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 500Gi
        storageClassName: standard-01
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: true
      xpack.security.authc.realms:
        ldap:
          ldap1:
            order: 0
            url: "ldaps://hh-ldap.mydomain"
            bind_dn: "ou=people, dc=example, dc=com"
            user_search:
              base_dn: "dc=example,dc=com"
              filter: "(uid={0})"
            group_search:
              base_dn: "example,dc=com"
EOF

[thbkrkr]

The volumeClaimTemplates and config fields must be under a nodeSet object. In your manifest, the http section is misplaced and breaks this.

Corrected manifest:

apiVersion: elasticsearch.k8s.elastic.co/v1beta1
kind: Elasticsearch
metadata:
  name: livelogs
  namespace: livelogs
spec:
  version: 7.5.0
  nodeSets:
  - name: livelogs
    count: 3
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          image: docker.elastic.co/elasticsearch/elasticsearch:7.5.0
          resources:
            limits:
              memory: 24Gi
              cpu: 4
          env:
          - name: ES_JAVA_OPTS
            value: "-Xms16g -Xmx16g"
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: true
      xpack.security.authc.realms:
        ldap:
          ldap1:
            order: 0
            url: "ldaps://hh-ldap.mydomain"
            bind_dn: "ou=people, dc=example, dc=com"
            user_search:
              base_dn: "dc=example,dc=com"
              filter: "(uid={0})"
            group_search:
              base_dn: "example,dc=com"
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 500Gi
        storageClassName: standard-01
  http:
    service:
      spec:
        type: NodePort
    tls:
      selfSignedCertificate:
        disabled: true

[iahmad-khan]

@thbkrkr still not able to login , which pod logs should I check to see any ldap related errors.

[iahmad-khan]

{"type": "server", "timestamp": "2020-01-08T12:50:38,453Z", "level": "WARN", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "livelogs", "node.name": "livelogs-es-livelogs-1", "message": "Authentication to realm ldap1 failed - authenticate failed (Caused by LDAPException(resultCode=89 (parameter error), diagnosticMessage='Simple bind operations are not allowed to contain a bind DN without a password.', ldapSDKVersion=4.0.8, revision=28812))", "cluster.uuid": "Ji3ZceuDSoml98DBNZlxmQ", "node.id": "ZEx_070DR_mkjQYtNI5evg"  }

but our ldap setup is working without password for other services.

[thbkrkr]

According to the documentation https://www.elastic.co/guide/en/elasticsearch/reference/master/active-directory-realm.html, you need to add a xpack.security.authc.realms.ldap.ldap1.secure_bind_password setting in the keystore when you use a bind user.

Since it is not related to ECK, please use https://discuss.elastic.co/c/elasticsearch/6 to ask this kind of questions about setting up Elasticsearch.