[BUG] The 2.12.0 initial deployment prevents the usage of the custom internal_users.yml file.

[tomte76]

Upgrade to 2.12.0, in my case using helm on kubernetes, failed with an error message related to setting OPENSEARCH_INITIAL_ADMIN_PASSWORD. The error message is:

Exception updating the admin password : /usr/share/opensearch/config/opensearch-security/internal_users.yml: Device or resource busy

As I understand the user pablo in the forum, you can use any deployment with defined internal users in the values.yml or in my case having an existing kubernetes secret used in the helm deployment and new parameter set in values.yaml:

extraEnvs:

• name: "OPENSEARCH_INITIAL_ADMIN_PASSWORD"
  value: "some_strong_password_here"

Please find more information in the forum discussion.

What is the expected behavior?

Upgrade succeeds and the existing security configuration is preserved.

What is your host/environment?

• OS: Debian 12 amd64
• Kubernetes v1.29
• Version: OpenSearch 2.11.1 to 2.12.0 upgrade with helm charts 2.17.0 to 2.18.0

[pawelw1]

The reported error appears when both internal_users and extraEnvs OPENSEARCH_INITIAL_ADMIN_PASSWORD are defined in values.yml during the deployment.
The working scenario is when only extranEnvs OPENSEARCH_INITIAL_ADMIN_PASSWORD is configured. This issue prevents deployment with custom internal_users.yml.

[DarshitChanpura]

@prudhvigodithi Can you please help here since I believe you are familiar with this issue?

[DarshitChanpura]

While we wait for inputs from @prudhvigodithi,
@tomte76 here is a comment from another user who faced similar issue: opensearch-project/helm-charts#519 (comment)

If your use-case is to setup demo certificates via demo config, you can instead generate custom certs and mount them. Here is a possible solution: opensearch-project/helm-charts#519 (comment)

Also, this issue seems to be duplicate of: #3891, and here is a solution from @prudhvigodithi to address that issue: #3891 (comment)

[prudhvigodithi]

Hey, to make it work with helm charts with this new security change, the way is to create the internal_users.yml as configmap and use initContainers to avoid the error Device or resource busy.
(#3891 (comment))

kubectl create configmap internal-users --from-file=internal_users.yml=internal_users.yml

extraVolumes:
  - name: internal-users-emptydir
    emptyDir: {}
  - name: internal-users
    configMap:
      name: internal-users
      items:
      - key: internal_users.yml
        path: internal_users.yml

extraVolumeMounts:
    - name: internal-users-emptydir
      mountPath: /usr/share/opensearch/config/opensearch-security/
      subPath: internal_users.yml
    - name: internal-users
      mountPath: /tmp/internal_users.yml
      subPath: internal_users.yml
   
extraInitContainers:
  - name: internal-users-init-container
    image: busybox
    command: ['sh', '-c', 'cp /tmp/internalusers/internal_users.yml /tmp/internal_users.yml']
    volumeMounts:
    - mountPath: /tmp/internal_users.yml
      subPath: internal_users.yml
      name: internal-users-emptydir
    - name: internal-users
      mountPath: /tmp/internalusers/internal_users.yml
      subPath: internal_users.yml

Set the securityConfig.config.dataComplete to false.

Adding @peterzhuamazon @bbarani

[scrawfor99]

[Triage] Hi @tomte76 thanks for filing this issue. It seems like there is already some work being put into this issue.

[DarshitChanpura]

Resolving it as duplicate of #3891