mgr: fixing dashboard configuration handling

[rkachach]

previously, the dashboard parameters supported by Rook were stored in the daemon configuration section (mgr.X, for example). This differs from Cephadm-based deployments, where all configurations are stored in the global mgr configuration section. This variance could result in configuration mismatches between the active and standby dashboards. Furthermore, all Ceph dashboard documentation exclusively points to the global mgr configuration section and makes no use of individual daemons sections.

Fixes: #13577

Upgrade test

Starting from previous vesion (image: rook/ceph:master) and using the following configuration:

  mgr:
    count: 2
    allowMultiplePerNode: true
  dashboard:
    enabled: true
    port: 7777
    ssl: false
    prometheusEndpoint: "http://192.168.39.54:30900"
    prometheusEndpointSSLVerify: false

Let's get the status of the config once the cluster is up&running:

bash-4.4$ ceph config get mgr.a
WHO     MASK  LEVEL     OPTION                                   VALUE                       RO
mgr.a         advanced  mgr/dashboard/PROMETHEUS_API_HOST        http://192.168.39.54:30900  * 
mgr.a         advanced  mgr/dashboard/PROMETHEUS_API_SSL_VERIFY  false                       * 
mgr.a         advanced  mgr/dashboard/server_port                7777                        * 
mgr.a         advanced  mgr/dashboard/ssl                        false                       * 
mgr           advanced  mgr/orchestrator/orchestrator            rook                          
mgr           advanced  mgr/prometheus/rbd_stats_pools                                       * 

bash-4.4$ ceph config get mgr.b
WHO     MASK  LEVEL     OPTION                                   VALUE                       RO
mgr.b         advanced  mgr/dashboard/PROMETHEUS_API_HOST        http://192.168.39.54:30900  * 
mgr.b         advanced  mgr/dashboard/PROMETHEUS_API_SSL_VERIFY  false                       * 
mgr.b         advanced  mgr/dashboard/server_port                7777                        * 
mgr.b         advanced  mgr/dashboard/ssl                        false                       * 
mgr           advanced  mgr/orchestrator/orchestrator            rook                          
mgr           advanced  mgr/prometheus/rbd_stats_pools                                       * 

bash-4.4$ ceph config get mgr  
WHO     MASK  LEVEL     OPTION                             VALUE                        RO
mgr           advanced  mgr/dashboard/PROMETHEUS_API_HOST  http://192.168.39.215:30900  * 
mgr           advanced  mgr/orchestrator/orchestrator      rook                           
mgr           advanced  mgr/prometheus/rbd_stats_pools                                  * 
bash-4.4$ 

Mgr daemons configuration after the upgrade (no specific per-daemon configuration anymore). The configuration related to ssl now is part of the main mgr section.

bash-4.4$ ceph config get mgr.a
WHO     MASK  LEVEL     OPTION                                   VALUE                       RO
mgr           advanced  mgr/dashboard/PROMETHEUS_API_HOST        http://192.168.39.54:30900  * 
mgr           advanced  mgr/dashboard/PROMETHEUS_API_SSL_VERIFY  false                       * 
mgr           advanced  mgr/dashboard/server_port                7777                        * 
mgr           advanced  mgr/dashboard/ssl                        false                       * 
mgr           advanced  mgr/orchestrator/orchestrator            rook                          
mgr           advanced  mgr/prometheus/rbd_stats_pools                                       * 

bash-4.4$ 
bash-4.4$ ceph config get mgr.b
WHO     MASK  LEVEL     OPTION                                   VALUE                       RO
mgr           advanced  mgr/dashboard/PROMETHEUS_API_HOST        http://192.168.39.54:30900  * 
mgr           advanced  mgr/dashboard/PROMETHEUS_API_SSL_VERIFY  false                       * 
mgr           advanced  mgr/dashboard/server_port                7777                        * 
mgr           advanced  mgr/dashboard/ssl                        false                       * 
mgr           advanced  mgr/orchestrator/orchestrator            rook                          
mgr           advanced  mgr/prometheus/rbd_stats_pools                                       * 
bash-4.4$ 

bash-4.4$ ceph config get mgr  
WHO     MASK  LEVEL     OPTION                                   VALUE                       RO
mgr           advanced  mgr/dashboard/PROMETHEUS_API_HOST        http://192.168.39.54:30900  * 
mgr           advanced  mgr/dashboard/PROMETHEUS_API_SSL_VERIFY  false                       * 
mgr           advanced  mgr/dashboard/server_port                7777                        * 
mgr           advanced  mgr/dashboard/ssl                        false                       * 
mgr           advanced  mgr/orchestrator/orchestrator            rook                          
mgr           advanced  mgr/prometheus/rbd_stats_pools                                       * 

Checklist:

• Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
• Reviewed the developer guide on Submitting a Pull Request
• Pending release notes updated with breaking and/or notable changes for the next minor release.
• Documentation has been updated, if necessary.
• Unit tests have been added, if necessary.
• Integration tests have been added, if necessary.

[jmolmo]

Good catch!!! LGTM.
Just review we are not "getting" dashboard options in other operator places.

[parth-gr]

bash-4.4$ ceph config get mgr  
WHO     MASK  LEVEL     OPTION                                   VALUE                       RO
mgr           advanced  mgr/dashboard/PROMETHEUS_API_HOST        http://192.168.39.54:30900  * 
mgr           advanced  mgr/dashboard/PROMETHEUS_API_SSL_VERIFY  false                       * 
mgr           advanced  mgr/dashboard/server_port                7777                        * 
mgr           advanced  mgr/dashboard/ssl                        false                       * 
mgr           advanced  mgr/orchestrator/orchestrator            rook                          
mgr           advanced  mgr/prometheus/rbd_stats_pools                                       * 

This is the active mgr configuration?

[rkachach]

bash-4.4$ ceph config get mgr  
WHO     MASK  LEVEL     OPTION                                   VALUE                       RO
mgr           advanced  mgr/dashboard/PROMETHEUS_API_HOST        http://192.168.39.54:30900  * 
mgr           advanced  mgr/dashboard/PROMETHEUS_API_SSL_VERIFY  false                       * 
mgr           advanced  mgr/dashboard/server_port                7777                        * 
mgr           advanced  mgr/dashboard/ssl                        false                       * 
mgr           advanced  mgr/orchestrator/orchestrator            rook                          
mgr           advanced  mgr/prometheus/rbd_stats_pools                                       * 

This is the active mgr configuration?

This the general mgr configuration and it affects to ALL the mgr daemons. Previously we were having the config per-daemon which could lead to some inconsistencies in the conf and is different from what cephadm based deployments (where all the config is at manager level)