MTLS automatically enabled on running `dapr upgrade` on a cluster with MTLS disabled

[mukundansundar]

Expected Behavior

dapr upgrade command does not change the mtls setting from disabled to enabled.

Actual Behavior

dapr upgrade command turns on mtls even if it was disabled previously.

Steps to Reproduce the Problem

Run `dapr init` with version 1.0.0 with mtls disabled

$ dapr init -k --runtime-version 1.0.0 --enable-mtls=false
⌛  Making the jump to hyperspace...
ℹ️  Note: To install Dapr using Helm, see here: https://docs.dapr.io/getting-started/install-dapr-kubernetes/#install-with-helm-advanced

✅  Deploying the Dapr control plane to your cluster...
✅  Success! Dapr has been installed to namespace dapr-system. To verify, run `dapr status -k' in your terminal. To get started, go here: https://aka.ms/dapr-getting-started

$ dapr mtls -k
Mutual TLS is disabled in your Kubernetes cluster
$ k get configurations.dapr.io -n dapr-system daprsystem -o yaml

apiVersion: dapr.io/v1alpha1
kind: Configuration
metadata:
  annotations:
    meta.helm.sh/release-name: dapr
    meta.helm.sh/release-namespace: dapr-system
  creationTimestamp: "2021-04-04T17:26:34Z"
  generation: 1
  labels:
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: dapr.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:meta.helm.sh/release-name: {}
          f:meta.helm.sh/release-namespace: {}
        f:labels:
          .: {}
          f:app.kubernetes.io/managed-by: {}
      f:spec:
        .: {}
        f:metric:
          .: {}
          f:enabled: {}
        f:mtls:
          .: {}
          f:allowedClockSkew: {}
          f:enabled: {}
          f:workloadCertTTL: {}
    manager: Go-http-client
    operation: Update
    time: "2021-04-04T17:26:34Z"
  name: daprsystem
  namespace: dapr-system
  resourceVersion: "178509"
  uid: c4f9664c-ae4e-442e-baad-5ec2198bb559
spec:
  metric:
    enabled: true
  mtls:
    allowedClockSkew: 15m
    enabled: false
    workloadCertTTL: 24h

Upgrade to 1.1.0 version

$ dapr upgrade -k --runtime-version 1.1.0
ℹ️  Dapr control plane version 1.0.0 detected in namespace dapr-system
ℹ️  Starting upgrade...
✅  Dapr control plane successfully upgraded to version 1.1.0. Make sure your deployments are restarted to pick up the latest sidecar version.

Check and see that mtls is enabled again

$ dapr mtls -k
Mutual TLS is enabled in your Kubernetes cluster
$ k get configurations.dapr.io -n dapr-system daprsystem -o yaml

apiVersion: dapr.io/v1alpha1
kind: Configuration
metadata:
  annotations:
    meta.helm.sh/release-name: dapr
    meta.helm.sh/release-namespace: dapr-system
  creationTimestamp: "2021-04-04T17:26:34Z"
  generation: 2
  labels:
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: dapr.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:meta.helm.sh/release-name: {}
          f:meta.helm.sh/release-namespace: {}
        f:labels:
          .: {}
          f:app.kubernetes.io/managed-by: {}
      f:spec:
        .: {}
        f:metric:
          .: {}
          f:enabled: {}
        f:mtls:
          .: {}
          f:allowedClockSkew: {}
          f:enabled: {}
          f:workloadCertTTL: {}
    manager: Go-http-client
    operation: Update
    time: "2021-04-04T17:26:34Z"
  name: daprsystem
  namespace: dapr-system
  resourceVersion: "178866"
  uid: c4f9664c-ae4e-442e-baad-5ec2198bb559
spec:
  metric:
    enabled: true
  mtls:
    allowedClockSkew: 15m
    enabled: true
    workloadCertTTL: 24h

Release Note

RELEASE NOTE: RESOLVED MTLS automatically enabled on running dapr upgrade on a cluster with MTLS disabled

[mukundansundar]

@yaron2 Please see the draft PR also which shows that this happens. Can you take a look at this?

[mukundansundar]

There is a workaround for this: Use --set flag to override the mtls value on upgrade if it was previously disabled.

$ dapr init --runtime-version 1.0.0 -k --enable-mtls=false
⌛  Making the jump to hyperspace...
ℹ️  Note: To install Dapr using Helm, see here: https://docs.dapr.io/getting-started/install-dapr-kubernetes/#install-with-helm-advanced

✅  Deploying the Dapr control plane to your cluster...
✅  Success! Dapr has been installed to namespace dapr-system. To verify, run `dapr status -k' in your terminal. To get started, go here: https://aka.ms/dapr-getting-started

$ dapr mtls -k
Mutual TLS is disabled in your Kubernetes cluster

$ dapr upgrade --runtime-version 1.1.0 -k --set global.mtls.enabled=false
ℹ️  Dapr control plane version 1.0.0 detected in namespace dapr-system
ℹ️  Starting upgrade...
✅  Dapr control plane successfully upgraded to version 1.1.0. Make sure your deployments are restarted to pick up the latest sidecar version.

$ dapr mtls -k
Mutual TLS is disabled in your Kubernetes cluster