rpk: mix of list and single-value configs can cause failures in config set

[travisdowns]

Version & Environment

Redpanda version: (use rpk version): v21.10.1 (rev e7b6714)

What went wrong?

rpk acts strangely when "list or single" values are set on top of each other.

For example, consider advertised_kafka_api, it can be either a list or one value.

list of one:

  advertised_kafka_api:
  - address: 10.1.1.2
    port: 9092

one value:

  advertised_kafka_api:
    address: 10.1.1.2
    port: 9092

Note they differ here only in a single -.

The problem arises when the list is in one format, then an rpk config set uses the other format.

Repro (I have stripped out some irrelevant config values):

create empty yaml config w/o advertised_kafka_api:

$ ./rpk config init --config /tmp/foo.yaml
Couldn't find config file at /tmp/foo.yaml. Generating it.
$ cat /tmp/foo.yaml
config_file: /tmp/foo.yaml
node_uuid: kgw5f4G8HNHdhTxK4PmrMvyaAdUVHsThth5riyyD2rxZ3q5pY
pandaproxy: {}
redpanda:
  admin:
  - address: 0.0.0.0
    port: 9644
  data_directory: /var/lib/redpanda/data
  developer_mode: true
  kafka_api:
  - address: 0.0.0.0
    port: 9092
  node_id: 0
  rpc_server:
    address: 0.0.0.0
    port: 33145
  seed_servers: []
rpk:
  coredump_dir: /var/lib/redpanda/coredump

Set the advertised api using list syntax:

$ ./rpk config set redpanda.advertised_kafka_api '[{ address: 10.1.1.2, port: 9092 }]' --format yaml -v --config /tmp/foo.yaml
Backing up the current config
Backed up the current config to /tmp/foo.yaml.vectorized.1bd366be6e1e0d1be8a84f4ca5e64a9a.bk
Removing previous backup file
Writing the new redpanda config to '/tmp/foo.yaml'
Configuration written to /tmp/foo.yaml.
$ cat /tmp/foo.yaml
config_file: /tmp/foo.yaml
node_uuid: kgw5f4G8HNHdhTxK4PmrMvyaAdUVHsThth5riyyD2rxZ3q5pY
pandaproxy: {}
redpanda:
  admin:
  - address: 0.0.0.0
    port: 9644
  advertised_kafka_api:
  - address: 10.1.1.2
    port: 9092
  data_directory: /var/lib/redpanda/data
  developer_mode: true
  kafka_api:
  - address: 0.0.0.0
    port: 9092
  node_id: 0
  rpc_server:
    address: 0.0.0.0
    port: 33145
  seed_servers: []
rpk:
  coredump_dir: /var/lib/redpanda/coredump

Note it ends up in list format in the yaml.

Now try to set it using single value syntax:

$ ./rpk config set redpanda.advertised_kafka_api '{ address: 10.1.1.3, port: 9092 }' --format yaml -v --config /tmp/foo.yaml
ERROR 2021/11/12 21:09:38 svType != tvType; key=advertised_kafka_api, st=map[string]interface {}, tt=[]interface {}, sv=map[address:10.1.1.3 port:9092], tv=[map[address:10.1.1.2 port:9092]]
Backing up the current config
Backed up the current config to /tmp/foo.yaml.vectorized.5efd32efed3207a93fb8ffed33266769.bk
Removing previous backup file
Writing the new redpanda config to '/tmp/foo.yaml'
Configuration written to /tmp/foo.yaml.
$ cat /tmp/foo.yaml
config_file: /tmp/foo.yaml
node_uuid: kgw5f4G8HNHdhTxK4PmrMvyaAdUVHsThth5riyyD2rxZ3q5pY
pandaproxy: {}
redpanda:
  admin:
  - address: 0.0.0.0
    port: 9644
  advertised_kafka_api:
  - address: 10.1.1.2
    port: 9092
  data_directory: /var/lib/redpanda/data

Note several things:

• There is an error message
• Yet the writing is apparently successful (per the output)
• The IP didn't change to 10.1.1.3! So writing was not actually successful.
• The return code (not shown) was 0 (success).

So the merge error isn't being handled properly.

This is breaking some workflows in deployment-automation, because they always write with the single-value syntax, but later it gets transformed to the list representation, and so subsequent updates of this value fail (silently, as mentioned above).

[twmb]

Related: #4176

This should addressed when addressing #4530 if we remove the implicit magic, we can make these failures explicit.

[twmb]

Everything above will be possible in v22.2.1