rpk start doesn't support multiple listeners

[BenPope]

Example config:

config_file: /etc/redpanda/redpanda.yaml
redpanda:
  admin:
    address: 0.0.0.0
    port: 9644
  advertised_kafka_api:
  - address: redpanda-0.redpanda.redpanda.svc.cluster.local
    name: internal
    port: 9092
  - address: redpanda-0.my-project.gcp.domain.com.
    name: external
    port: 9092
  advertised_rpc_api:
    address: redpanda-0.redpanda.redpanda.svc.cluster.local
    port: 33145
  data_directory: /var/lib/redpanda/data
  developer_mode: true
  kafka_api:
    address: 0.0.0.0
    port: 9092
  node_id: 0
  rpc_server:
    address: 0.0.0.0
    port: 33145
  seed_servers: []
rpk:
  coredump_dir: /var/lib/redpanda/coredump
  enable_memory_locking: false
  enable_usage_stats: true
  overprovisioned: true
  tune_aio_events: false
  tune_clocksource: false
  tune_coredump: false
  tune_cpu: false
  tune_disk_irq: false

Error:

Command "start" is deprecated, use 'rpk redpanda start' instead.
Error: 1 error(s) decoding:
* 'redpanda.advertised_kafka_api' expected a map, got 'slice'

Some things to consider, when setting the config with rpk config set:

rpk config set redpanda.advertised_kafka_api '[{"address": "redpanda-0.redpanda.redpanda.svc.cluster.local", "port": 9092, "name":"internal"},{"address": "redpanda-0.my-project.gcp.domain.com.", "port": 9092, "name":"external"}]'

If the redpanda.advertised_kafka_api is already set in the config as map, then changing it to a list will fail, so the entire node needs replacing, not its contents.

And both map and list forms need to be supported.

[BenPope]

This blocks redpanda-data/helm-charts#16

[0x5d]

I'm working on a fix as we speak, but the main obstacle for "sane" config management I see is that we don't have clearly-defined versions.

So we could say config version 1 only supports a single advertised kafka addr and it requires seed_servers' items to have a {"host": {"address": "<address>", port: <port>}, "node_id": "<id>"} structure. Then we could consider v2 to support a list of adv. kafka addrs and to require that seed_servers' items be just {"address": "<address>", port: <port>}. But in reality we support any combination.

I think we should have clear config versions to make this easier.

[0x5d]

Alternatively we could have multiple versions not at the config file level but at the field level. But this would of course make config mgmt more complex.

[ivotron]

+1 on having versioned config schemas. A pain to maintain but necessary in the long run in order to maintain sanity. This also will eventually result in having to publish a compatibility table between redpanda/rpk and config schema versions

[emaxerrno]

+1 on versioned schemas.

[emaxerrno]

Everything storage related, should really come with versions. I can't think of anything we send over the wire or on-disk that should not have a version tbh.

[BenPope]

This is not fixed.

If the redpanda.advertised_kafka_api is already set in the config as map, then changing it to a list will fail, so the entire node needs replacing, not its contents.

docker run -it vectorized/redpanda:v21.2.2 redpanda start --advertise-kafka-addr=external://redpanda-0.example.com:9092,internal://192.168.1.2:9092 --kafka-addr=external://10.0.1.2:9092,internal://192.168.1.2:9092

ERROR 2021/02/18 14:57:43 svType != tvType; key=kafka_api, st=[]interface {}, tt=map[string]interface {}, sv=[map[address:10.0.1.2 name:external port:9092] map[address:192.168.1.2 name:internal port:9092]], tv=map[address:0.0.0.0 port:9092]