Hard coded static routes not respecting namespace

[jbury]

I'm one of the few users that can't install nats using helm, hence my recent changes adding support to the install scripts/bootstrapping images to support namespacing. In using the install scripts, I'm installing nats in the nats namespace. Unfortunately, the default configuration for the nats-config ConfigMap hard-codes some routes that aren't valid if you're running in any namespace other than default. I believe the relevant version of the config map is this one, though I'm not 100% sure as the configmaps themselves don't contain a reference to their original filename.

https://github.com/nats-io/k8s/blob/main/nats-server/simple-nats.yml

I'm not sure how this should be handled, though I'm more than happy to actually do the implementation myself if it'll take some legwork.

The immediate fix I've implemented is that I manually edited the configmap to replace

        nats://nats-0.nats.default.svc:6222
        nats://nats-1.nats.default.svc:6222
        nats://nats-2.nats.default.svc:6222

with

        nats://nats-0.nats.nats.svc:6222
        nats://nats-1.nats.nats.svc:6222
        nats://nats-2.nats.nats.svc:6222

Which I've tested and it works just fine, but I'd prefer to fix it for everyone using the install script, and not just me :) Any ideas/guidance?

[wallyqs]

that's right, those would be the required changes to connect. I think that one workaround that might work is in the config yaml to do this instead:

 env:
        - name: CLUSTER_ADVERTISE
          value: $(POD_NAME).nats.$(POD_NAMESPACE).svc
        - name: CLUSTER_ROUTES
          value: "[ nats-0.nats.$(POD_NAMESPACE).svc,nats-1.nats.$(POD_NAMESPACE).svc,nats-2.nats.$(POD_NAMESPACE).svc ]"

and then in the ConfigMap change the configuration so that it becomes:

routes = $CLUSTER_ROUTES

https://github.com/nats-io/k8s/blob/main/nats-server/simple-nats.yml#L13-L17

[Jarema]

@wallyqs wouldn't #421 and using short domain names by default clear this out?

[jbury]

@Jarema correct me if I'm wrong, but it looks like the short domain names you're talking about still hard-code the namespace (default), and that's the main issue here.

[Jarema]

@jbury no, the namespace is calculated ("$(POD_NAME).%s.$(POD_NAMESPACE)" (include "nats.fullname").

[jbury]

Oho, that'd do the trick then, I think @Jarema.

@wallyqs On a scale from 1-10, 1 being "jbury is the official mantainer now" to 10 being "It should be 1:1", how maintained are these non-helm setup scripts compared to the helm setup process? Is it worth me even patching these? Is there a plan to just remove these setup scripts altogether?

[wallyqs]

I think we should remove some of them but we could keep a few, like the simple-nats one to keep as a reference.

[jbury]

@wallyqs I have no vision for what to keep/what to nuke, but if one of ya'll maintainers wants to clean out the stuff we explicitly don't want, I'd be happy to make this change in whatever is left over.

[jbury]

Per convo in 396, I'm just closing this one altogether and letting @wallyqs deal with the scripts cleanup however he pleases.