[nats helm 1.x] fix JS mount

[caleblloyd]

Fixes #716

In 1.0.0-beta.2 and below we are mounting to /data/jetstream and the store dir is /data

So it looks like NATS Server is detecting the size of /data which is outside of the PVC mount

This is going to be a problem when upgrading from 0.x anyways, because in 0.x we are mounting to /data and the store dir is /data

This updates the mount dir and store dir to match and default to /data

There is a one-time-fix initContainer that will be released as part of 1.0.0-beta.3 and removed after that

[samuelattwood]

LGTM

[caleblloyd]

Well the initContainer fixup is working, but NATS is still not auto-detecting the correct storage size from the mount - it should be 10GB

1.0.0-beta.2

kubectl logs nats-0 -c nats | grep -B3 -A1 -F 'Store Directory'
[7] 2023/05/09 15:43:32.082047 [INF] ---------------- JETSTREAM ----------------
[7] 2023/05/09 15:43:32.082049 [INF]   Max Memory:      0 B
[7] 2023/05/09 15:43:32.082051 [INF]   Max Storage:     7.25 GB
[7] 2023/05/09 15:43:32.082052 [INF]   Store Directory: "/data/jetstream"
[7] 2023/05/09 15:43:32.082053 [INF] -------------------------------------------


kubectl exec -it nats-0 -c nats -- find /data -maxdepth 2
/data
/data/jetstream
/data/jetstream/lost+found
/data/jetstream/$SYS
/data/jetstream/$G

This PR

kubectl logs nats-0 -c nats | grep -B3 -A1 -F 'Store Directory'
[16] 2023/05/09 15:49:46.241840 [INF] ---------------- JETSTREAM ----------------
[16] 2023/05/09 15:49:46.241842 [INF]   Max Memory:      0 B
[16] 2023/05/09 15:49:46.241845 [INF]   Max Storage:     7.25 GB
[16] 2023/05/09 15:49:46.241847 [INF]   Store Directory: "/data/jetstream"
[16] 2023/05/09 15:49:46.241848 [INF] -------------------------------------------


kubectl exec -it nats-0 -c nats -- find /data -maxdepth 2
/data
/data/lost+found
/data/jetstream
/data/jetstream/$SYS
/data/jetstream/$G

[caleblloyd]

Going further down the rabbit hole - it turns out that the auto-disk calculation is 75% of available space, but it's got an issue because it will shrink after restarting

nats-io/nats-server#4140

So it seems that the best path forward right now is to adopt the 0.x behavior of defaulting max_file_storage to the PVC size. Although there is still the option to override it with maxSize.

It is not all that useful to be set to the PVC Size, because the actual disk is smaller when formatted. For example a 10Gi disk has 9.7Gi usable after formatting. So a max_file_storage of 10Gi will never be hit, the disk will fill up first. But it is the same behavior that 0.x has today so I digress

[caleblloyd]

Alright at least the logs look right now 😄

[15] 2023/05/09 20:01:53.040642 [INF] ---------------- JETSTREAM ----------------
[15] 2023/05/09 20:01:53.040645 [INF]   Max Memory:      0 B
[15] 2023/05/09 20:01:53.040648 [INF]   Max Storage:     10.00 GB
[15] 2023/05/09 20:01:53.040650 [INF]   Store Directory: "/data/jetstream"
[15] 2023/05/09 20:01:53.040651 [INF] -------------------------------------------