Allow referencing env.sh vars in configs

[nineinchnick]

Allow referencing env vars from env.sh in configs.

Since this would allow users to add their custom vars to env.sh treat it as a config file so it won't be removed automatically on uninstall. Note that by default node.properties also won't be removed, as reported in #314

[findepi]

@nineinchnick what about adding the -E switch conditionally (off by default)?
It could be configured by changing a variable in the init script, or by overriding it in env.sh script.

Or, maybe even better, we could ask the env.sh script to define an array variable (or comma-separated) with names of all env vars it intends to export.
Then the sudo invocation would export them to the subprocess, perhaps using --preserve-env=..., or by injecting additional env k="${k}" .. under sudo call.

wdyt?

[nineinchnick]

@nineinchnick what about adding the -E switch conditionally (off by default)?
It could be configured by changing a variable in the init script, or by overriding it in env.sh script.

Or, maybe even better, we could ask the env.sh script to define an array variable (or comma-separated) with names of all env vars it intends to export.
Then the sudo invocation would export them to the subprocess, perhaps using --preserve-env=..., or by injecting additional env k="${k}" .. under sudo call.

wdyt?

It looks cool! Turns out I can declare -A CONFIG_ENV before sourcing env.sh! To avoid quoting issues I'm only passing names of exported vars to --preserve-env=.... I hope this will be clear enough to users, I added one example value in env.sh.

[findepi]

Looks great!

[findepi]

looks like local testing modification

it would be nice to have a test for that, but let's make it a dedicated one

[nineinchnick]

It's not a local testing modification. It extends the test coverage to ensure envs are passed properly. This is an integration test, not a unit test. I'm concerned that a dedicated test would increase the total duration. Are you sure it's worth putting in a separate test?

[findepi]

How much time we would add and how much time the job already takes today?

BTW the test is not very directly (there is no explicit assertion, so it could in theory become ignored by the server.)
Can we inject eg node.environment and verify what's the actual value of that?

[nineinchnick]

Trino throws an error when configuration files reference an unset env var.

[nineinchnick]

These two tests in ServerIT run in the pipeline in slightly over a minute:

[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.948 s - in io.trino.server.rpm.ServerIT

so I'd expect a separate test to add ~30 seconds.

[findepi]

And the job total time is 20 minute. I think it's reasonable to have this as a separate test, but i think i am OK keeping this inline.

would it be possible to add an explicit assertion for the feature?

[nineinchnick]

I added a custom node id and an assert for it.

[findepi]

How much time we would add and how much time the job already takes today?

BTW the test is not very directly (there is no explicit assertion, so it could in theory become ignored by the server.)
Can we inject eg node.environment and verify what's the actual value of that?