Load env vars from file or archive at startup#4053
Conversation
Adds LOAD_ENV_FROM_FILE and LOAD_ENV_FROM_ARCHIVE so a generic pack or artifact can ship its own container configuration (TYPE, VERSION, etc.) inside-out. Loaded values override anything passed via docker -e, and the load runs in start-configuration after proxy/Java setup but before TYPE is dispatched, so the loaded TYPE drives the deploy step. LOAD_ENV_FROM_ARCHIVE_ENTRY selects the file inside the archive (default: .env). EULA is intentionally NOT loadable this way since the EULA check runs earlier. Closes itzg#4037
itzg
left a comment
There was a problem hiding this comment.
Thanks for taking care of this. I obviously got myself distracted before starting on this. It ended up looking very clean.
Just a couple of things to consider, but am good with merging either way.
extract() now accepts optional trailing paths-within-archive arguments and forwards them to unzip/tar, so callers needing a single file from a multi-MB pack no longer pay to expand the whole thing. Existing callers that pass no extra args extract everything as before. loadEnvFromArchive uses this to extract just the entry it sources. Addresses PR review feedback on itzg#4053.
Some users hit issues with the container's /tmp (size limits, tmpfs unavailable, etc.). /data is already required to be writable and other parts of the codebase use /data/.tmp/... for transient unpack workspaces, so use the same location for the env-loading download and extraction scratch dirs. Addresses PR review feedback on itzg#4053.
Adds LOAD_ENV_FROM_GENERIC_PACK=true so a packed artifact's own .env can drive TYPE/VERSION/etc. without the user having to repeat the pack URL in LOAD_ENV_FROM_ARCHIVE. Each entry in GENERIC_PACKS (after PREFIX/ SUFFIX expansion) is probed for a top-level .env and sourced in pack order — later packs override earlier ones, matching the cp -R layering of the regular unpack step. Packs without a .env are skipped silently. URLs are downloaded into /data/packs/ using the same get invocation that start-setupModpack later runs with --skip-up-to-date, so the archive is not fetched twice. Load order across the three primitives: 1. LOAD_ENV_FROM_GENERIC_PACK (per-pack .env, in pack order) 2. LOAD_ENV_FROM_FILE 3. LOAD_ENV_FROM_ARCHIVE later loads override earlier, and all override docker -e. Closes itzg#4037
|
Added Behaviour:
The New test under |
itzg
left a comment
There was a problem hiding this comment.
Fantastic! Thanks for implementing this. It sounds like it'll help you out quite a bit.
Closes #4037
Summary
Adds four new variables that let a generic pack ship its own container configuration inside-out, so a pack artifact can declare its own
TYPE,VERSION, mod loader, etc. instead of relying on the consumer's outer env.LOAD_ENV_FROM_GENERIC_PACK— whentrue, each entry inGENERIC_PACKS(afterGENERIC_PACKS_PREFIX/SUFFIXexpansion) is probed for a top-level.envand sourced in pack order. Packs without a.envare skipped silently.LOAD_ENV_FROM_FILE— container path or URL of a shell-style env file.LOAD_ENV_FROM_ARCHIVE— container path or URL of a zip/tar archive containing an env file.LOAD_ENV_FROM_ARCHIVE_ENTRY— relative path of the env file inside the archive (default.env).Why
Per the issue, packs shipped as artifacts shouldn't break their consumers when the recommended
TYPE/VERSIONfor the pack changes, the pack itself should be the source of truth. TheLOAD_ENV_FROM_GENERIC_PACKflag is for the original ask (env file lives inside the generic pack); theFROM_FILE/FROM_ARCHIVEprimitives cover the orthogonal case of an env source that isn't a generic pack (separate config bundle, packwiz zip used with a different TYPE, etc.).Implementation notes
loadEnvFromFile/loadEnvFromArchive/loadEnvFromGenericPackinscripts/start-utils, reusing the existingapplyResultsFile(set -a; source; set +a) pattern called out in the issue.scripts/start-configurationright aftercd /dataand beforeDECLARED_TYPE/DECLARED_VERSIONare captured, so the loadedTYPE/VERSIONflow through the existing dispatch.GENERIC_PACK→FROM_FILE→FROM_ARCHIVE; later wins, and all overridedocker run -e/ composeenvironment:.LOAD_ENV_FROM_GENERIC_PACKdownloads each pack URL into/data/packs/using the sameget --skip-up-to-datecall thatstart-setupModpackmakes later, so the archive is not fetched twice.extract()now accepts optional trailing paths-within-archive arguments and forwards them tounzip/tar(per review feedback), so probing a 100 MB pack for a 200 B.envdoesn't expand the whole thing./data/.tmprather than/tmp(per review feedback), matching the existing convention instart-setupModpack.EULAis intentionally not settable this way — the EULA check runs earlier instart-configuration, before any env is loaded. Documented as such.!!! warningadmonition flags that the env file is sourced by bash and should only point at trusted sources.Tests
Three new setup-only tests under
tests/setuponlytests/:load-env-from-generic-pack/— setsMOTD: from-composein compose andMOTD=from-generic-packinside the pack's.env; assertsserver.propertieshas the pack's value AND that the pack'sconfig/dummy.ymlmade it into/data(so the regular unpack step still ran on the same archive).load-env-from-file/— setsMOTD: from-composein compose andMOTD=from-env-filein the loaded file; asserts the env-file value won.load-env-from-archive/— same idea using a zip with.envat its root.Out of scope
The packwiz
pack.toml→ TYPE/VERSION inference path is being tracked separately in #1818, per discussion on the issue.Test plan
tests/setuponlytests/load-env-from-generic-packPASSED verifytests/setuponlytests/load-env-from-filePASSED verifytests/setuponlytests/load-env-from-archivePASSED verifysetuponlyandfulltestsstill PASSbash -n scripts/start-configuration scripts/start-utilsclean