Skip to content

Load env vars from file or archive at startup#4053

Merged
itzg merged 4 commits into
itzg:masterfrom
ChipWolf:feat/load-env-from-pack
May 12, 2026
Merged

Load env vars from file or archive at startup#4053
itzg merged 4 commits into
itzg:masterfrom
ChipWolf:feat/load-env-from-pack

Conversation

@ChipWolf
Copy link
Copy Markdown
Contributor

@ChipWolf ChipWolf commented May 10, 2026

Closes #4037

Summary

Adds four new variables that let a generic pack ship its own container configuration inside-out, so a pack artifact can declare its own TYPE, VERSION, mod loader, etc. instead of relying on the consumer's outer env.

  • LOAD_ENV_FROM_GENERIC_PACK — when true, each entry in GENERIC_PACKS (after GENERIC_PACKS_PREFIX/SUFFIX expansion) is probed for a top-level .env and sourced in pack order. Packs without a .env are skipped silently.
  • LOAD_ENV_FROM_FILE — container path or URL of a shell-style env file.
  • LOAD_ENV_FROM_ARCHIVE — container path or URL of a zip/tar archive containing an env file.
  • LOAD_ENV_FROM_ARCHIVE_ENTRY — relative path of the env file inside the archive (default .env).

Why

Per the issue, packs shipped as artifacts shouldn't break their consumers when the recommended TYPE/VERSION for the pack changes, the pack itself should be the source of truth. The LOAD_ENV_FROM_GENERIC_PACK flag is for the original ask (env file lives inside the generic pack); the FROM_FILE/FROM_ARCHIVE primitives cover the orthogonal case of an env source that isn't a generic pack (separate config bundle, packwiz zip used with a different TYPE, etc.).

Implementation notes

  • New helpers loadEnvFromFile / loadEnvFromArchive / loadEnvFromGenericPack in scripts/start-utils, reusing the existing applyResultsFile (set -a; source; set +a) pattern called out in the issue.
  • Wired into scripts/start-configuration right after cd /data and before DECLARED_TYPE / DECLARED_VERSION are captured, so the loaded TYPE/VERSION flow through the existing dispatch.
  • Load order: GENERIC_PACKFROM_FILEFROM_ARCHIVE; later wins, and all override docker run -e / compose environment:.
  • LOAD_ENV_FROM_GENERIC_PACK downloads each pack URL into /data/packs/ using the same get --skip-up-to-date call that start-setupModpack makes later, so the archive is not fetched twice.
  • extract() now accepts optional trailing paths-within-archive arguments and forwards them to unzip/tar (per review feedback), so probing a 100 MB pack for a 200 B .env doesn't expand the whole thing.
  • Download/scratch dirs live under /data/.tmp rather than /tmp (per review feedback), matching the existing convention in start-setupModpack.
  • EULA is intentionally not settable this way — the EULA check runs earlier in start-configuration, before any env is loaded. Documented as such.
  • A !!! warning admonition flags that the env file is sourced by bash and should only point at trusted sources.

Tests

Three new setup-only tests under tests/setuponlytests/:

  • load-env-from-generic-pack/ — sets MOTD: from-compose in compose and MOTD=from-generic-pack inside the pack's .env; asserts server.properties has the pack's value AND that the pack's config/dummy.yml made it into /data (so the regular unpack step still ran on the same archive).
  • load-env-from-file/ — sets MOTD: from-compose in compose and MOTD=from-env-file in the loaded file; asserts the env-file value won.
  • load-env-from-archive/ — same idea using a zip with .env at its root.

Out of scope

The packwiz pack.toml → TYPE/VERSION inference path is being tracked separately in #1818, per discussion on the issue.

Test plan

  • tests/setuponlytests/load-env-from-generic-pack PASSED verify
  • tests/setuponlytests/load-env-from-file PASSED verify
  • tests/setuponlytests/load-env-from-archive PASSED verify
  • Existing setuponly and fulltests still PASS
  • bash -n scripts/start-configuration scripts/start-utils clean

Adds LOAD_ENV_FROM_FILE and LOAD_ENV_FROM_ARCHIVE so a generic pack or
artifact can ship its own container configuration (TYPE, VERSION, etc.)
inside-out. Loaded values override anything passed via docker -e, and
the load runs in start-configuration after proxy/Java setup but before
TYPE is dispatched, so the loaded TYPE drives the deploy step.

LOAD_ENV_FROM_ARCHIVE_ENTRY selects the file inside the archive
(default: .env). EULA is intentionally NOT loadable this way since the
EULA check runs earlier.

Closes itzg#4037
Copy link
Copy Markdown
Owner

@itzg itzg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking care of this. I obviously got myself distracted before starting on this. It ended up looking very clean.

Just a couple of things to consider, but am good with merging either way.

Comment thread scripts/start-utils Outdated
Comment thread scripts/start-utils Outdated
ChipWolf added 3 commits May 10, 2026 22:09
extract() now accepts optional trailing paths-within-archive arguments
and forwards them to unzip/tar, so callers needing a single file from a
multi-MB pack no longer pay to expand the whole thing. Existing callers
that pass no extra args extract everything as before.

loadEnvFromArchive uses this to extract just the entry it sources.

Addresses PR review feedback on itzg#4053.
Some users hit issues with the container's /tmp (size limits,
tmpfs unavailable, etc.). /data is already required to be writable
and other parts of the codebase use /data/.tmp/... for transient
unpack workspaces, so use the same location for the env-loading
download and extraction scratch dirs.

Addresses PR review feedback on itzg#4053.
Adds LOAD_ENV_FROM_GENERIC_PACK=true so a packed artifact's own .env can
drive TYPE/VERSION/etc. without the user having to repeat the pack URL
in LOAD_ENV_FROM_ARCHIVE. Each entry in GENERIC_PACKS (after PREFIX/
SUFFIX expansion) is probed for a top-level .env and sourced in pack
order — later packs override earlier ones, matching the cp -R layering
of the regular unpack step. Packs without a .env are skipped silently.

URLs are downloaded into /data/packs/ using the same get invocation that
start-setupModpack later runs with --skip-up-to-date, so the archive is
not fetched twice.

Load order across the three primitives:
  1. LOAD_ENV_FROM_GENERIC_PACK (per-pack .env, in pack order)
  2. LOAD_ENV_FROM_FILE
  3. LOAD_ENV_FROM_ARCHIVE
later loads override earlier, and all override docker -e.

Closes itzg#4037
@ChipWolf
Copy link
Copy Markdown
Contributor Author

ChipWolf commented May 10, 2026

Added LOAD_ENV_FROM_GENERIC_PACK=true in 565e2fb for where the .env lives inside the generic pack and the user doesn't have to repeat the URL.

Behaviour:

  • Iterates each GENERIC_PACKS entry (after PREFIX/SUFFIX expansion) in order and probes for a top-level .env.
  • Each found .env is sourced. Later packs override earlier, matching the cp -R layering of the regular unpack step, so the same precedence the user already gets on file content carries through to the env they declare.
  • Packs without a .env are silently skipped, so it composes with packs that just ship content.
  • URLs are downloaded into /data/packs/ using the same get --skip-up-to-date call start-setupModpack makes later, so nothing is fetched twice. The extract entry arg from the previous review comment is what makes this cheap — only the .env is pulled out for the probe.

The LOAD_ENV_FROM_FILE / LOAD_ENV_FROM_ARCHIVE primitives stay in for the orthogonal case (env source that isn't a generic pack — a separate config bundle, a packwiz zip used with a different TYPE, etc.). Load order is GENERIC_PACKFROM_FILEFROM_ARCHIVE.

New test under tests/setuponlytests/load-env-from-generic-pack/ covers it end-to-end (both that the pack's .env wins over compose MOTD AND that the regular unpack still applies config/dummy.yml from the same archive).

Copy link
Copy Markdown
Owner

@itzg itzg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic! Thanks for implementing this. It sounds like it'll help you out quite a bit.

@itzg itzg merged commit ffbd905 into itzg:master May 12, 2026
5 checks passed
@itzg itzg changed the title feat: load env vars from file or archive at startup Load env vars from file or archive at startup May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inside-out configuration from generic packs/packwiz

2 participants