Skip to content

Releases: NatLabRockies/sparkctl

v0.5.0

Choose a tag to compare

@daniel-thom daniel-thom released this 14 Jun 00:18
Bump version to 0.5.0

0.4.1

Choose a tag to compare

@daniel-thom daniel-thom released this 30 May 21:45
Add HPC environment module and SPARKCTL_SETTINGS_FILE support

Add a ready-to-deploy environment module (TCL and Lua flavors) under
hpc/environment_module/ so HPC users can run:

    module load sparkctl
    sparkctl configure --start

The module activates a shared pre-installed virtualenv, sets
SPARKCTL_SETTINGS_FILE, and puts spark-submit/pyspark on PATH. It reads
the Spark/Java/Hadoop binary paths from the shared sparkctl.toml so that
settings file is the single source of truth (TCL parses it natively; Lua
parses it via awk since Lmod's sandbox lacks reliable file I/O).

Wire up the SPARKCTL_SETTINGS_FILE environment variable in config.py
(it was already documented in the deploy guide but never implemented).
Settings files now load in increasing precedence: ~/.sparkctl.toml, then
$SPARKCTL_SETTINGS_FILE, then ./.sparkctl.toml.

Document the module and the env var in the deploy guide, and bump the
version to 0.4.1 (first release supporting SPARKCTL_SETTINGS_FILE).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

v0.4.0

Choose a tag to compare

@daniel-thom daniel-thom released this 30 May 20:31
Bump version to v0.4.0

v0.3.2

Choose a tag to compare

@daniel-thom daniel-thom released this 29 May 20:48
Bump version to v0.3.2

v0.3.1

Choose a tag to compare

@daniel-thom daniel-thom released this 21 Nov 19:55

Set the version of postgres to v17.

v0.3.0

Choose a tag to compare

@daniel-thom daniel-thom released this 15 Jul 14:42
34e2cb8

Initial release