Skip to content

v1.23.0

Choose a tag to compare

@mortenoh mortenoh released this 14 Sep 19:16
· 155 commits to main since this release
v1.23.0
b859e71

The IG publisher's heap is no longer a number written down in a file. It is derived from the machine the build runs on, and when the kernel kills a build, the build says so.

The heap is sized to the machine

JAVA_HEAP was a fixed 8g in every scaffolded project, and how wrong that is depends entirely on the instance behind it. On a demonstration instance of 1,332 organisation units the publisher peaks at 2.6 GB for the registry and 6.6 GB for the guide - comfortably inside 8g, and the figure most projects will see. On a national instance publishing all five levels - 12,581 organisation units, 25,162 instances - the same two builds peak at 14-16 GB and 9.3-9.7 GB. Both sit above the ceiling they were given, and what you get for it is make: *** [build] Error 137 twenty minutes in, an empty ig/output, and nothing on screen saying which of two opposite knobs to turn.

The national end is what the ceiling is designed for, because clearing it clears everything below it. The problem with writing a number down is that no single number clears both.

A literal is wrong somewhere either way - too small for a national registry on a workstation, too large to survive on a laptop. The docker VM knows, so it is asked:

docker VM ceiling
8 GB 6g
16 GB 14g
32 GB 30g
64 GB 62g

The VM's memory less 2 GB, floored at 4g, falling back to 8g when docker cannot be asked. The 2 GB is the heap + ~2G rule the documentation already stated, so the two now agree. Every build states what it derived:

publisher heap 22g, docker VM 24.0 GB

It stays a ?= default. Set it on the command line or in the environment whenever you want to say, and the derivation is skipped entirely:

make build JAVA_HEAP=8g

A publisher build wants the machine to itself - the publisher and Jekyll share one container, and Jekyll renders the site at the very end with the JVM still resident. Stop your other containers for the run.

A kill explains itself

The ceiling is generous on purpose, so the failure had to become unmistakable:

================================================================
  BUILD KILLED - out of memory (exit 137)
================================================================
  The kernel SIGKILLed the publisher. This is not a build error:
  ig/output is empty or partial because the site is written last.

    heap ceiling   -Xmx14g
    docker VM      16.0 GB

  Other containers were holding this VM during the build:
      dhis2-db  postgres:16

  Ways out, best first:

    1. Stop any other containers and run it again - a publisher
       build wants the machine to itself.
    2. Raise the VM:  Docker Desktop > Settings > Resources > Memory
    3. Lower the ceiling:  make build JAVA_HEAP=<smaller>

  The opposite failure looks different: OutOfMemoryError with a Java
  stack trace means the ceiling is too SMALL. Raise it instead.
================================================================

Naming both directions is the point. A ceiling too large for the box is a SIGKILL; a ceiling too small for the guide is an OutOfMemoryError with a Java stack trace. They look alike from a distance and need opposite fixes. The list of other containers is omitted when there are none, so the message never advises a fix that does not apply.

make build also prints the peak the container actually reached, read from its own cgroup before it exits. On a kill that is the evidence; on a successful build it is the only thing that answers "is the ceiling big enough" with a measurement rather than a guess.

One heap knob for a split guide

A guide and its registry package were scaffolded with REGISTRY_HEAP and GUIDE_HEAP, on the reasoning that the two builds are not alike. They are not, but -Xmx is a ceiling rather than a reservation: identical work under -Xmx2g and -Xmx12g peaks at 0.85 GB and 1.02 GB resident. A guide under a large ceiling simply never grows into it.

So there is one JAVA_HEAP on the root Makefile, reaching both projects. It is forwarded only when you set it, leaving each project its own derived default otherwise.

Upgrading

d2w fhir init --refresh in an existing project takes the new Makefile. Nothing else changes: the heap is a Makefile knob, no configuration key, no CLI flag, and an override you already set on the command line or in the environment keeps winning.