Skip to content

Release v0.3.0

Latest

Choose a tag to compare

@homer6 homer6 released this 16 Nov 07:29
28eded2

Released 2026-11-16. A minor release. Since v0.2.0: model-written profiles are checked before anything is written (safe names, bounded size), the licence, the security policy and base images pinned by digest. Image: docker.io/oramasearch/agent-eval-generator:v0.3.0, not published (see Known issues).

This release is the tree of dev with these notes and the version.

What it includes

Every pull request merged into dev since the v0.2.0 tag:

PR What it does
#27 MIT LICENSE and COPYRIGHT (OramaSearch, Inc.), an SPDX-License-Identifier: MIT line in every source file (tools/spdx-headers.py), THIRD_PARTY_LICENSES, CONTRIBUTING.md, CLA.md, CODE_OF_CONDUCT.md, and the SBOM (sbom.spdx.json, sbom.cdx.json, tools/sbom.py)
#24 Security: a model-written profile is checked before anything is written or generated (safe names, bounded size)
#25 The Dockerfile names its base images by digest as well as by tag (python:3.12-slim, ghcr.io/astral-sh/uv:0.11.7), so a later build of the same source gets the same base
#23 SECURITY.md: how to report a vulnerability
#19 Records the v0.2.0 image digest in the v0.2.0 notes (documentation only)
#18 The v0.2.0 release promotion (dev → main), merged back into dev

A model-written profile is checked before anything is written — #24

  • The model's profile_id and table names were used as file paths, so a turn could write and delete outside its temporary folder in a pod every workspace shares (#21). synthesize() had no limit on tables or columns, so one turn could use up the pod's memory (#22).
  • New limits.check_profile() runs right after parsing, in service.run_turn, in the batch command line and again in synthesize(). Names that become paths must match ^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$; a profile has at most 12 tables, 64 columns per table, 128-character column names, no table declared twice, and at most 5,000,000 generated cells.
  • A refused profile is answered invalid profile: <reason> and its trace graded failed.
  • records.package() and Table.write_jsonl() resolve every path inside their directory and refuse one outside it; an old records/ is removed only when it is a real directory inside the output folder, never through a symbolic link.

The licence — #27

Everything in this repository that Orama wrote is under the MIT licence. Contributors sign CLA.md once. THIRD_PARTY_LICENSES lists every runtime dependency with its licence text; none is incompatible with MIT distribution.

Before it rolls: the settings it needs

Nothing new. The variables are unchanged from v0.2.0.

What it does not include

Deferred, not in this release:

  • #26: claiming bus tasks by reference and naming the task on every call; it needs the matching Lake change.
  • #28: the public-tree cleanup, which also makes the Hugging Face upload target a setting (EVAL_PROFILES_DATASET) and so changes the batch command line's behaviour.

Otherwise unchanged from v0.2.0.

Known issues

Security-relevant first. All open at this tag.

  1. #29 (release blocker for publishing): the image is built on python:3.12-slim, which bundles GPL and LGPL userland, so the image is not published. It is built and kept in the project's non-public Docker Hub repository until it moves to a scratch base with static binaries.
  2. The public-tree cleanup (#28) is not in this release: the batch command line still uploads to the Hugging Face dataset named in the code, and some files still carry internal references. This tree is not the one to publish.
  3. #6: the task's workspace is not passed explicitly on data-plane calls. Lake scopes the task credential to the task's workspace.

Where it has run

The v0.3.0 tree had not run on a development environment when it was released. Lakes run the operator's default image of this agent; the changes since v0.2.0 are proven by the test suite only. Moving a Lake to :v0.3.0 is a separate operator change.

The image

Built from the tag v0.3.0 (28eded2) with BuildKit (linux/amd64, the repository's Dockerfile), then tagged :v0.3.0 on that digest without rebuilding:

docker.io/oramasearch/agent-eval-generator:28eded2@sha256:74c5b48f80a6eec2274d3f092e1bd5ac64523dc79b05af111ab1992095c3f440, also tagged :v0.3.0 (same digest).

It is pushed only to the project's non-public Docker Hub repository, which answers 404 anonymously. Pinning it on a Lake is a separate change.

Verification

Proof Result
Test suite (uv run --frozen pytest -q -ra, macOS arm64, Python 3.12, release branch) 66 passed, 0 failed, 0 skipped; 66 tests. Includes tests/test_release.py (the declared version has these notes, indexed)
SPDX headers (python3 tools/spdx-headers.py --check) Every source file carries the header
SBOM (python3 tools/sbom.py --version v0.3.0, then --check) Regenerated on the release commit; --check passes
Batch generation, and a live turn through Lake's model gateway Not run (needs a live model and a Lake)
Linux image build (BuildKit, linux/amd64, from the tag) Built from v0.3.0: oramasearch/agent-eval-generator:28eded2, also :v0.3.0, sha256:74c5b48f80a6eec2274d3f092e1bd5ac64523dc79b05af111ab1992095c3f440