Repository navigation
Releases
Release v0.3.1
Released 2026-11-16. A patch release. One image, docker.io/oramasearch/agent-creator:v0.3.1, built on agent-runtime v0.3.1.
Since v0.3.0: one clean header in the Dockerfile. The definition itself (prompt, the rule checks in defects, the turn answer, and how Lake reaches it) is unchanged, and so is the runtime underneath: agent-runtime v0.3.1 changes only its own Dockerfile comments.
This release is the tree of dev after #32 (dc435a4), with these notes and the version.
What it includes
Every pull request merged into dev since the tag v0.3.0 (git log --first-parent v0.3.0..origin/dev):
| PR | What it does |
|---|---|
| #31 | Records the v0.3.0 image digest in the v0.3.0 notes; documentation only |
| #32 | Gives the Dockerfile a clean header: the comment no longer references an internal issue and says the image is published on Docker Hub |
On agent-runtime v0.3.1, underneath
The image is FROM oramasearch/agent-runtime:${RUNTIME_TAG} built with RUNTIME_TAG=v0.3.1 (notes). That runtime release changes only Dockerfile comments; its code is v0.3.0's.
The vendored copy of agent-runtime's skills (tests/fixtures/agent-runtime-skills.json) is the same as agent-runtime v0.3.1's skills.json.
Settings it needs
No setting is new or removed since v0.3.0.
Not included
- Bounded workspace facts, sent to the model as untrusted data (#24): deferred for review.
- The agent-runtime base named by tag and digest (#25): deferred for review. The Dockerfile still names the runtime by
RUNTIME_TAGalone.
Known issues
Security-relevant first. Unchanged from v0.3.0.
- Build logs and App events go unbounded into the system prompt and can steer the proposal (#22). The fix is #24, not in this release.
- The runtime writes every turn's trace to the shared pod's disk (agent-runtime#38): one AgentCreator pod serves every tenant, so every tenant's turn content stays under
TRACE_DIR. - The image bundles GPL and LGPL code (#28): it is built on agent-runtime's
python:3.12-slimbase (Debian: glibc, bash, coreutils, dpkg, readline). The move toscratchwith static binaries is deferred; it is noted here and does not block this release. - The runtime base is named by tag only (#25, deferred): a later build of the same tree could get a different runtime if the tag moved.
- Workspace identity is not passed explicitly on every call (#9, with agent-runtime#6): App and build reads carry only
customer_id. - The structured answer is shipped, the issue is still open (#1), for Lake's side to read
result.
Where it has run
The v0.3.1 tree had not run on a development environment when it was released. Lakes take the operator's default AgentCreator image; moving it to :v0.3.1 is a separate operator change.
The image
Built from the tag v0.3.1 (58fcd01) with BuildKit (linux/amd64, the repository's Dockerfile, on oramasearch/agent-runtime:v0.3.1@sha256:166a83183de2e7eaa368c67f8eb1def327641815a6690f9c824c4e850882f654 with RUNTIME_TAG=v0.3.1), then tagged :v0.3.1 on that digest without rebuilding:
docker.io/oramasearch/agent-creator:58fcd01-1@sha256:25fefa983f755036b842b5e8d4bf2f945e0ce2d9ceaf5dff994c57f3d28ebda5, also tagged :v0.3.1 (same digest). The image is published on Docker Hub; its GPL/LGPL base is a known issue (#28).
Pinning it on a Lake is a separate change to the operator's defaults or the Lake's resource.
Verification
| Proof | Result |
|---|---|
Test suite (uv run --frozen -p 3.12 --group dev pytest -q -ra, macOS arm64, Python 3.12) |
29 passed, 0 failed, 0 skipped: tests/test_creator.py 25, tests/test_grants.py 3, tests/test_release.py 1 (reads the version from pyproject.toml and requires these notes and their index line) |
| Skills fixture | tests/fixtures/agent-runtime-skills.json is byte for byte agent-runtime v0.3.1's skills.json |
SPDX headers (python3 tools/spdx-headers.py --check) |
Passed |
SBOM (python3 tools/sbom.py --check) |
Passed, regenerated for v0.3.1 |
| Offline evals | None: every eval case runs against a live workspace |
Live evals (evals/run.py) |
Not run (needs a live model and a live workspace) |
| Dev environment | Not run (above) |
| Linux image build (BuildKit, from the tag) | Built from v0.3.1 on agent-runtime v0.3.1: oramasearch/agent-creator:58fcd01-1, also :v0.3.1, sha256:25fefa983f755036b842b5e8d4bf2f945e0ce2d9ceaf5dff994c57f3d28ebda5 |
Release v0.3.0
Released 2026-11-16. A minor release. One image, docker.io/oramasearch/agent-creator:v0.3.0, built on agent-runtime v0.3.0.
Since v0.2.0: the MIT licence, a security policy and the tree prepared for a public export. Underneath, the agent-runtime it runs on bounds its bus consumer and conversation memory and pins its base images by digest. The definition itself (prompt, the rule checks in defects, the turn answer, and how Lake reaches it) is unchanged from v0.2.0.
This release is the tree of dev after #27 (597250b), with these notes and the version.
What it includes
Every pull request merged into dev since the tag v0.2.0 (git log --first-parent v0.2.0..origin/dev):
| PR | What it does |
|---|---|
| #19 | Merges main back into dev after the v0.2.0 release; nothing new |
| #20 | Records the v0.2.0 image digest in the v0.2.0 notes; documentation only |
| #23 | Adds a security policy, SECURITY.md: how to report a vulnerability privately, and what we commit to |
| #26 | Prepares the tree for a public export: internal links, names and wording removed from the README, the docs and the release notes |
| #27 | The MIT licence: LICENSE, COPYRIGHT, an SPDX header in every source file, THIRD_PARTY_LICENSES, CONTRIBUTING.md, CLA.md, CODE_OF_CONDUCT.md, and the SBOM (sbom.spdx.json, sbom.cdx.json, tools/sbom.py) |
The licence — #27
agent-creator is released under the MIT licence, copyright OramaSearch, Inc. (LICENSE, COPYRIGHT). Every source file carries SPDX-License-Identifier: MIT, added and checked by tools/spdx-headers.py. THIRD_PARTY_LICENSES lists the licence of every dependency. Contributions are made under CLA.md (contributors assign copyright to OramaSearch, Inc.), following CONTRIBUTING.md and CODE_OF_CONDUCT.md. tools/sbom.py writes the software bill of materials in SPDX 2.3 and CycloneDX, regenerated for this release.
From agent-runtime v0.3.0, underneath
The image is FROM oramasearch/agent-runtime:${RUNTIME_TAG} built with RUNTIME_TAG=v0.3.0, so it carries every change in agent-runtime since its v0.2.0 (notes):
- the bus consumer claims a task only when a worker is free, at most
AGENT_BUS_WORKERS(default 4) at once andAGENT_BUS_TENANT_WORKERS(default one fewer) for one customer; - conversation memory is bounded (
AGENT_CONVERSATIONS,AGENT_CONVERSATION_TURNS,AGENT_CONVERSATION_BYTES,AGENT_CONVERSATIONS_BYTES,AGENT_CONVERSATION_IDLE_SECONDS), and a bus turn continues only the conversation itsprior_turnleft in the same account and workspace; - the runtime's base images are pinned by digest;
- the runtime's MIT licence and security policy.
The vendored copy of agent-runtime's skills (tests/fixtures/agent-runtime-skills.json) is the same as agent-runtime v0.3.0's skills.json.
Settings it needs
No setting is required. The seven optional settings agent-runtime v0.3.0 adds (above) have defaults. The definition reads no new setting.
Not included
- Bounded workspace facts, sent to the model as untrusted data (#24): deferred for review.
- The agent-runtime base named by tag and digest (#25): deferred for review. The Dockerfile still names the runtime by
RUNTIME_TAGalone.
Known issues
Security-relevant first.
- Build logs and App events go unbounded into the system prompt and can steer the proposal (#22). The fix is #24, not in this release.
- The runtime writes every turn's trace to the shared pod's disk (agent-runtime#38): one AgentCreator pod serves every tenant, so every tenant's turn content stays under
TRACE_DIR. - The image bundles GPL and LGPL code (#28): it is built on agent-runtime's
python:3.12-slimbase (Debian: glibc, bash, coreutils, dpkg, readline), so the image is not published. It moves with agent-runtime toscratchwith static binaries before it is. - The runtime base is named by tag only (#25, deferred): a later build of the same tree could get a different runtime if the tag moved.
- Workspace identity is not passed explicitly on every call (#9, with agent-runtime#6): App and build reads carry only
customer_id. - The structured answer is shipped, the issue is still open (#1), for Lake's side to read
result.
Where it has run
The v0.3.0 tree had not run on a development environment when it was released. Lakes take the operator's default AgentCreator image; moving it to :v0.3.0 is a separate operator change.
The image
Built from the tag v0.3.0 (62d8260) with BuildKit (linux/amd64, the repository's Dockerfile, on oramasearch/agent-runtime:v0.3.0@sha256:6917619aeb7cc03a93df29c0bbff38ef6e93d5314c3cd162a0123e7bb1390b52 with RUNTIME_TAG=v0.3.0), then tagged :v0.3.0 on that digest without rebuilding:
docker.io/oramasearch/agent-creator:62d8260-0@sha256:d0e4bd43ff7d31cdd2f0fe6b920cf7dc870364f9b0bcdc87b94d1151ab1515a0, also tagged :v0.3.0 (same digest). The image is not published (#28).
Pinning it on a Lake is a separate change to the operator's defaults or the Lake's resource.
Verification
| Proof | Result |
|---|---|
Test suite (uv run --frozen -p 3.12 --group dev pytest -q -ra, macOS arm64, Python 3.12) |
29 passed, 0 failed, 0 skipped: tests/test_creator.py 25, tests/test_grants.py 3, tests/test_release.py 1 (reads the version from pyproject.toml and requires these notes and their index line) |
| Skills fixture | tests/fixtures/agent-runtime-skills.json is byte for byte agent-runtime v0.3.0's skills.json |
SPDX headers (python3 tools/spdx-headers.py --check) |
Passed |
SBOM (python3 tools/sbom.py --check) |
Passed, regenerated for v0.3.0 |
| Offline evals | None: every eval case runs against a live workspace |
Live evals (evals/run.py) |
Not run (needs a live model and a live workspace) |
| Dev environment | Not run (above) |
| Linux image build (BuildKit, from the tag) | Built from v0.3.0 on agent-runtime v0.3.0: oramasearch/agent-creator:62d8260-0, also :v0.3.0, sha256:d0e4bd43ff7d31cdd2f0fe6b920cf7dc870364f9b0bcdc87b94d1151ab1515a0 |
Release v0.2.0
agent-creator v0.2.0
Released 2026-10-25. A minor release (Steve named the version on 2026-10-25). One private image,
docker.io/oramasearch/agent-creator:v0.2.0, built on
agent-runtime v0.2.0. Release
tracking: agent-creator#16.
Since v0.1.0, the Agent Creator Agent can read the methodology registry again, and its
tools are limited to exactly the reads a turn makes. Underneath, the agent-runtime it runs on carries
the 2026-10-21 security-review fixes. Everything v0.1.0 lists is otherwise unchanged: prompt v5, the
rule checks in defects, the turn answer, and how Lake reaches it.
What it includes
Every pull request merged into dev since the tag v0.1.0 (git log --first-parent v0.1.0..origin/dev):
| PR | What it does |
|---|---|
| #15 | The methodology registry is readable (agent-creator#11). definition/record.json grants search, summarize and pipelines; in agent-runtime only train and pipelines grant list_methodologies, and train would also give direct check_pipeline and start_run. definition/agent.toml now has a tools allow-list of exactly the twelve reads a turn makes (list_solutions, get_solution, list_apps, get_app, receive_app_events, list_builds, get_build, receive_build_events, list_methodologies, list_datasets, list_projects, get_project), so none of pipelines' proposals, and no other tool, is one of its tools. New tests/test_grants.py runs the turn's reads against a workspace that answers only what agent-runtime would make a tool for this record (from a vendored copy of agent-runtime's skills.json) |
| #14 | Records the v0.1.0 image digest in the v0.1.0 notes |
| #13 | The v0.1.0 release pull request (dev promoted to main), carried back into dev |
What this changes for a person: methodology-bound proposals and declines pass their checks again,
and approved Pipelines that name a runnable methodology are offered for grants instead of being
listed as not runnable.
From agent-runtime v0.2.0, underneath
The image is FROM oramasearch/agent-runtime:${RUNTIME_TAG} built with RUNTIME_TAG=v0.2.0, so it
carries every change in agent-runtime since its v0.1.0:
- agent-runtime#19:
POST /v1/turncompares
the service token in constant time, and refuses every turn with503when
AGENT_SERVICE_TOKENis unset or empty (it used to be open). A wrong or missing bearer is401. - agent-runtime#21: a turn's credential
goes only to Lake at this service'sLAKE_URL. The turn's ownlake_urlis no longer read; with
LAKE_URLunset or not an absolute http(s) address without user info, the turn is refused before
any request goes out. - agent-runtime#26: dataset-confined agents
are limited to qualified dataset reads (LSR-181). AgentCreator's work area isworkspace, so its
own turns are unaffected; an agent it proposes with work areadatasetgets these limits (see
Known issues).
Settings it needs
No new setting of its own. agent-runtime v0.2.0 now requires AGENT_SERVICE_TOKEN and
LAKE_URL (and reads AGENT_MODEL when the record names no model; the record names gpt-5.5). The
Orama operator already renders AGENT_SERVICE_TOKEN, LAKE_URL and AGENT_MODEL for every library
agent, so no operator change has to roll first. A hand-run service without AGENT_SERVICE_TOKEN now
refuses every turn.
What it does not include
Unchanged from v0.1.0:
- It creates nothing. Creating the approved agent is lake's (lake#233).
- It does not change existing agents (lake#271).
- It proposes only what agent-runtime's twelve skills can do.
- It checks that Python compiles, not that it does the right thing.
- No live eval of the current prompt. The only recorded eval is prompt v1 (15/16).
- Conversation memory is per pod and in memory; a restart loses it.
Known issues
Security-relevant first.
- Workspace identity is not passed explicitly on every call
(agent-creator#9, with
agent-runtime#6): App and build reads
carry onlycustomer_id, and the idle bus poll runs every 10 seconds. Related, in lake:
lake#918 (LSR-009, agent launch and schedule
credentials escaping their workspace and inner action scopes). - The runtime underneath keeps unbounded conversation histories and claims bus tasks before a
worker is free (agent-runtime#25,
LSR-076; agent-runtime#24, LSR-075). - Lake's shared agent service token is accepted by every library agent, and a user's service
launch holds it (agent-runtime#14, on
hold). - The runtime still writes a trace per turn under
TRACE_DIRin the pod. - Dataset-confined agents lose some reads on agent-runtime v0.2.0
(agent-runtime#26): index search, App and
build reads, Pipeline and MCP tools are refused or omitted indatasetwork areas. An agent
AgentCreator proposes with work areadatasetand a skill that needs those reads will not get them. - The structured answer is shipped, the issue is still open
(agent-creator#1), for lake's side to read
result. - The skills fixture is vendored (
tests/fixtures/agent-runtime-skills.json, from agent-runtime
cf964f4). When agent-runtime's skills change, refresh it asdocs/methodologies/README.mdsays.
Where it has run
The v0.2.0 tree has not run on the dev environment. No dev-<sha> build of agent-creator ran on
the dev member of lake-nonprod; every Lake still takes the operator's default
oramasearch/agent-creator:v0.1.0 (orama-operator internal/controller/lake_defaults.go on dev).
Moving it to :v0.2.0 is a separate step, through testing (the e2e gate), staging and prod.
The image
Built from the tag v0.2.0 (db8a750) on the cluster BuildKit with build-agent.sh (linux/amd64, the repository's Dockerfile, built on oramasearch/agent-runtime:v0.2.0 with RUNTIME_TAG=v0.2.0), then tagged :v0.2.0 on that digest without rebuilding:
docker.io/oramasearch/agent-creator:db8a750-0@sha256:441eec21b709c06aa5c4d0f6d3d6fc125f54d42609275408be502ed3e544f644, also tagged :v0.2.0 (same digest). The Docker Hub repository answers 404 anonymously (private).
It is not rolled anywhere yet: pinning it on an environment is a separate change to the operator's set files, through dev, testing, staging and prod in order.
Verification
| Proof | Result |
|---|---|
Test suite (uv run --frozen --group dev pytest -q -ra, macOS arm64, Python 3.14 and again on 3.12) |
29 passed, 0 failed, 0 skipped on each: tests/test_creator.py (25), tests/test_grants.py (3), tests/test_release.py (1, reads the version from pyproject.toml and requires these notes and their index line) |
| Offline evals | None: every eval case runs against a live workspace |
Live evals (evals/run.py) |
Not run (needs a live model and a live workspace) |
| Linux image build (cluster BuildKit, from the tag) | Built from v0.2.0 with build-agent.sh: oramasearch/agent-creator:db8a750-0, also :v0.2.0, sha256:441eec21b709c06aa5c4d0f6d3d6fc125f54d42609275408be502ed3e544f644; private (404 anonymously) |
Release v0.1.0
agent-creator v0.1.0
Released 2026-10-17. The first release. One private image, docker.io/oramasearch/agent-creator:v0.1.0.
It carries the Agent Creator Agent (AgentCreator), the agent behind lake's Agent Factory
(Design → Agents: Describe it to the Agent Factory). A person describes the agent they want; it
reads their workspace, then asks one question, declines work the workspace engine cannot run, or
proposes the agent in Studio's shape: native or Python, and for Python whether its code is stored in
Zetta or in a lake repository. It creates nothing: lake shows the proposal and creates the agent
only when the person approves it (lake#233). Proprietary to Orama; private repository and image.
It is not a program of its own. It is a definition (definition/) that runs on
agent-runtime; the image is the runtime plus the
definition. This release is the same tree as commit 6b1cc36, the commit Lake already runs, plus
these notes, the release test and the README and docs index lines that point at them.
What it includes
The definition
| File | What it is |
|---|---|
definition/record.json |
Its agent record: id agent-creator, name AgentCreator, kind orama, work area workspace, skills search and summarize, model gpt-5.5 |
definition/agent.toml |
Entrypoint agent.main:run, model gpt-5.5, dependency openai; no tools allow-list, so the runtime's derived tools are not narrowed further |
definition/instructions.md |
The prompt, prompt v5 (the version is the file's first heading; there is no constant in code) |
definition/agent/main.py |
The turn: read the workspace, call the model, check the answer with defects, one repair round, then answer |
What one turn does (agent.main:run)
- Reads the workspace, live, every turn (nothing is cached between turns), through the tools
agent-runtime derives for the person who sent the turn:- solutions:
list_solutions, thenget_solutionfor every returned id. An older engine's
unknown_capabilitymeans the solution registry is not deployed; there is no fallback solution; - Apps and builds:
list_apps,get_app,receive_app_events(from sequence 1, no wait) for
each App, andlist_builds,get_build,receive_build_eventsfor each build, each only when
the runtime granted those reads for this turn. App and build calls carry the turn's
customer_id; - methodologies:
list_methodologies. An older engine'sunknown_capabilityswitches to the
documented compatibility contract: only M-1 (churn / GBDT classification) runs; - datasets:
list_datasets; - Pipelines:
list_projects, thenget_projectfor each project; only graphs with
approval: approvedwhose methodology (if any) is runnable are offered for grants, the others
are listed as not runnable.
- solutions:
- Calls the model (
context.client, which agent-runtime points at lake's model gateway with the
turn's own key) with the prompt, the workspace facts and the conversation so far, asking for a
JSON object. - Checks the answer by rule (
defects). An answer is aquestion, adecline(naming a
methodology or a solution, with its tracking issue) or aproposal. A proposal is rejected when:- its type is not
nativeorpython, or the agent has no name; - it grants a skill outside agent-runtime's twelve:
search,generate-data,train,
summarize,call-api,drives,transcripts,repositories,datasets,streams,
pipelines,usage(agent-runtime#7); - its work area is not
workspace,projectordataset; - a methodology it records is not in this turn's registry or not runnable, or it trains without
recording one; - an App agent is not Python, reads outside the turn's App grant, subscribes to a topic other
than a liveapp-events.<app-id>, a livebuild-events.<build-id>orbuild-events.*,
proposes an App or build change outside the grant, does not delegate changes to
agent-applicationwith turn kindapp, or itsagent.tomltoolsdiffer from its reads plus
proposals; - a solution binding names a solution that is not in this turn's registry, is a sketch or is not
runnable; its capabilities are not exactlydecide,list_ledger,get_monitor,
get_roi_readout; its autonomy differs from the solution's autonomy defaults; or an A1 action
class is not approval-gated; - a Pipeline grant is not an approved, runnable Pipeline of this workspace, or its methodology is
not recorded; - a native agent carries code or a trigger;
- a file path leaves the definition; Python does not compile or imports something outside the
standard-library list and the runtime's packages (openai,httpx,pydantic,pandas,
numpy);agent.tomlasks for other dependencies; something looks like a secret; - a Python agent is not stored in
zettaorlake, or is stored inzettawith more than 900 KiB
of code.
- its type is not
- One repair round: a defective answer goes back to the model once with its defects. A second
defective answer is returned with its defects, never as clean. - Answers
{"reply": <one sentence for a person>, "result": {kind, question | decline | proposal, defects, facts}}(agent-creator#1).
How it is served (agent-runtime's service mode)
The Dockerfile sets AGENT_MODE=service, AGENT_PER_TURN_CREDENTIALS=1 and
AGENT_DEFINITION_DIR=/definition. In that mode agent-runtime:
- serves HTTP on
PORT(8080):GET /healthz:{ok, agent, name, tools, model};POST /v1/turn: a JSON body withmessage(orrequest),conversation_id,lake_token,
customer_id, and optionallyworkspace_id,pageandlake_url. WhenAGENT_SERVICE_TOKEN
is set, the request must carryAuthorization: Bearer <that token>. The answer is
{ok, conversation_id, trace_id, reply, result, calls, rounds, finished, proposals}:200when
it answered,422when the turn failed,400for a body that is not JSON or has no message,
401without the token;
- works per turn: each turn's
lake_tokenandcustomer_id(both required) are the person's
delegated credential; the tools are derived from lake's OpenAPI for that credential, so it reads
only what the person may read, and the model is called through lake's gateway with that key; - keeps each conversation's messages in the pod's memory, keyed by
conversation_id; - writes a trace per turn under
TRACE_DIR.
This repository adds no HTTP route and no version endpoint; the runtime owns HTTP.
How Lake reaches it
-
The Orama operator runs it as a library agent: one Deployment (one replica) and one Service
per Lake, port 8080 namedhttp, startup, readiness and liveness probes on/healthz, user 10001. The operator gives the pod
LAKE_URL(lake's in-cluster address),AGENT_SERVICE_TOKEN,AGENT_BUS_TOKEN(its own bus
credential) andAGENT_BUS_AGENT=agent-creator. -
Lake learns its address from
LAKE_AGENT_ENDPOINTS(agent-creator=<url>, among the other
library agents), and presentsLAKE_AGENT_SERVICE_TOKENas the bearer. -
Turns arrive over the agent bus (lake
docs/api/agent-bus.md, lake#269). Lake publishes each
task id onagent-tasks.agent-creatoron lake's Zetta (task ids only, never content). The
runtime's bus client (agent_runtime.lake_bus, started only whenAGENT_PER_TURN_CREDENTIALS=1
andLAKE_URL,AGENT_BUS_TOKENandAGENT_BUS_AGENTare all set):- long-polls
receive_agent_tasks(10-second wait) from sequence 1 at start; - calls
claim_agent_taskfor each task; aconflictornot_foundmeans another replica won;
the claim answers the turn body with the task'slat.credential in it; - runs up to four turns at once; for each it publishes one
progressevent (Working on it),
runs the same turn handler as/v1/turn, then publishesresult(the whole answer) or
failed.
It sends no further progress events during a turn; Lake renews the lease on each capability call
made with the task credential. - long-polls
CLI entry points
None of its own (package = false, no [project.scripts]). The image's entrypoint is
agent-runtime's agent-runtime. evals/run.py is the eval runner (below).
Environment variables
Read by the definition: none. Set by the Dockerfile: AGENT_DEFINITION_DIR, AGENT_MODE,
AGENT_PER_TURN_CREDENTIALS, TRACE_DIR. Read by agent-runtime in this mode:
| Variable | Meaning |
|---|---|
LAKE_URL |
lake's address, for turns, the model gateway and the bus |
AGENT_SERVICE_TOKEN |
the bearer /v1/turn requires, when set |
AGENT_BUS_TOKEN, AGENT_BUS_AGENT |
its bus credential and the agent key it takes tasks under |
AGENT_MODEL |
the model when the record names none (the record names gpt-5.5) |
PORT |
the HTTP port (8080) |
TRACE_DIR |
where turn traces are written (the operator sets /tmp/traces) |
AGENT_DEFINITION_DIR, AGENT_MODE, AGENT_PER_TURN_CREDENTIALS |
the built-in definition and the mode |
Read by evals/run.py: LAKE_URL, LAKE_TOKEN (a person's token), CUSTOMER_ID, AGENT_MODEL, and
the OpenAI client's own OPENAI_API_KEY.
Evals and tests
evals/run.py: asks generated from the skills × work areas × with or without code, plus live
App/build, methodology and solution cases, run against a real workspace and graded by rule
(defectsplus case facts; no judge). Reports land inreports/; the one recorded run is
reports/2026-09-27-evals.md, 15/16 with prompt v1. No report exists for prompt v3 or v5.tests/test_creator.py: 25 tests with a fake workspace and scripted model replies.tests/test_release.py: the release test (below).
Contracts: [Apps and builds](https://github.com/oramasearch/agent-creator/blob/v0.1.0...