AKMON v0.3.0

AKMON 0.3.0 follows 0.2.0 with fixes for the first things a new user hits
(unreadable files that were accepted without a word, PDFs that held only a file
name, a long install printout, a quickstart that built a large image), one
security fix, and the fixes merged since 0.2.0. If you run 0.2.0, see
"Upgrading from 0.2.0". AKMON's Python package and command are named okg, so
every command below starts with okg.

Get it

• Python package: the wheel okg-0.3.0-py3-none-any.whl is attached to
  this release. Install it with uv tool install or pip install, then
  follow the quickstart (docs/start/quickstart.md).
• Containers and Kubernetes: the Install section at the end lists the
  runtime and PostgreSQL images and the Helm chart, each with its digest. The
  chart names both images, so an install needs no image settings
  (docs/operate/deployments/kubernetes.md).

What's new

First-run fixes

• okg deployment add-source warns when a folder holds files AKMON cannot
  read. The preview and the result name each unreadable file type with its
  count, list the formats read today, and say what to do. When none of the
  matched files can be read, it refuses instead of publishing records that hold
  only file names.
• PDF text is searchable. A local folder source reads the text of each PDF
  page by page, so a search for a word inside a paper finds it, with the page
  number. It reads the text layer only (not scanned pages, figures or tables),
  up to 50 MiB and 5,000 pages per file. The preview warns about a PDF with no
  text, such as a scan, and about a PDF that will not open; a file over the size
  limit is recorded with no text and a warning. With the default settings,
  reading PDFs makes no model call. See "Breaking changes" for which graphs
  start reading PDFs.
• The README is now short. It says what AKMON is (a consistent, versioned
  knowledge base of an organization's data, read by people and agents), the
  problem it solves and the pipeline in one line, then points to the
  quickstart and the documentation.
• GitHub sources can import discussion. include_discussion=true now
  publishes issue and pull request comments; before, every import with it was
  refused. The new optional setting commit_repo names the git repository in
  the same graph and links merged pull requests to their merge commits. It
  accepts a git source that comes from a code_repos block, and refuses one
  that is switched off or missing, with the fix. Without commit_repo, merged
  pull requests import unlinked and the import step says so.
• The quickstart installs a tagged release and starts the published database
  image instead of building one, on Docker or podman. One variable,
  AKMON_VERSION, picks the release. OKG_PG_IMAGE names the database image
  for the bundled Compose file, so two checkouts no longer retag each other's
  image.
• The quickstart and the first-run troubleshooting page now cover the gaps a
  first-time reader hit on a cold run: a database check that could wait
  forever, the podman error about the limit on kernel keys (the fix is
  keyring = false under [containers] in containers.conf), the line
  search profile: none (unscoped) in search output, and what to do when none
  of the four supported agents is installed.
• okg install ends with a ten-line summary: the graph's name, how many
  records and links it holds, where the database is, one search to try, and how
  to connect an agent (with the extra step for a database on another host). --verbose brings back the full report; --json has the same
  keys as before. In an activated virtual environment, hints say okg, not
  uv run okg.
• okg agent connect --agent claude-code works on a machine where the claude
  command is not installed: it writes Claude Code's configuration and tells you
  to install Claude Code before you start a session.
• okg jobs status, okg jobs enqueue-source, okg jobs find-source-run and
  okg chat source-retry print a one-line error that names --deployment when
  you give no deployment or database address, not a Python traceback.
• When a source fails its health check, okg ingest and okg install now name
  the source and the connector's own reason, for example a missing credential.
  The message says whether the other sources were published without it or not
  published, and gives the command that leaves the failed source out for one
  run. That note about the other sources appears when okg ingest runs in the
  same process, not yet when the work is queued (see Known issues). A failed
  okg install names the cause instead of the tail of a log. A successful
  publish no longer warns that there was nothing to rebuild.
• okg install --force over an existing database, when its final publish is
  refused because the database still holds the files claimed before the
  reinstall, now says so, points to okg status, and names the two commands
  to run next in order: okg catalog ownership claim, then okg run --once.
  It still exits 1 and never claims ownership itself.
• okg chat sync no longer says that an existing chat preset cannot be
  updated. Sync does update it; each census line is now labelled as a note,
  not a write failure.

Security

• Error and health messages, and bug reports posted to GitHub, now hide more
  kinds of secret: signed-URL values (such as ?sig=), Cookie and
  Set-Cookie header values, auth= values, Authorization credentials
  (Bearer, Basic, token, Digest and Negotiate) and token or password
  assignments. A bug report is refused before it is posted if any such value is
  left. A large slowdown when masking very long text is also fixed.
• When a deployment's access rules are on, a live read from an outside system
  (expand with live) is now refused by name before anything leaves the
  process. See "Breaking changes".

Agents get clearer answers

• A lookup in a live service that finds nothing says "not found", not that the
  service is down. A cut answer from a live service says where it stopped and
  how to ask for more, never "all shown".
• A search over file names says that file contents need a different search.
• Searching for the exact name of a record lists that record first, marked
  match_kind: exact_name. This covers the name of a class, function or
  method, and the name of any other record, such as the release
  CMSSW_14_0_0. Before, look-alike records could fill the top places, or
  only imports and references came back and the definition was missing.
  Imports and references that repeat a symbol's name follow its definition. Up
  to 50 same-name symbols are listed and counted exactly; more is a lower bound
  that says why. Under enforced access rules the count is never called exact.
  On a graph of 635,000 records the extra read added at most 28 ms to a
  search. Only a name that looks like code takes this route (see Known
  issues).
• When an agent calls inspect with kind and node_id at the top level
  instead of inside target, the refusal now shows the corrected call, ready
  to send, with other arguments such as select and max_bytes kept. The call
  is still refused, not served.
• A reply that goes on in another call prints a short 40-character handle for
  the next page. A refused handle names the call that restarts the read. A
  handle lives in the server process that issued it: it ends at a restart,
  after 2 hours, or when the server drops it to stay inside its limits. With a
  shared signing key set in the environment variable
  OKG_MCP_CONTINUATION_SIGNING_KEY, the reply prints the full token, which any
  server process can open.
• okg inspect texts no longer prints a stray code fence in the middle of a
  word when a page holds two pieces of one passage.

Access rules

• With per-user access rules on, keyword search applies the reader's rules
  before its read limits, so records a reader may not see cannot fill them. It
  also remembers each reader's access decisions instead of checking them again
  on every search; the cost measured at about 3 ms per search. Servers where
  each request names its reader, grep limits, relevance scores, search by
  meaning and records with no read class are not covered yet (see Known issues).
• Under access rules, keyword search for one kind of record, such as documents,
  keeps using the text index instead of a slower route the database wrongly
  picked. On one test graph the database time fell from 183-221 ms to 85-104 ms
  with identical results. Four other reads still use the plain filter.

Reading data that lives in another system

• Data held in an outside system can now be read like ordinary graph
  records, without storing it. A deployment declares a live type once: its id
  prefix, its fields, the outside calls that fill them, and its links. Agents
  then use the usual filter, map, aggregate, inspect and expand
  tools on it. The server plans the outside calls, sends the conditions the
  outside system can apply to it, fetches only the fields the call needs, and
  applies every condition itself before answering.
• A read that cannot work is refused before any outside call: a misspelled
  field, metric, grouping or link, a missing argument, an unsafe value, a
  plan over 1,500 calls, or a pinned graph version on a live type. Text from
  the outside system is marked as untrusted before an agent sees it.
• Each live answer starts with one line naming the system, the read time, how
  many records were read and listed, the calls made, and the conditions the
  outside system applied. A read that made more than 20 calls, or whose
  answer was cut, is kept in the server for 300 seconds, so the next call
  reads it without calling the outside system again.
• A new tool, run, chains reads in one call: live reads, value lists,
  differences and overlaps between result sets, and totals over the rows it
  holds, with one cap on outside calls per system. A deployment that lists
  the tools it serves offers run only when it names it; otherwise every
  tool surface has it.
• Four read fixes. grep matches its terms anywhere in one record, not only
  inside one passage. aggregate can merge the variants of one value (case,
  punctuation and word order) into one group. grep lists one line per
  record by default: id, title, path, match count and first matching line. A
  ranked search with a filter ranks 1,000 rows deep before filtering, instead
  of 200, so it no longer comes back nearly empty or slow.

Keeping a graph current

• After you undo a killed source run, the next publish no longer leaves the
  graph unable to publish. The publisher now writes a new version of the graph
  whenever rows were written since the last one, even if they cancelled each
  other out.
• Stopping the background worker now stops it. The supervised worker exits
  after its 10-second shutdown wait instead of waiting for a running publish to
  end, and
  PostgreSQL frees the publish lock within about 5 seconds. Work a cut-off
  statement had not committed is discarded; committed work stays, and the
  workflow engine recovers the run at the next start. Every okg database
  session now asks PostgreSQL to check that its client is still connected every
  5 seconds. OKG_CLIENT_CONNECTION_CHECK_INTERVAL_MS changes the interval,
  and 0 turns it off. It needs PostgreSQL 14 or later.
• A large catch-up publish no longer times out at the step that copies
  unchanged attribute rows forward. The copy now runs in small batches of about
  50,000 rows, each statement taking about 2 seconds on test data, instead of
  one statement that could run past the 15-minute limit.
• A command that is told to stop (SIGTERM, SIGHUP or Ctrl-C) now frees its
  connection claim and exits with the stop code, 143. Before, a stop that landed
  at the wrong moment could leave the claim for five minutes, or make the
  command exit 1.
• When a publish deletes a node for one source, it no longer counts other
  sources' outgoing links at that node as touched, so it does less unneeded
  work. No other source's link was removed before.
• The operator console's link counts by type stay correct when a different
  source or linker produces a link.

For people who build AKMON

• okg[cern] is no longer an install option, and the Docker image installs the
  plain package. An install line that still says okg[cern] only prints a pip
  warning that the extra does not exist. CERN readers come from the
  archi-okg add-on package.
• When okg pact refuses a version 5 change, it prints a command in the
  version 5 form that records the missing proof, where it used to print the
  version 4 form, which version 5 rejects. It prints one only when the change
  names a real test to run; otherwise it says to attach the check you ran.
• The release check accepts a Tests run started by hand on a tag of the
  release commit, so a release no longer needs dev frozen while its tests run.
  The newest finished run decides, and the release plan says how many earlier
  runs were not green.
• This repository's build check runs the installed package from outside the
  checkout, as a user would. Inside a source checkout, okg refuses to run a
  copy that is not that checkout's code, which made the check fail after
  0.2.0.

Breaking changes

These change how an existing installation behaves. A new install is not
affected.

• Live reads are refused while access rules are on (security). When a
  deployment sets nomos.runtime_enforcement to audit or enforce,
  expand with live is now refused with the code
  live_refused_under_access_rules, and no call goes to the outside system.
  Before, the call went out and its answer came back, checked only as the
  deployment, not as the reader. The same refusal applies when the server
  cannot tell whether the rules are on, and to the next page of a live answer
  kept from before the rules were turned on. This includes
  deployments/opensearch-reference, which runs in audit mode with live
  bindings: it no longer serves live reads until runtime_enforcement is
  off. The operator setting OKG_NOMOS_RUNTIME_ENFORCEMENT=off, set where the
  server starts, also turns the rules off for this check. Deployments with the
  rules off are unchanged.
• Sensitive deployments need a token for the viewer and the console. When a
  deployment sets nomos.runtime_enforcement to audit or enforce,
  okg ui and python -m okg.viz now refuse to start without a token, even on
  loopback (viz_read_auth_required). Put a secret in an environment variable
  and pass its name with --auth-token-env VARIABLE_NAME. If a supervised
  environment starts the viewer, set ui.auth_token_env to the variable's name
  in the environment file and list the variable under runtime.credential_refs;
  install refuses until you do. Export that variable where install, status and
  ready run. The health check sends the token only after it
  has shown that the port belongs to AKMON's own viewer process. Deployments
  with enforcement off are unchanged.
• Local folders that include PDFs can now parse them. A graph upgraded from
  0.2.0 does not read PDFs until okg deployment add-source adds the PDF reader
  to its extractors.yaml. From then on, every local folder source whose
  include matches PDFs parses them on its next sync: expect new records and a
  longer first sync there. Graphs created with okg deployment bootstrap have
  the PDF reader from the start. Reading PDFs makes no model call with the
  default reader file and embeddings off.
• Kept answers from live expand now go through the live reader. A kept
  answer of expand with live is now read the same way as a live type. It
  refuses a misspelled field and gains contains and order_by, so some
  answers can differ from 0.2.0. Reads of stored records are unchanged.
• Two rarely used arguments left the lean tool list. predicate on
  aggregate and args on expand are no longer listed on the lean tool
  surface; the full surface still lists them.
• Stricter checks when a deployment declares live types. The server
  refuses to start when one outside relation is declared on two record types.
  Server start and catalog composition refuse an undeclared or unbound
  relation, an unknown argument name and a duplicate id prefix. A publish is
  refused when it would store a record that uses a live type's id prefix.

Upgrading from 0.2.0

1. Install the 0.3.0 wheel, then run okg up <deployment> --plan and
   okg up <deployment>. okg up needs pg_dump and pg_restore from
   PostgreSQL 17 on your PATH.
2. This release has no database migration and no change to the database image,
   so the PostgreSQL steps in the 0.2.0 notes do not repeat. If you skipped
   0.2.0's move to pg_textsearch 1.5.1, do it now: see
   docs/operate/lifecycle.md, "Upgrade to 0.2.0".
3. If a deployment uses access rules at audit or enforce, add
   --auth-token-env to the viewer or console you run, and expect live reads
   to be refused until you set runtime_enforcement: off (see Breaking
   changes).
4. PDFs are not read after the upgrade alone. Once okg deployment add-source
   adds the PDF reader to extractors.yaml, every local folder source whose
   include matches PDFs reads them on its next sync.
5. After the upgrade, the first publish that would otherwise change nothing
   writes one new graph version with the same content, once per deployment
   branch, while it records the new check for live types.

Known issues

• Live types are not documented yet. How to declare one is described in
  the code and the error-code reference only, and no bundled deployment
  declares one.
• Some run hints and pages with live steps are off. When a run is
  pinned to an older graph version, its live steps are still read now, while
  the reply names the older version. The hint to send one live step on its
  own adds a graph version, so following it is refused. Page 2 of a live step
  inside run reads the outside system again, so rows that change between
  pages can be skipped or repeated.
• Codex is reported as connected on a machine without Codex. okg install
  creates the folder ~/.codex, and okg agent connect then takes that folder
  as a sign that Codex is installed, writes ~/.codex/config.toml and says
  "Codex connected". Check that codex runs before relying on it; the other
  agents are detected correctly.
• Removing a source does not remove its records. If you take a source out
  of the source registry, the records it produced stay in the graph and in
  search results after the next publish, including pages whose address has
  since changed.
• A fresh install can fail on its first ingest. When the code-structure
  source runs before the repository checkout exists, it fails with the given reference name 'refs/heads/.invalid' is not valid and okg ingest exits 1.
  Run okg ingest a second time; it passes.
• code_graph alone does not load. Composing only the code_graph module
  fails the catalog load without saying why. It needs the modules
  agent_sessions, quality_evidence, ci and openspec; compose all five.
• Queued ingest gives no note about the other sources. When the runtime is
  on and okg ingest queues the work, a failed source still gets the new reason
  but not the note saying what became of the other sources. The suggested
  command that leaves a source out also needs --inline there.
• Some rare secret forms are still shown. The new masking does not cover
  signed-URL values after # or ;, URL-encoded forms, JSON-style "sig"
  values, X-Goog-Signature, apikey, OAuth code=, and auth: value.
• Graph content from Git and GitHub sources is not masked with the new
  rules. The new masking applies to messages and bug reports. Text stored in
  the graph from the github, github_actions, pact and Git-history sources
  can still hold signed-URL, cookie, auth= and token values that the new rules
  would hide.
• Install output can print a default password. When a database answer is
  written as ${VARIABLE:-default}, the install output shows the default
  value as written. Keep a real password out of the default, or read it from
  the environment with no default.
• Adding one source to an existing profile install means a full reinstall.
  okg deployment add-source covers local folders, GitHub and manifest
  sources. For another source of a bundled profile, run
  okg install --force --yes, which takes 3 to 10 minutes.
• Code parsing on a slow network. On first use, code parsing downloads
  about 500 MB of language grammars, and the download stops after about a
  minute. On a slow connection, download them in advance; see
  docs/runbooks/troubleshooting/first-run.md.
• The quickstart was followed on podman. The Docker commands in it are the
  same commands, but they were not run on a Docker host for this release.
• One missing merge commit stops the whole GitHub source. With
  commit_repo set, a merged pull request whose merge commit the git source
  does not hold (a shallow clone, or a branch it does not read) makes the graph
  refuse the whole GitHub batch on every sync. Sync the git source first, or
  clear commit_repo.
• Exact-name search skips plain words. Only a name that looks like code
  (an underscore, a dot, or camel case such as getUser) gets the exact-name
  route. A one-word name such as Serializer still returns the pages it did
  before, not its definition first. Under enforced access rules, an
  exact-completeness search for a code-shaped name returns no count.
• Upgrade, restore and copy need PostgreSQL client tools. okg up,
  okg deployment backup restore and okg db clone need pg_dump and
  pg_restore at the database server's major version or newer (17 for the
  bundled database).
• The background worker is off by default. To keep a graph current
  without running okg ingest yourself, set runtime.enabled: true in the
  deployment's deployment.yaml, run okg runtime bootstrap, and keep
  okg runtime worker running; it checks each source every 5 minutes. With
  the worker on, okg ingest waits for it. okg doctor can report red for
  the first few minutes after the first worker starts. On Linux, okg up
  refuses a graph that has run a worker unless the worker service is
  installed (okg dogfood service install --deployment <deployment> --adapter systemd --apply) and okg up runs as root.
• Agents and a password kept out of the command line. If you installed
  with --postgres-dsn '${OKG_DSN}' (single quotes, so okg reads the
  address from the environment), agents cannot see your shell's
  environment: copy the address into the deployment's .okg/local.env
  first. okg agent connect prints the exact command.
• Publishing a large code source takes about 30 minutes, and other
  updates of the same graph wait for it. On a graph of about 785,000
  records it measured 1,720 to 1,780 s. No data or search results are lost.
• A very large catch-up publish may still time out. Copying unchanged
  attributes forward now runs in small batches, but that fix is not yet proven
  on a graph of millions of rows, and the copy still scales with the whole graph,
  not only the changes. Another full-graph step could hit the 15-minute
  statement limit first, and the next attempt would hit it again. If a publish
  stops that way, set
  publisher.hard_statement_timeout_seconds (for example 7200) in the
  deployment's deployment.yaml; also set
  publisher.allow_raised_statement_timeout: true so the readiness checks
  accept the longer limit. Restore the default after it publishes. Normal
  publishes and new installs are not affected.
• The viewer's read pages need a token only when you ask for one, or when the
  deployment is sensitive. Started the default way on a deployment that is not
  sensitive, python -m okg.viz and the operator console (okg ui) show graph
  data to anyone who can reach their port, and on a shared machine every
  account can reach 127.0.0.1. Start them with --auth-token-env VARIABLE_NAME to require the token in that variable for the read API
  (/viz/api/); okg ui then requires it for pages too. okg ui listens on
  loopback only. python -m okg.viz refuses a network address unless both this
  token and OKG_VIZ_OPERATOR_TOKEN are set. Operator controls always need
  OKG_VIZ_OPERATOR_TOKEN.
• With access rules on, search can still hint at hidden text. Keyword
  search now applies the reader's rules before its read limits, which closes
  the case where many hidden matches changed the shape of the answer. Servers
  where each request names its reader, grep limits, relevance scores, search
  by meaning and records with no read class are not covered yet, so
  a reader there can still learn that a word exists somewhere they cannot read. If the existence of a term must stay secret, keep that data
  in a separate deployment rather than relying on access rules alone.
• --verbose is ignored by two install paths. Profiles with their own
  installer (academic-researcher) and --template installs keep their old,
  longer output.

Install

Everything below was built from the commit this tag names. The tags v0.3.0 (images) and 0.3.0 (chart) are never moved or overwritten; pin a digest to be certain.

Artifact	Reference	Digest
Runtime image	ghcr.io/mitdbg/akmon:v0.3.0	sha256:b7041a3affed98e3f1d3958dba76f0691affe0132fb3a42b0487b84622ab33c1
PostgreSQL image	ghcr.io/mitdbg/akmon-postgres:v0.3.0	sha256:968403b661b2a40e080e8ea86a1bfbdfcda19e71c14c67e40c279e19c57c3969
Helm chart	oci://ghcr.io/mitdbg/charts/akmon version 0.3.0	sha256:56e725a8a17e88261ec7756e993594cb971c8b44afcec4c283817fda2870d952

helm install okg oci://ghcr.io/mitdbg/charts/akmon --version 0.3.0 -f values.yaml

The chart's appVersion is v0.3.0: with images.okg.tag and digest empty it runs ghcr.io/mitdbg/akmon:v0.3.0, and its PostgreSQL image is ghcr.io/mitdbg/akmon-postgres:v0.3.0.