Skip to content

Releases

Chronos DB 0.4.10

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 09 Nov 22:13

Rules for agents that run on their own. Merge policies gain rules on values, a breaker for a fleet, a guard
against prompt injection, and a reviewer agent that can decide every merge without being able to approve what only a
person should.

The macOS builds need macOS 13 or later, as for 0.2.0. While the build service is offline, this release was
built on the maintainer's Mac with the release's own toolchain (Rust 1.95.0, zig 0.15.2 for every target) and
signed as before, as the chronos-release service account.

  • row_rules: conditions on the rows a merge changes, over old.col and new.col, read as a CHECK
    ('payments: new.amount >= -50; prices: new.price <= old.price * 1.2'). Reasons name rows by key, never values.
  • max_nulled: most values one merge may empty.
  • all_rows_per_hour, all_deletes_per_hour, all_effects_per_hour: limits on every merge into main by any
    agent keeping to a policy, together, a reviewer's approvals included.
  • untrusted_columns, untrusted_guards: a world that may have read text from outside (tickets.body) and
    changes a guarded table waits for a person. The policy's worlds keep their reads without holding for stale ones.
  • spot_check: one merge in n, at random, waits for a person.
  • review_every: every merge waits for review, so a reviewer agent decides each one.
  • person_only: the rules whose holds a reviewer agent may not approve (by default deletes, schema, untrusted
    reads, the fleet limits and spot checks); an admin's agent can't merge those either. SHOW REVIEWS has a
    needs_person column.
  • Holds notify chronos_review_held, so a reviewer listens instead of polling.
  • untrusted_columns = '*' and trusted_columns: every text column untrusted by default, with exceptions.
    'tickets.*' takes every text column of one table.
  • Tainted rows: rows a merge writes after its world read untrusted text are tainted, and reading one counts as
    reading untrusted text. Kept in the tallies file; a person's own merge or a delete clears a row.
  • Reasons without values for agents: row keys that are text, what a merge check found and what a SELECT gave show
    as … to agents, a reviewer among them, in refusals, SHOW REVIEWS, CHECK READS and check_merge_policy.
    People see them as they are.
  • watch_totals, max_shift: scalar queries run before and after a merge; one that moves more than max_shift
    percent (10 by default) holds it, for a person by default.
  • max_unusual: a merge far bigger than the policy's usual (the median of its last 50) waits: drift.
  • canary, canary_wait: a big merge lands a few rows first (55000), the rest after the wait.
  • effects_delay: effects an agent's merge lands are delayed: no claim, no NOTIFY, and UNDO MERGE
    meanwhile takes them back unsent.
  • A run's own row_rules: CREATE WORLD ... WITH (row_rules = '...') adds conditions on top of the policy's.
  • Changed: another agent with no policy can't merge a world held for person-only rules, but the world's own agent
    whose policy an admin removed still can.

Every change in detail: https://github.com/Abhishekxdg/chronosdb/blob/main/CHANGELOG.md

Chronos DB 0.4.9

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 08 Nov 22:16

Writes that fit the database you already run. Agents can write as the end user's Postgres role, tables get
write-back rules, big worlds merge in chunks, a multi-tenant database can be attached a tenant at a time, merges
take fewer round trips, and attach works beside MySQL and MariaDB too.

The macOS builds need macOS 13 or later, as for 0.2.0. While the build service is offline, this release was
built on the maintainer's Mac with the release's own toolchain (Rust 1.95.0, zig 0.15.2 for every target) and
signed as before, as the chronos-release service account.

  • License files in the public repository: the Business Source License is now LICENSE at the root (it was LICENSE-ENGINE), and the MIT License moved to licenses/MIT.txt, so GitHub no longer labels the whole repository MIT. No change to either license or to what each covers.
  • Merges in quick succession no longer re-read whole tables: an attached folder's merge looks for rows triggers wrote besides its own, from Postgres's counts on the connection. Those also hold the connection's earlier commits for up to a second, so a merge soon after another took them for trigger writes, and main refreshed the table. It now compares the counts before and after its writes. Main no longer picks up other rows changed in Postgres at that moment by chance: they arrive with a refresh, --follow, or a conflict.
  • Agents write back as the end user: an agent can be given a Postgres role and settings (create agent ... with (pg_role = 'authenticated', pg_settings = '{"request.jwt.claims": {"sub": "u-17"}}')). Worlds it forks are written to an attached Postgres as that role, with those settings: rows are still locked and compared as the attach role, but the inserts, updates and deletes run after SET LOCAL ROLE, so Postgres's own grants and row-level security policies judge them as they would the end user's own writes. A row the policies hide or refuse fails the merge with 42501, and nothing is written. The same applies when a person merges the world, or rehearses the merge. Only custom settings (prefix.name) are accepted. The attach role must be a member of the role, and a superuser or BYPASSRLS role is refused. SHOW AGENTS shows the role, but never the settings' values.
  • Write-back rules per table (writeback.json): an attached folder can say, table by table, how merges reach Postgres. delta columns (stock, balances) are written as increments and never conflict, ignore and keep_live columns aren't compared, check holds a merge whose rows break a condition in Postgres, writable refuses changes to rows that are settled (closed deals), scope keeps a merge inside any SQL condition (tenancy that isn't one column), resolver settles a row both sides changed with your own SQL function instead of refusing the merge, procedure applies a table's changes through your own function, and outbox records each change in the same transaction for your worker. mode: "observe" runs every write-back and rolls it back, and isolation: "serializable" makes Postgres refuse write skew. The file is read at every merge. One that doesn't parse refuses merges.
  • MERGE WORLD ... IN CHUNKS OF n: a big world merges n rows at a time, each chunk its own write-back transaction, and can be resumed after a failure. People and admins only.
  • Loading a tenant at a time (chronos attach --lazy-tenant org_id): tables with the tenant column, and tables whose foreign key points straight at one, start empty. Main takes a tenant's rows from Postgres the first time a world is forked for it (an agent held to the tenant, or WITH (tenant = ...)), or at chronos refresh <dir> --tenant 42. Refreshes and --follow read only the tenants loaded, so a big multi-tenant database is never copied whole.
  • Faster merges into an attached Postgres: the write-back sends its statements in pipelines, several per round trip. A small merge now takes five round trips instead of nine or more, whatever tables and kinds of change it has, and writing as the end user adds one instead of two. Big chunks still go one statement at a time.
  • Attach beside MySQL and MariaDB: chronos attach with a mysql:// CHRONOS_SOURCE_URL copies InnoDB tables with a primary key. Merges into main are written to MySQL first, in one transaction of three round trips, and a row changed there meanwhile is a conflict with nothing written. A crash around COMMIT is settled from chronos_merges, and chronos refresh brings main up to MySQL. Not yet with MySQL (refused, saying so): --follow, dry runs, check_reads, merge checks in the source, writeback.json, pg_role, --lazy-tenant, binary and generated columns, and inserts into AUTO_INCREMENT tables. See docs/guides/attach-mysql.md.

Every change in detail: https://github.com/Abhishekxdg/chronosdb/blob/main/CHANGELOG.md

Chronos DB 0.4.8

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 02 Nov 23:39

Counts read right, and rustls with its security fixes. No change to how merges, checks or storage work from 0.4.7.

The macOS builds need macOS 13 or later, as for 0.2.0.

  • Words: a refusal or review counting one row says "1 row", not "1 rows".
  • Dependencies: rustls 0.23.46, for its security advisories (the worst: a session's Debug output could show its TLS secrets).

Every change in detail: https://github.com/Abhishekxdg/chronosdb/blob/main/CHANGELOG.md

Chronos DB 0.4.7

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 02 Nov 22:02

The Studio, in the look of chronos.2090.tech. The same views and the same engine as 0.4.6; only the Studio's looks changed.

The macOS builds need macOS 13 or later, as for 0.2.0.

  • Studio: redrawn in Signal, the design system of chronos.2090.tech: near-black grounds, hairlines, hard corners and lime for the one thing that matters; Inter Tight, JetBrains Mono and Doto; the Chronos mark and a favicon; nothing under 12px. Dark by default (light, or following the system, in Settings). main is blue, not amber, and held changes are amber.
  • Studio: on a new database with only main, the worldline's labels no longer lie on top of each other, and now is at the right. A world's label no longer touches the line below it; the row count says "1 row"; a world's tooltip no longer says "created" with no date.

Every change in detail: https://github.com/Abhishekxdg/chronosdb/blob/main/CHANGELOG.md

Chronos DB 0.4.6

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 01 Nov 04:15

A tenant's run is held only by its own tenant's rows. A world read by an agent held to a tenant, as every gateway run is, was held whenever another tenant's rows changed in a table it read, and a child table of more than 100,000 rows couldn't be checked against Postgres at all, so such a run could only be approved past its read check. Now only the tenant's rows count, in Chronos and in Postgres.

The macOS builds need macOS 13 or later, as for 0.2.0.

Fixed:

  • A tenant's reads are checked against its tenant's rows only. A world read by an agent held to a tenant (as every gateway run is) was held by any change to a table it read whole: the app writing another tenant's rows, in main or in Postgres, held it, and a child table of more than 100,000 rows couldn't be checked against Postgres at all ("read lead_notes more broadly than can be checked against Postgres"), so the run could only be approved past its read check. The read set now keeps whom its reads were made for. When every read was made for one tenant, a changed row counts only if it's that tenant's, as it was or as it is, followed through foreign keys as the tenant policy follows them; against Postgres the table is read by the tenant's condition in Postgres's SQL, so the 100,000 rows counts the tenant's. Reads by anyone else (a person's SELECT in the world) still count every row.

Every change in detail: https://github.com/Abhishekxdg/chronosdb/blob/main/CHANGELOG.md

Chronos DB 0.4.5

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 31 Oct 11:28

A run held to a tenant can change child tables. In 0.4.4 a tenant agent could write a table held to it through a foreign key, but the merge then took every such row for another tenant's: the change was held, and refused when approved. It merges now, and another tenant's child rows are still refused.

The macOS builds need macOS 13 or later, as for 0.2.0.

Fixed:

  • A world held to a tenant merges its changes to child tables. A row of a table held to the tenant through a foreign key (0.4.4) has no tenant column, and the merge's scope check took it for another tenant's: an agent's insert of a note under its own lead was held "outside tenant", then refused when approved. The check now follows the row's foreign keys to its tenant: as the row was, in the parent now, and as the world has it. A world forked with a tenant by any user is checked the same way, so a child row of another tenant, or of none, is still refused.

Every change in detail: https://github.com/Abhishekxdg/chronosdb/blob/main/CHANGELOG.md

Chronos DB 0.4.4

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 30 Oct 21:42

Attach to the Postgres you already run: tested on Postgres 12 to 18 and Supabase, and tables with row-level security, without a primary key, with generated columns or partitioned are attached. Main keeps up with Postgres as it changes (serve --follow), masked columns work on attached folders, and agents held to a tenant are held to it in child tables too.

Changed from 0.4.3:

  • An agent held to a tenant can't write a table shared by every tenant. A table with neither the tenant column nor a foreign key to a table that has it (up to three hops) is still read whole, but writes to it are refused (42501), where they used to change it for every tenant. Give such a table the column or a key, or change it outside the agent.

The macOS builds need macOS 13 or later, as for 0.2.0.

  • Attach: tested on Postgres 12 to 18, and two fixes it found. A database whose encoding isn't UTF8 (LATIN1, WIN1252, ...) with any non-ASCII text couldn't be attached ("invalid byte sequence for encoding UTF8"): Chronos's connections now ask Postgres for UTF8, as libpq's clients do. A refresh now takes enum labels added in Postgres (ALTER TYPE ... ADD VALUE), in Postgres's order, before the rows that may hold them; until then no world could write a new label, even after a refresh (chronos refresh prints them, HTTP refresh lists them as labels). tests/attach_compat.rs runs about 80 cases (column types, key shapes, constraints, foreign keys, quoted names, a minimal role) under four database setups against whichever Postgres CHRONOS_TEST_PG names; tests/attach.rs runs on Postgres 12 too.
  • Attach: tables with row-level security. Attach refused every table with row-level security on, which on Supabase is every table its dashboard makes. Now it takes one the role reads past (a superuser, BYPASSRLS, the owner of a table that doesn't FORCE it, or a role a USING (true) WITH CHECK (true) policy for all commands shows every row, with no restrictive policy on it), since the copy then holds every row. It still refuses one whose policies hide rows from the role, now saying how to fix it, and a refresh still stops on such a table.
  • Attach: Postgres 11 or older is refused, saying so. It failed with Postgres's own "column attgenerated does not exist"; now attach says the version is too old and that it needs 12 or later, before it reads or makes anything there.
  • Attach: tables without a primary key. A table with none was refused. Now a unique index on NOT NULL columns stands for one (no WHERE, no expressions, not deferrable; a constraint's first, then the fewest columns): the write-back finds rows by it, main gets those columns as its primary key, and a refresh stops if the index goes or a column of it may be NULL. A table with neither is still refused, saying so.
  • Attach: tables with generated columns. A table with a GENERATED ALWAYS AS (...) STORED column was refused. Now it's attached: Postgres works the column out, and main too from the same expression; merges never send it, and it's never compared with Postgres's, so a value the two print differently isn't a conflict. A column that becomes generated, or stops being, stops the refresh.
  • Attach: partitioned tables. A partitioned table was refused. Now it's attached whole, as one plain table in main holding all its partitions' rows; a merge writes the partitioned table and Postgres puts each row in its partition (moving one whose partition key changed). Rows written by a trigger on a partition are found as on any table, and the table refreshed. A partition alone is refused, saying to attach its partitioned table.
  • Attach: refresh takes columns dropped, renamed or retyped in Postgres. A refresh took only added columns; any other change to a table's columns left it out until a new attach. Now a column dropped, renamed or retyped in Postgres is so in main too (altered in the refresh's report), and its rows then take Postgres's values. Attach keeps each column's attnum so a rename is told from a drop and an add (older folders fill it in at their next refresh). A world forked before such a change is refused at its merge, saying to fork again: its rows of that table were made in the old shape. A changed key, row-level security or generated column still needs a new attach.
  • Attach: masked columns. A folder attached to Postgres refused ALTER TABLE ... SET MASKED, so its columns couldn't be masked from agents. Now main takes SET MASKED and DROP MASKED (they change only what agents read, never Postgres); worlds forked after read the column masked, a refresh keeps the mask, and HTTP attached marks each masked column (mask) and gives each table's SQL name (sql). In a world it's refused, saying to mask on main.
  • Attach: serve --follow takes Postgres's changes as they happen. A refresh reads every row of every attached table, so keeping main fresh meant a refresh every few minutes, and rows no merge touched could be that old. Now chronos serve --follow reads a logical replication slot (test_decoding) every second and re-reads only the rows it names, by key, under the write-back lock; a merge's own writes coming back take nothing. A TRUNCATE, a delete under REPLICA IDENTITY NOTHING or a column change refreshes that table whole. It needs wal_level = logical and a role with REPLICATION; chronos refresh <folder> --unfollow drops the slot, which keeps Postgres's WAL until read.
  • Attach: a merge by columns settles Postgres's changes to other columns by itself. A row the world changed that someone changed in Postgres meanwhile was a conflict, even when the two changed different columns, and the merge had to be run again. Now a merge BY COLUMNS (the gateway's by_columns: true) that meets such rows runs once more by itself, after main took Postgres's rows: the world's columns and Postgres's combine, and only a column both changed is still a conflict.
  • Agents held to a tenant: child tables through foreign keys, and shared tables read-only. An agent made WITH (tenant = 'org_id = 42') was held to its tenant only in tables with the org_id column, and read and wrote every other table whole. Now a table without the column but with a foreign key to a table that has it, up to three hops away (order_items.order_id to orders.id, a note on an item through the item), shows it and lets it change only the rows whose key names a row of its tenant's, and a row it writes must name one (a null key names none). A table with neither is shared by every tenant: it still reads it whole, but an INSERT, UPDATE, DELETE, MERGE or COPY FROM of it is refused (42501), as are the row API's writes of it, since another tenant would see what it wrote. SHOW AGENT name PRIVILEGES says how each table is held to the tenant.

Every change in detail: https://github.com/Abhishekxdg/chronosdb/blob/main/CHANGELOG.md

Chronos DB 0.4.3

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 30 Oct 05:51

Postgres compatibility from running real apps (PostgREST and Supabase's setup, psycopg's server-side cursors, Django's explain(), deferred UNIQUE constraints, a third batch of PostgreSQL 16's regression suite), and agents on production data held tighter: masked columns that triggers can't leak, tenants that must name a real column, low-priority scans, and an attached Postgres's write-back you can rehearse and undo.

Changed from 0.4.2:

  • A bare VACUUM or ANALYZE reads every table to record the planner's statistics (it did nothing before), and an agent that may only read can't run them any more.
  • An agent's tenant must name a column some table has: create_agent and ALTER AGENT ... SET (tenant = ...) refuse one that no table on main has, where a typo made an agent that saw every tenant's rows.
  • An agent without unmask can't UPDATE or DELETE rows whose trigger reads a masked column: the statement is refused (42501), where the trigger used to copy the value somewhere the agent could read.

The macOS builds need macOS 13 or later, as for 0.2.0.

  • A window frame's ROWS or GROUPS offset may read parameters. ROWS BETWEEN x PRECEDING AND x FOLLOWING, x a SQL function's argument (or $1), is worked out when the window runs, as Postgres allows; a column there is still refused (42P10). window.sql's unbounded_syntax tests pass.

  • VACUUM and ANALYZE record the planner's statistics, and joins can follow PostgreSQL's order. pg_class.relpages, reltuples and relallvisible were always 0, -1 and 0; now VACUUM and ANALYZE record them as Postgres would have them, the pages from a model of Postgres's heap (8 kB pages, each row laid out as Postgres lays it out), and so do CREATE INDEX and REINDEX, as Postgres's index build does. reltuples is a float4, and UPDATE pg_class SET reltuples = ... is kept too. TRUNCATE, and an ALTER TABLE Postgres rewrites the table for, reset them; ANALYZE of a partitioned table records its partitions' rows; VACUUM (ONLY_DATABASE_STATS) records none. A bare VACUUM or ANALYZE, which did nothing, now reads every table to count its rows (writers go on meanwhile). They write the world, so an agent that may only read can't run them any more. A new setting, chronos.follow_pg_plan, joins the tables of a query PostgreSQL 16 can only run as nested loops (cross joins, and joins on conditions such as a.x < b.y or geometric operators) in the order Postgres's planner would pick from those statistics, so their rows come out in Postgres's order. The same rows are read in any order, but with three tables or more the work done can differ. It is computed by a port of Postgres's cost model and join search (src/sql/opt, under the PostgreSQL License: see NOTICE), and anything it doesn't model exactly keeps Chronos's own order; a table whose rows were deleted, updated or loaded by COPY takes pages in Postgres that Chronos doesn't count, and may be ordered differently. On in exact mode (CHRONOS_PG_EXACT=1), off otherwise. See join order.

  • Postgres compatibility: a third batch of the PostgreSQL 16 regression suite. EXPLAIN takes FORMAT JSON, XML and YAML, VERBOSE, SETTINGS, SUMMARY and GENERIC_PLAN (BUFFERS and WAL show zeros), and fails on the data errors Postgres's planner meets (1/0 in a constant part); catalog tables have a ctid; indexes keep their access method, DESC/NULLS FIRST, INCLUDE columns and operator classes, and the index property functions (pg_index_column_has_property and its kin) answer from them; CREATE ACCESS METHOD, DROP ACCESS METHOD and table access methods (USING, default_table_access_method, ALTER TABLE ... SET ACCESS METHOD).

  • Every encoding Postgres has converts, as Postgres converts it. convert_to and convert_from knew UTF8, LATIN1 and SQL_ASCII; they and the new convert(bytes, from, to) now take all 42 of Postgres's encodings (EUC_JP, SJIS, GB18030, Big5, KOI8R, the WIN and ISO 8859 sets...), by a port of Postgres 16's own conversion code (src/common/wchar.c, conv.c and the 84 functions of conversion_procs, with their generated maps converted by tools/pg_unicode_maps.py), with its messages and SQLSTATEs (invalid byte sequence for encoding "UTF8": 0xe8 0xb1 0xff, 22021; character with byte sequence ... has no equivalent in encoding ..., 22P05). pg_char_to_encoding takes every alias (unicode, win, mskanji...). CREATE [DEFAULT] CONVERSION checks its encodings and function as Postgres does, and refuses a second default for a pair in a schema. The regression suite's conversion.sql passes: its test_enc_setup and test_enc_conversion (regress.c) run as their C does.

  • An aggregate in a subquery of only the outer query's columns is the outer query's, as in Postgres. HAVING EXISTS (SELECT 1 FROM b WHERE sum(DISTINCT a.four) = b.four) sums the outer group, and SELECT (SELECT count(*) FILTER (WHERE outer_c <> 0) FROM ...) FROM t1 makes the outer query aggregate (one row); these failed with an aggregate of only the outer query's columns in a subquery is not supported. In a WHERE's subquery or a subquery in FROM they are still refused.

  • Attribute notation, functional notation and nested set-returning calls, as Postgres reads them. p.hobbies, where p has no column hobbies but a function hobbies takes its row, calls hobbies(p); (h.equipment).name likewise; name(x) is x.name where no function name takes x; and a set-returning function in a select list nested in another's arguments (name(equipment(p.hobbies))) runs first, a level each, as Postgres's ProjectSet runs them. SELECT * beside such a call is the FROM items' columns only. The regression suite's misc.sql goes from 58% to 98%.

  • pg_get_catalog_foreign_keys() and ten more hash functions, ported from Postgres 16. pg_get_catalog_foreign_keys() lists the 219 foreign keys between the system catalogs, generated by Postgres's own genbki.pl from the catalog headers (tools/pg_catalog_fks.py), so the regression suite's oidjoins.sql checks every one. time_hash, timetz_hash, timestamp_hash, uuid_hash, pg_lsn_hash, hashinet, hashmacaddr, hashmacaddr8, hashoidvector and hash_numeric, and their extended forms (with a seed), give a real server's numbers.

  • Object addresses for aggregates, text search objects, transforms, subscriptions and Postgres's own casts. pg_get_object_address, pg_identify_object and pg_identify_object_as_address found no oid for an aggregate made with CREATE AGGREGATE, a text search parser, dictionary, template or configuration made with CREATE TEXT SEARCH, a subscription, a domain's constraint, a built-in cast or a member of a built-in operator family (operator 1 (integer, integer) of pg_catalog.integer_ops USING btree); each now has one, and names it as Postgres does. CREATE TRANSFORM and DROP TRANSFORM are taken (kept as the type and language: no language here runs transforms). CREATE TEXT SEARCH PARSER takes its end function, and a text search object is in the schema it was made in. DROP SCHEMA ... CASCADE drops the schema's aggregates and text search objects too, listed in the order they were made. CREATE SUBSCRIPTION ... WITH (connect = false) gives Postgres's warning that the subscription isn't connected.

  • PL/pgSQL: a polymorphic function's CONTEXT names it as declared, and an expression's subquery reads the catalog. An error in a call of foreach_test(anyarray) gave CONTEXT: PL/pgSQL function foreach_test(integer[]) line 4 ..., the types of the call; it now names the function as Postgres does, foreach_test(anyarray). RETURN (SELECT ... FROM pg_class) and other expressions whose subqueries read a catalog table failed with relation "pg_class" does not exist; they now read it.

  • The catalog's rows name each other as Postgres's do. Postgres's own catalog tables and system views (pg_proc, pg_type, pg_roles, ...) are rows of pg_class at the oids Postgres gives them, so what pg_description, pg_depend and pg_init_privs name by classoid is there; a table's reltoastrelid names its TOAST table, a row of pg_class in the pg_toast schema (now in pg_namespace); and the input and output functions of hstore's and ltree's types are rows of pg_proc. The regression suite's oidjoins, which checks every foreign key pg_get_catalog_foreign_keys lists, finds no violations.

  • GRANT ... ON DATABASE postgres (and template1) grants on main, the world a connection to those maintenance databases is in, as Supabase's init scripts expect; it was refused as naming another world.

  • contrib/bloom: CREATE EXTENSION bloom is accepted, and CREATE INDEX ... USING bloom (...) WITH (length = ..., col1 = ...) (Django's BloomIndex) makes an index kept as a btree on its columns. Its parameters are checked as bloom checks them, its columns need bloom's int4 or text operator classes, and pg_am lists bloom once an index uses it.

  • psycopg's server-side cursors work. A cursor made with SQL DECLARE is a portal of its name, as in Postgres, so the extended protocol's Describe of it gives its columns; psycopg 3's named cursors (Django's .iterator()) describe theirs so, and failed with portal "..." does not exist.

  • Deferred UNIQUE constraints. UNIQUE ... DEFERRABLE INITIALLY DEFERRED, and DEFERRABLE with SET CONSTRAINTS ... DEFERRED, let a transaction hold a key on two rows until it commits (swapping two rows' positions), as in Postgres; the COMMIT fails with 23505 if two rows still hold it, and SET CONSTRAINTS ... IMMEDIATE checks what was put off at once. INITIALLY DEFERRED was refused (0A000), which stopped Django's and Rails' test schemas from loading. pg_constraint and pg_get_constraintdef show the deferral.

  • Built-in functions take GRANT and ALTER FUNCTION ... OWNER TO (`GRANT EX...

Read more

Chronos DB 0.4.2

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 25 Oct 13:21

Joins, merges and memory, from running BranchBench (Columbia's benchmark of agent workflows on branching databases): joins on several columns and joins too big for memory, merges where both sides changed a table's columns, a merge that counts its conflicts, DIFF ... SUMMARY and UNDO DIFF, and a memory limit for every query.

Changed from 0.4.1: every statement has a memory limit. A statement past CHRONOS_QUERY_MEM (default a quarter of the machine's memory; 0 for none, as before) fails alone with 53200, and statements together are held to twice that, where a runaway join used to take the server down. See the entry below.

  • Joins on several columns too big for memory hash on all of them too. A hash join whose other side outgrows CHRONOS_WORK_MEM goes to disk, split into files by the join key; that split, and each file's hash table, used the first equality alone. On TPC-C's (w_id, d_id, ...) keys that put a warehouse's rows in one file and paired each with all the others there: CH Q3 and Q18 took 17 minutes at five warehouses (16 MB of work memory: at one), where they now take 0.6 s. Found by BranchBench.
  • Joins take the equalities their ON clauses imply. customer c JOIN new_order n ON n.w = c.w AND n.d = c.d JOIN orders o ON o.w = n.w AND o.d = n.d AND o.id = n.o AND o.c = c.id (TPC-C's CH Q3) joined customer to new_order first, as written: every new order of a district met every customer of it, and the rows held grew past 12 GB at one warehouse (at five, the server was killed for memory). Equal columns now carry across (o.w = n.w and n.w = c.w give o.w = c.w, between columns of one type and collation), and the next table joined is the one linked by its whole key, then by the most columns: orders comes second. Q3 at one warehouse: 0.35 s and 850 MB. Found by BranchBench.
  • Joins on several columns hash on all of them. A join on (w_id, d_id, o_id) built its hash table on the first equality alone and checked the others row by row, so where that column has few values (one warehouse) every row probed the whole warehouse: TPC-C's order_line JOIN orders at one warehouse took over two minutes, orders LEFT JOIN new_order 4 s, customer JOIN orders 27 s (Postgres: 74 ms, 3 ms, 11 ms). An in-memory hash join now keys on every a.x = b.y of the same kind; the same three take 56 ms, 14 ms and 15 ms. A join bigger than CHRONOS_WORK_MEM (on disk) still hashes on one column. Found by BranchBench, whose TPC-C consistency checks are such joins.
  • MCP tells agents to settle conflicts by column first. The instructions used to say "decide which side wins and call merge again with resolve set to ours or theirs", and agents did, overwriting main's changes to columns they never touched. They now say that merging the same way again fails the same way, and give the safest order: merge_preview, then by_columns=true (with resolve to pick a side only for the columns both changed), then rows for single rows, and resolve alone, which takes whole rows, last. A conflict from merge says the same, also when the call already had by_columns, rows or the like (before, only a plain merge got the hint), and for a conflict with an attached Postgres. The resolve and by_columns descriptions and the TypeScript and Python clients' merge docs say what each does to the other side's columns.
  • The log ships to cloud storage as it is written. With --remote, a database's log used to stay on local disk until the next checkpoint (every 64 MB of log), so losing the machine lost everything since. It now goes to the bucket in chunks (wal-<segment>-<offset> objects) within wal_shipping_delay (1 s) of being written, and a --takeover from an empty folder on another machine replays it. ALTER SYSTEM SET wal_shipping = sync makes a commit to main or a merge return only once its log is in the bucket, so a takeover loses no acknowledged write; off ships only at checkpoints, as before. A takeover fences the old writer at the end of the shipped log: its next upload fails and it ships nothing more. Checkpoints delete the shipped log the bucket no longer needs. Also new: gs://bucket/prefix remotes (Google Cloud Storage). See operations and the design.
  • USE WORLD 'name@when' switches a session to a world as it was. The docs said use world 'main@2026-09-20 10:00' reads the past for the rest of the session, as connecting to the database main@... does, but USE WORLD (and SWITCH WORLD) only took a name or an ID and failed with branch "main@..." does not exist. It now takes 'name@when' too: read only (writes fail with 25006), 22023 for a time outside the history window, 3D000 for a world that doesn't exist. Found by BranchBench, whose commit refs read a branch at a commit.
  • DIFF sees a column's new size. ALTER TABLE t ALTER COLUMN v TYPE varchar(10) on a varchar(5) (or numeric(10,2) to numeric(12,2), integer to smallint, text to varchar(20)) showed in DIFF as an alter table with no column named in columns, and DIFF ... AS SQL left the change out, so running it elsewhere kept the old size. Both compare the whole type now, size included: v: varchar(5) -> varchar(10), and alter table t alter column v type varchar(10);. Found by BranchBench, whose schema diffs read columns.
  • Attach: rows Postgres triggers write to other tables come into main with the merge. A trigger on leads that keeps accounts.lead_count, writes an audit row, or deletes a lead's notes (and a foreign key's ON DELETE CASCADE) left those rows as main had them until the next refresh, and a world that changed one of them meanwhile got a conflict. The write-back now counts what its transaction wrote per attached table (deferred triggers included), and the tables written besides the merge's own rows are refreshed alone right after it; after a crash, recovery does that refresh.
  • REINDEX DATABASE rebuilds the database's indexes. It reported success and rebuilt nothing (it was read as REINDEX SYSTEM). It now rebuilds every index of the tables and materialized views you own, moves them with REINDEX (TABLESPACE t) DATABASE, and, as in Postgres, can't run inside a transaction block. REINDEX SYSTEM still rebuilds only the catalogs'.
  • Temporary tables follow temp_tablespaces. A temporary table or index made without TABLESPACE went where default_tablespace said, so a CREATE INDEX on one failed with 42501 when that named a tablespace you can't create in. It now goes to the first tablespace of temp_tablespaces you may create in, else the database's default, as in Postgres. CREATE TEMP TABLE ... TABLESPACE t works (it said tablespace "t" does not exist), and DROP TABLESPACE refuses one that holds the session's temporary tables or indexes.
  • BY COLUMNS USING OURS (or THEIRS) settles a row both sides changed column by column. The side named wins only the columns both changed; a column only the other side changed keeps its value. It used to take the named side's whole row, putting back the fork's value in columns only the other side changed (an email main changed went back to what it was). The same goes for USING OURS|THEIRS into a table set to merge_by_columns. USING OURS without BY COLUMNS still takes whole rows. Under a merge policy, only a column ours actually overwrites counts as an overwrite.
  • MERGE ... DRY RUN says what each side changed, by column. A new columns column (HTTP: columns) gives {here, there, both} for each row both sides changed, and a conflict's detail says what USING OURS / THEIRS would discard against BY COLUMNS with that side (both changed age; only here: name; only the parent: email. USING OURS overwrites the parent's age, email, BY COLUMNS USING OURS only age; ...). Settled rows explain themselves the same way, instead of both sides changed this row.
  • SHOW MERGE OUTCOMES counts a merge's conflict with Postgres as a conflict, not a stale read. In an attached folder, a merge refused because Postgres changed its rows was counted under stale_reads, and so were refusals because Postgres was busy (a lock timeout) or a world's scope changed meanwhile. Postgres conflicts count as conflicts now, and the others aren't counted.
  • Attach: a refresh takes columns added in Postgres. A migration that adds a column (ALTER TABLE leads ADD COLUMN status text NOT NULL DEFAULT 'new') no longer stops a table's refresh and every merge into it until you attach again: the next chronos refresh (or serve --refresh-every, or HTTP refresh) adds the column to main's table with Postgres's type and default, then main takes its values, and says so (added). Merges into the table are refused only until that refresh. A world forked before the column existed leaves Postgres's values in it alone for rows it changes, even settled USING OURS. Dropped, renamed or retyped columns still need a new attach.
  • Attach: a tenant's read of a big table no longer holds every merge. A world that checks its reads and ran select ... where org_id = 42 and status = 'open' on an attached table of more than 100,000 rows was always held for a person ("read leads more broadly than can be checked against Postgres"), as the check read the table whole from Postgres. The check now asks Postgres only for the rows the scan's conditions can match, at the fork and now, so the app writing other tenants' rows doesn't hold it, and a matching row changed or added there still does. A scan through an index keeps its conditions as a scan without one does, rather than the index entries it read, inside Chronos too: a row its other conditions rule out (status = 'won') no longer holds it when it changes.
  • Attach: a row a world inserts after Postgres added columns gets Postgres's defaults. A world forked before a column was add...
Read more

Chronos DB 0.4.1

Choose a tag to compare

@Abhishekxdg Abhishekxdg released this 23 Oct 07:59

Fixes from the first run of chronos init against Chronos Cloud, attach that keeps up with Postgres, and agents held to one tenant.

Changed from 0.4: EXPLAIN (COSTS OFF) shows how Chronos runs a statement by default. 0.2 to 0.4 showed the plan PostgreSQL 16 would choose for simple statements (one table read, partitions pruned), which Chronos may not run that way: it can answer an equality from its search index where that plan has a Filter, and read every partition where it shows them pruned. Now that plan is shown only when asked for, SET chronos.explain = postgres, with a NOTICE that Chronos may run the query differently; chronos (the default) always shows Chronos's steps, and any other value is 22023. SHOW chronos.explain, SHOW ALL and pg_settings list it. See EXPLAIN.

  • chronos init and chronos cloud login keep waiting through a slow network. While you approve the sign-in code, the CLI asks the API every few seconds whether you have; one ask that got no answer within 30 seconds ended the command (Resource temporarily unavailable (os error 35) on macOS). It tries again now, until the code expires, and a timeout says no answer within 30 seconds. The help no longer says the sign-in is GitHub's: it's a code you approve at chronos.2090.tech/device, signed in with an email code or GitHub.
  • World names with a dash work unquoted in SQL. MERGE BRANCH add-todos and DIFF task-1 read add-todos and task-1 as names, where they used to fail with world names with a dash need quotes; agents fork such names and write them as they are. A dash with nothing after it (task-) still says how to quote it.
  • diff puts a new table in words. The shell's diff and MCP's diff showed a table made in a world as its catalog row (+ _sonos_tables/todos {"checks":[],"columns":[...]}); they show + create table todos (id integer not null, title text not null) now, views, functions and the like as DIFF describes them, and a type or domain made or dropped in a world with its definition (+ type mood ...), where it used to be left out. The rows behind constraints, indexes and sequences are left out.
  • Attach: refresh. chronos refresh <folder>, chronos serve --refresh-every 5m and HTTP refresh bring main up to Postgres: rows changed, added or removed there straight since the attach (or the last refresh) come into main in one change, compared as the write-back compares them, under its lock. A world that changed one of those rows too settles it at its merge as any conflict. A table whose shape changed in Postgres is left as it was, and named. HTTP attached says what a folder is attached to and how old its untouched rows may be. Before, main stayed as old as the attach until you attached again into a new folder.
  • Attach: undo a merge that reached Postgres. UNDO MERGE of a merge into main of an attached folder was refused. It now puts the rows back through Postgres, as a merge of a world of main: every row locked and compared there, a row changed since a conflict, nothing written.
  • Attach: merges up to 50,000 rows by default, sent 5,000 at a time in one transaction (was 10,000, in one statement per table); CHRONOS_ATTACH_MAX_ROWS raises it to 1,000,000.
  • Agents held to one tenant, and tokens that expire. CREATE AGENT run_42 WITH (tenant = 'org_id = 42', valid_for = '1 hour') (HTTP create_agent with tenant and valid_for): the agent reads and changes, in every world, only rows of tables with that column holding that value, whatever its role or the table's own row security; every world it forks is declared that tenant's; it sees changes as rows only of its own worlds (DIFF, dry runs, CHECK READS, history; SHOW REVIEWS, SHOW AUDIT refused); its token stops working after valid_for. SHOW AGENTS has tenant and expires columns. For a product whose agents act for one customer at a time.
  • Postgres compatibility: two more batches of the PostgreSQL 16 regression suite. Tablespaces, composite types in ALTER TYPE, views over joins, DDL locks, type-name casts, EXPLAIN and SET in PL/pgSQL, and table row types as function parameters (89.12% of expected lines, 60 files exact).

The macOS builds need macOS 13 or later, as for 0.2.0.

Every change in detail: https://github.com/Abhishekxdg/chronosdb/blob/main/CHANGELOG.md