Repository navigation
dendro v0.6.0
This is 0.6.0, not 0.5.2. Archives this release creates are schema version
5, which releases up to 0.5.1 refuse (unsupported archive schema version 5). A patch release would reach dendro = "0.5" callers on a cargo update, and a process on the new build would then write files that their
other processes, still on the old one, cannot open: the same break-by-upgrade
the minor exists to prevent (see 0.2.0 below). Error gains variants, which
#[non_exhaustive] allows, and Error::UnsupportedSchema now reports
reads: 4.
Added
- Archive-level documents: named JSON documents that belong to the archive
rather than to a source, in a newdocumentstable (FORMAT.md §3.6). A
document is(kind, name)with a caller-chosenschemastring naming its
body's type and version. Writes append a revision and never overwrite one;
the current revision is the newest, and a delete is a revision with no
body. Written withArchiveMut::put_document,put_document_if_current
(which takes the uuid of the revision the caller read, and is refused as
the newError::DocumentConflictwhen another revision is current) and
delete_document, which return the new revision'sarchive::RevisionId
(id and uuid), so conditional saves chain without a read between them;
the same three onTransaction, and onWriterfor an archive being
recorded, which anArchiveMutcannot open. Read with
Archive::document,documents,document_kindsanddocument_history
(typesarchive::{Document, Deletion, Revision}). A body must be JSON
(RFC 8259, no nesting limit). Kinds beginningdendro.
(archive::RESERVED_KIND_PREFIX) are refused. rewrite::copy_documents_into, which copies every document revision of
one archive into a destination transaction, skipping revisions the
destination already holds (by uuid) and failing on one it holds with
different contents. It returnsarchive::CopiedDocuments: the revisions
inserted, the documents whose current revision came from this input, and
which of those displaced a revision they do not supersede (forked, each
anarchive::Forkwith the displaced and current revisions' uuids), for
a combine that must choose or rename.Archive::schema_version, andError::SchemaTooOld, returned for a
document written into a v4 archive.
Changed
- Breaking (format): the schema version is 5 (was 4). Every archive this
release creates has thedocumentstable and is stamped 5; this release
reads 4 and 5. A v4 archive reads as holding no documents, and is still
written in place as v4 (a writer resumes it, metadata edits and compaction
run), but no document is written into it: copy it forward with
rewrite::copy_sources_intointo a new archive, which is v5. The reason
is in FORMAT.md §8: 0.5.1's copier drops a table it does not know, and
documents are data people wrote. - Breaking (format): on schema 5 the caller's per-segment index is a
row of a newsegment_indexestable, keyed like its segment, instead of
thesegments.caller_indexcolumn (FORMAT.md §3.2). SQLite stores a
row's columns in order and spills a large BLOB onto overflow pages, so an
index after the segment bytes was reachable only through them: reading
every index of a cold archive cost as much I/O as reading the segments.
Archive::read_segment_indexesnow reads no segment row on v5. The table
cascades on segment delete, so retention, compaction and
Transaction::delete_segmentremove an index with its segment. A v4
archive keeps its indexes in the column, is written there when written in
place, and is read through the same API; a copy into a new archive moves
them to the table. The Rust API is unchanged:SegmentRow::caller_index
andinsert_segment_with_indexcarry the index as before. - Breaking:
SegmentEncoder::encodeandencode_aftertake
&segment::EncodeContextin place of the stream name, and
segment::materializetakes one too. The context carries the stream
(ctx.stream), the source's row id in the archive (ctx.source_id) and
its uuid (ctx.source_uuid). Before, an encoder was told only the stream
name, which one writer with several sources (two hosts recordingcpu)
repeats, so an encoder keeping per-stream state could not key it. The
writer, a reader materializing a tail and a copy all pass the source the
rows are in; a copy passes the source archive's, not the destination's.
The struct is#[non_exhaustive], so fields are added without a break.
For an implementor: replacestream: &strwithctx: &EncodeContext<'_>
and readctx.stream; calls an implementation makes to its ownencode
orencode_after(anencode_afterthat falls back toencode, say)
passctxinstead of the stream too. metriken-storage needed about eight
lines in three files. A caller driving an encoder itself (a test) builds
one withEncodeContext::new(source_id, stream). Key state that outlives
one archive bysource_uuid: a copy renumberssource_id. rewrite::copy_sources_intoalso copies the source archive's documents,
every revision, whatever theCopySpecselects. Called once per input,
as a combine does, it carries each input's documents; call
copy_documents_intofirst for the report.