open dict

Open dict & open account — cross-platform interchange (spike, #25)

A design note for the portable interchange format that lets dictionaries, the VOC/MD, and whole accounts survive crossing between MultiValue platforms through git — so an account committed on MVX clones/rebuilds correctly on UniData (and back). It complements the settled Open account format in DECISIONS.md (the git-boundary translation, the mvx.openaccount flag); this note pins down the two schemas that translation moves through: open dict and the .mv-account open-account descriptor. The mvx↔open round-trip has a working prototype in tests/opendict.b.

The problem

MV dictionaries mean the same thing everywhere — a field's name, source attribute, type, conversion, format, heading, association role — but each platform encodes them differently (mvx's 6-attribute D-item, UniData/UniVerse D/I/PH items with a different attribute order and extra control fields). A raw dictionary committed in one platform's form doesn't load correctly in another. The same holds for the VOC/MD (the verb/pointer catalog) and for an account's file list. The fix is a canonical, self-describing serialization of the semantic content that each platform converts to and from.

Open dict

An open-dict item is a legible canonical MV record (the record id is the field name). It keeps mvx's native attribute order for 1–6 — mvx is the reference dialect — and appends the derived cross-platform semantics a target needs:

#	field	notes
1	type	D data field · I derived/computed · PH phrase
2	source	attribute number (D), the expression (I), or phrase text (PH)
3	conv	conversion code (MD2, D2/, MT, …) — largely shared across dialects
4	heading	column heading / display name
5	format	display format (12L, 8R)
6	assoc	association name (empty = unassociated)
7	single/multi	S single-valued · M multi-valued — derived
8	role	C controlling · D dependent · empty — derived

Fields 7–8 are derived on export and advisory for the importer: M when the field is in an association; role = C for the association's controlling member (the one with the lowest source attribute number) else D. Import needs only 1–6, so native → open → native round-trips exactly. Records serialize with the record-git legible hybrid (marks ↔ newlines), so they diff on GitHub.

mvx ⇄ open

Near-identity: mvx's D-item 1=D/I 2=attr#/expr 3=conv 4=heading 5=format 6=assoc is open fields 1–6. Export copies 1–6 and computes 7–8 from the association context; import drops 7–8 and writes 1–6 back. Lossless — the prototype asserts it over a dictionary that includes an association (INV: QTY controlling, PRICE/DESC dependent) and a derived I item.

UniData ⇄ open (sketch — where the real translation lives)

UniData dictionaries carry the same semantics in a different shape (a D item's attributes are location / conversion / name / format / single-multi / assoc, with I/V for derived and PH for phrases). udt-git's converter maps UniData's attribute order and control fields onto the canonical 1–8 above and back. Divergences to resolve there (not on MVX): conversion-code spelling differences, association encoding (UniData names controlling/dependent explicitly via S;assoc — open field 8 captures it), and I-type expression syntax. Because open dict is dialect-neutral, adding a field in UniData → committing → cloning in mvx yields a correct mvx dictionary, and vice versa.

Open VOC/MD

The VOC/MD splits into portable and local entries:

• Portable (travel in open form): F/Q file pointers (attr 1 the abstract class DIR/hash, attr 2 data, attr 3 dictionary — see CREATE-FILE), PH phrases, and verb/keyword definitions (name, kind, sentence grammar).
• Local (rebuilt on checkout, never committed): executable paths and cataloged binaries. BUILD re-catalogs these from the account's source after the portable catalog is in place.

Open account — .mv-account

The account-level analogue: a portable descriptor at the account root that both describes the account and drives the conversion. It is the git-side counterpart of the on-disk native .mvx (name/version marker); on checkout .mv-account rebuilds .mvx, and UniData — which has no on-disk descriptor — reads it transiently to build the account and synthesises it on commit. A legible key = value descriptor with file lines (same family as .mvx):

# .mv-account — open (portable) account descriptor
name        = demo
version     = 1
openaccount = 1                 # conversion policy: store dict/VOC in open form
hash        = lmdb              # the account's default hash backend (checkout maps `hash` -> this)
description = MVX demo account

# file <logical-name> <kind: hash|DIR> [dict:<dict-ref>]
file VOC       hash dict:VOC
file BP        DIR  dict:BP
file CUSTOMERS hash dict:CUSTOMERS

• Identity — name, version, description.
• Files — each file's logical name, abstract kind (hash/DIR — the semantic behind today's %FILE%), and its dictionary reference; never the platform storage details (LMDB paths, driver names — those are local BINDINGS).
• Conversion — openaccount = <version> (the account-level home for the open-form policy) and hash = (the default backend a hash file maps to on checkout; the prompt is per-platform, see DECISIONS.md).

Replaces vs complements .mvx. .mv-account is the portable superset; on MVX the on-disk descriptor stays .mvx (native, minimal) and .mv-account is the committed open form the engine maps to/from — two files, each with one job (detection vs interchange). They share the key = value grammar so the mapping is mechanical.

Rebuild ordering

On checkout the master dictionary is rebuilt first (#59): the VOC/MD F pointers and the account's dictionaries must exist before data files are created and indexes rebuilt (%INDEXES% → CREATE-INDEX). Then BUILD re-catalogs the local verb binaries.

Hook points

• Record-git engine (mv_git) — on commit, translate native dict/VOC/%FILE% → open (guided by .mv-account); on checkout, build the native account directly from open blobs, master dictionary first; status/diff translate on-disk native up to open before comparing.
• %FILE% normalization — lmdb/any hash backend → hash, dir → DIR in git; reverse on checkout maps hash → the account's hash = default and DIR → the directory driver.
• Conversion policy — the mvx.openaccount git-config flag (→ $MVX_OPENACCOUNT) gates all of the above; without it the engine stores the platform's own legible form (current behaviour).

Follow-up implementation issues

1. Open-dict converter as a reusable subroutine (this note's prototype → mv_git-callable), both directions, over a whole .DICT.
2. %FILE% DIR/hash normalization at the git boundary + the hash = default on checkout (#73 foundation).
3. .mvx ⇄ .mv-account conversion. Done (mvx#73): a real conversion, not a rename — the portable form carries identity + openaccount/hash and drops MVX-local permit/deny policy (re-seeded locally, never in git); checkout rebuilds a minimal native .mvx; status/diff compare in open-space so a local policy edit is invisible to git. One schema, shared with udt-git via mv_git_desc_open.
4. Open VOC/MD translation (portable entries) + master-dictionary-first rebuild.
5. udt-git: the UniData ⇄ open converter.