Follow-up to #78 / #79 (which shipped the content half as dol.content). This issue tracks the record/metadata half of the content–metadata bifurcation: a queryable facade for the small rows that sit next to the blobs.
Motivation
dol.content gives a flat blob store + ContentRef. The other half of the pattern is a collection/record facade with CRUD + query, so an app can list/filter/paginate records whose large fields are offloaded to a content store (holding a ContentRef in their place). Our TypeScript side (zodal) already models exactly this as DataProvider<T>, and a Python mirror keeps the two ecosystems contract-compatible.
Proposed scope
Mirror zodal's @zodal/store contract in Python:
DataProvider-style facade — get_list(params) -> {data, total}, get_one(id), create, update, update_many, delete, delete_many (+ optional upsert). Structured sort/filter/search/pagination params, with a shared client-side filter_to_function fallback for backends that can't push predicates down.- Capability discovery — a
ProviderCapabilities-style object so callers degrade gracefully (which ops / which pushdowns a backend supports).
- Bifurcated composition — a
bifurcated_provider(metadata_provider, content_provider, content_fields) that splits records on write (blobs → dol.content, rest → the metadata store), replaces content fields with ContentRefs on read, and runs the metadata-is-commit-point saga (delete content-first; compensate metadata on content-write failure).
wrap_provider codec layer (the analog of wrap_kvs; zodal's wrapProvider already cites dol).
Placement (per the zero-dep rule)
- The abstract contract + in-memory/
dict reference provider + bifurcated composition are pure stdlib → candidate for dol (or xdol if any helper needs a dep).
- Concrete queryable backends (SQL, Supabase, etc.) carry deps → their own packages, never core
dol.
Reference
- zodal contract:
zodal/packages/store/src/{data-provider,capabilities,bifurcated-provider,wrap-provider}.ts
- design corpus:
zodal/docs/research/ + misc/docs/dol_content_metadata_bifurcation.md
Deferred from the phase-1 decision ("blob now, DataProvider later").
Follow-up to #78 / #79 (which shipped the content half as
dol.content). This issue tracks the record/metadata half of the content–metadata bifurcation: a queryable facade for the small rows that sit next to the blobs.Motivation
dol.contentgives a flat blob store +ContentRef. The other half of the pattern is a collection/record facade with CRUD + query, so an app can list/filter/paginate records whose large fields are offloaded to a content store (holding aContentRefin their place). Our TypeScript side (zodal) already models exactly this asDataProvider<T>, and a Python mirror keeps the two ecosystems contract-compatible.Proposed scope
Mirror zodal's
@zodal/storecontract in Python:DataProvider-style facade —get_list(params) -> {data, total},get_one(id),create,update,update_many,delete,delete_many(+ optionalupsert). Structuredsort/filter/search/paginationparams, with a shared client-sidefilter_to_functionfallback for backends that can't push predicates down.ProviderCapabilities-style object so callers degrade gracefully (which ops / which pushdowns a backend supports).bifurcated_provider(metadata_provider, content_provider, content_fields)that splits records on write (blobs →dol.content, rest → the metadata store), replaces content fields withContentRefs on read, and runs the metadata-is-commit-point saga (delete content-first; compensate metadata on content-write failure).wrap_providercodec layer (the analog ofwrap_kvs; zodal'swrapProvideralready cites dol).Placement (per the zero-dep rule)
dictreference provider + bifurcated composition are pure stdlib → candidate fordol(orxdolif any helper needs a dep).dol.Reference
zodal/packages/store/src/{data-provider,capabilities,bifurcated-provider,wrap-provider}.tszodal/docs/research/+misc/docs/dol_content_metadata_bifurcation.mdDeferred from the phase-1 decision ("blob now, DataProvider later").