chore(level): migrate from leveldown to the abstract-level family#119
Merged
Conversation
Adds classic-level, memory-level and abstract-level alongside the existing legacy Level stack so the migration can proceed file by file with the test suite staying green. No code switched over yet. See docs/level-migration.md for the full plan.
Broadens test/shared/level/wkb-test.js from a single LineString case to every geometry type wkx round-trips (Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, GeometryCollection) plus batch/getMany, iterator and del. Runs against the current leveldown/encoding-down stack as the safety net for the migration. The wkb.js encoding-format conversion moves into phase 2: its exported encoding object is consumed by index.js, and the old encoding-down shape and the new level-transcoder shape are incompatible, so it cannot be switched independently of its consumer. Plan updated accordingly.
Two couplings the initial plan underestimated:
- The leveldb({ down }) factory path is used by Store.js (PartitionDOWN)
and ipc-test.js (IPCDownClient), coupling phase 2 to phases 3/4.
levelup is kept as a temporary bridge for that path.
- abstract-level replaced the put/del/batch events with a single write
event; PreferencesStore, SearchIndex and SpatialIndex listen directly
on the db objects and need an explicit migration (new phase 2b).
WIP checkpoint on the abstract-level migration branch.
- index.js: leveldb() factory now builds classic-level / memory-level
databases and abstract-level sublevels; stream readers reimplemented on
async iterators; get() handles the undefined-on-missing semantics.
- wkb.js: WKB value encoding ported to the abstract-level / level-transcoder
shape ({ name, format, encode, decode }).
- PreferencesStore / SearchIndex / SpatialIndex: migrated from the
put/del/batch events to abstract-level's single write event.
- wkb-test: del case asserts the new undefined-on-missing get semantics.
Known failing (4): PartitionDOWN and IPCServer tests. These custom stores
still use the abstract-leveldown callback iterator API, incompatible with
their now abstract-level child databases. They are migrated in phases 3/4,
which must land together with this change to reach a green suite — see
docs/level-migration.md.
Reimplements the two custom abstract-leveldown stores as plain adapter
objects over abstract-level, exposing get/getMany/put/del/batch/iterator/
keys/values — the subset the L.* helpers and Store use. Neither was ever
a real database, so subclassing AbstractLevel (and its encoding contract)
is unnecessary.
- PartitionDOWN: routes a value's geometry to wkbDB and its remaining
properties to jsonDB; the iterator merges both ascending child
iterators by key. Null/undefined argument validation preserved.
- ipc.js: IPCDownClient is a plain IPC-proxy adapter; IPCServer's
ITERATOR handler uses db.iterator() instead of createReadStream.
- index.js: the leveldb({ down }) factory path and the levelup bridge
are removed.
- Store.js: this.db is now `new PartitionDOWN(jsonDB, wkbDB)` directly.
- Tests rewritten against the new API; the previous suite relied on
try/catch blocks that passed vacuously when no error was thrown.
Full suite green (328 passing).
- ProjectList-services.js: the command-queue store no longer builds on levelup/memdown/subleveldown directly; it uses the L.leveldb factory. - transfer.js / preload.js: comments updated to abstract-level / classic-level. - Removed the legacy Level stack from package.json: leveldown, levelup, subleveldown, abstract-leveldown (and transitively memdown, encoding-down, deferred-leveldown). The migration to the abstract-level family is complete. Full suite green (328 passing), npm audit clean.
The legacy library names no longer reflect what the code is: - PartitionDOWN -> PartitionStore (class + file): it is a plain value router, not an abstract-leveldown "down" implementation. - IPCDownClient -> IPCClient: an IPC proxy, pairs with IPCServer. - leveldb() factory option `up:` -> `parent:` (the parent database a sublevel is created on); the `down:` option was already removed. - Stale comments referring to the removed levelup bridge and the old leveldown/encoding-down stack updated. Pure rename, no behaviour change. Suite green (328 passing).
The renderer and main webpack configs externalized `leveldown` so the
native module is required at runtime rather than bundled — node-gyp-build
must resolve the prebuilt binary relative to node_modules, not dist/. The
level migration swapped leveldown for classic-level but left these
ExternalsPlugin entries pointing at the removed package, so classic-level
got bundled and failed at load with "No native build was found".
Both ExternalsPlugin entries now list `classic-level`. memory-level and
abstract-level are pure JS and do not need externalizing.
Verified: dist/main.js now contains `require("classic-level")`; the app
launches without the native-build error.
SessionStore listened on sessionDB via the put/del events, which
abstract-level removed in favour of a single write event. Missed in the
phase 2b event-model migration; abstract-level threw a ModuleError when
the listeners were registered.
Now consumes the write event and re-emits its own put/del to consumers.
SessionStore was the last abstract-level event consumer on the old API;
the remaining store.on('batch') listeners are on ODIN's Store emitter,
not an abstract-level database.
matrix-client-api 3.0.0 moves its command queue to the abstract-level API. ODIN's level migration needs that version — the 2.x queue requires a levelup-compatible database, which this branch removed. Completes the abstract-level migration: collaboration (the Matrix command-queue sublevel path) verified working against 3.0.0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the legacy Level stack (
leveldown,levelup,memdown,subleveldown,encoding-down,abstract-leveldown,deferred-leveldown— all superseded and unmaintained) with the actively maintainedabstract-levelfamily (classic-level,memory-level,abstract-level).The on-disk LevelDB format is unchanged and
abstract-levelsublevels are key-compatible withsubleveldown, so no data migration is required.Full plan and rationale:
docs/level-migration.md.Changes
shared/level/index.js):leveldb()buildsclassic-level/memory-leveldatabases andabstract-levelsublevels; stream readers reimplemented on async iterators;get()handles abstract-level's undefined-on-missing semantics. ExportedL.*API kept stable, so the ~21 helper consumers are untouched.shared/level/wkb.js): ported to the abstract-level / level-transcoder encoding shape.PartitionDOWN→PartitionStoreandIPCDownClient→IPCClient, reimplemented as plain adapter objects over abstract-level.leveldown/levelupnaming artefacts removed.PreferencesStore,SearchIndex,SpatialIndex,SessionStoremigrated from the removedput/del/batchevents to abstract-level's singlewriteevent.classic-level(native) externalized in place ofleveldown.@syncpoint/matrix-client-api^3.0.0, whose command queue was moved to the abstract-level API (Use abstract-level API for the command queue matrix-client-api#16).wkb,PartitionStoreandipcsuites rewritten against the new API.Validation
npm run lintclean · 328 tests passing ·npm audit0 findingselectron-builder install-app-depsrebuildsclassic-levelfor Electron 42.2.0command-queuesublevel path, against matrix-client-api 3.0.0).Merge order
matrix-client-api 3.0.0 is published; the dependency bump is included here. This PR is ready to merge.