Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop support of memdb #83

Closed
vweevers opened this issue Oct 13, 2019 · 0 comments · Fixed by #89
Closed

Drop support of memdb #83

vweevers opened this issue Oct 13, 2019 · 0 comments · Fixed by #89
Labels
semver-major Changes that break backward compatibility

Comments

@vweevers
Copy link
Member

In favor of the newer level-mem. I already planned to drop memdb; I'm writing this because @bcomnes asked for some context in bcomnes/level-hookdown#29 (comment).

Warning: this is a long story. You can skip it if you accept "memdb is old" as a good enough reason 😉

When we moved subleveldown to the Level org and updated it to work with latest Level modules, we did not account for memdb (as level-mem was already the preferred module). It's pure luck that wrapping memdb with subleveldown still works today. I recently added an integration test (#75) to avoid breaking it (after seeing that memdb is still depended upon). However, we're now at a point where continued support is just madness.

A typical db passed to subleveldown consist of several "layers". E.g. leveldown wrapped in encoding-down wrapped in deferred-leveldown wrapped in levelup. Behind the scenes, subleveldown unwraps that db to get to leveldown, then wraps it in a layer that prefixes keys, then (re)wraps that with encoding-down and levelup. For more background, see Level/community#82.

The problem with memdb is that it depends on an old version of deferred-leveldown (< 2.0.0) which when closed, doesn't expose its underlying db. So subleveldown can only unwrap after opening.

This limitation means we can't add a manifest (Level/community#83), which allows to inspect what features a db supports via db.supports. For example, db.supports.permanence indicates whether the underlying storage is persistent. Would be true for leveldown, false for memdown. For subleveldown to determine that, it must inspect the manifest of the unwrapped db, and this must happen synchronously, possibly before the db is open. Then you would be able to do:

var sub = require('subleveldown')(db)

if (!sub.supports.permanence) {
  throw new Error('Persistent storage is required')
}

If we drop support of memdb, we can unwrap synchronously and expose a manifest.

Manifests also make the long-term merging of levelup with abstract-leveldown (Level/community#58) easier. They allow us to determine which features a db already has and skip adding those. Because at some point, db's will be a mix of level(up) and abstract-leveldown features. Eventually, the unwrapping in subleveldown will also become simpler (thanks to needing less layers). Full circle :)

@vweevers vweevers added the semver-major Changes that break backward compatibility label Oct 13, 2019
@vweevers vweevers added this to Backlog in Level (old board) via automation Oct 13, 2019
@vweevers vweevers moved this from Backlog to To Do in Level (old board) Apr 3, 2020
@vweevers vweevers moved this from To Do to In progress in Level (old board) Apr 4, 2020
vweevers added a commit that referenced this issue Apr 4, 2020
DeferredOpen means that the db opens itself and defers operations
until it's open. Currently that's only supported by level(up) and
friends. Before, subleveldown would also accept abstract-leveldown
db's that were not wrapped in levelup.

Opening and closing a sublevel no longer opens or closes the parent
db. The sublevel does wait for the parent to open (which in the
case of levelup already happens automatically) but never initiates
a state change.

Drops support of old modules:

- memdb (use level-mem instead)
- deferred-leveldown < 2.0.0 (and thus levelup < 2.0.0)
- abstract-leveldown < 2.4.0

Closes #84, #83 and #60.
vweevers added a commit that referenced this issue Apr 4, 2020
DeferredOpen means that the db opens itself and defers operations
until it's open. Currently that's only supported by level(up) and
friends. Before, subleveldown would also accept abstract-leveldown
db's that were not wrapped in levelup.

Opening and closing a sublevel no longer opens or closes the parent
db. The sublevel does wait for the parent to open (which in the
case of levelup already happens automatically) but never initiates
a state change.

Drops support of old modules:

- memdb (use level-mem instead)
- deferred-leveldown < 2.0.0 (and thus levelup < 2.0.0)
- abstract-leveldown < 2.4.0

Closes #84, #83 and #60.
@vweevers vweevers moved this from In progress to Review in Level (old board) Apr 4, 2020
vweevers added a commit that referenced this issue Apr 5, 2020
DeferredOpen means that the db opens itself and defers operations
until it's open. Currently that's only supported by levelup (and
levelup factories like level). Previously, subleveldown would also
accept abstract-leveldown db's that were not wrapped in levelup.

Opening and closing a sublevel no longer opens or closes the parent
db. The sublevel does wait for the parent to open (which in the
case of levelup already happens automatically) but never initiates
a state change.

If one closes the parent but not the sublevel, subsequent
operations (like get and put) on the sublevel will yield an error,
to prevent segmentation faults from underlying stores.

Drops support of old modules:

- memdb (use level-mem instead)
- deferred-leveldown < 2.0.0 (and thus levelup < 2.0.0)
- abstract-leveldown < 2.4.0

Closes #84, #83 and #60.
Level (old board) automation moved this from Review to Done Apr 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver-major Changes that break backward compatibility
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

1 participant