Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support sublevel and plugin extensions #8

Merged
merged 29 commits into from Apr 21, 2013

Conversation

Projects
None yet
2 participants
@dominictarr
Copy link
Collaborator

dominictarr commented Apr 13, 2013

This pull request allows multilevel to support sublevels, and plugins.

You just have to pass a manifest from the server to the client multilevel:

var db = levelup(path)
sublevel(db)

//setup db...

var foo = MapReduce(db, 'map',...) //etc

fs.writeFileSync(JSON.stringify(manifest(db, true)), './manifest.json'))

//later...

stream.pipe(multilevel.server(db)).pipe(stream)

and then the client browserifys that in.

stream.pipe(multilevel.client(db, require('./manifest.json'))).pipe(stream)

If you don't provide a manifest you will still be able to use the normal methods on the base level.

This will work great if you have browserify setup to generate lazily,
on the first request, as that will mean the manifest.json is surely generated before browserify is run.

I'm making this pull request as a work in progress, I've disabled events,
and the tests that need them. Am also considering some way to add functions to the manifest, for cases where you need some piece of code to run on the front and the back-end. this would be very handy for scuttlebutts.

@juliangruber

This comment has been minimized.

Copy link

juliangruber commented on lib/client.js in 230ea78 Apr 14, 2013

this hides mdm's methods, we don't want to expose them

@dominictarr

This comment has been minimized.

Copy link
Collaborator Author

dominictarr commented Apr 14, 2013

Writing this has also given me several ideas for a better mux-demux,
(possibly using redis-protocol instead of json - or maybe we just need a snappy.js implementation?
also, we probably need to use an mtu and exponential back off, so the client doesn't drown in data)

Also, maybe, we can add an event emitter syntax to the manifest, where you can declare what events should be exposed (for security)

Another thing that I am considering is some way to (mostly) transparently handle reconnections,

Streams can be reconnected (for a live stream, it can buffer a few recent updates - else give the client the whole stream again, and for a non live stream, reconnect since the last update received)

I've already figured that stuff for scuttlebutts.

@juliangruber

This comment has been minimized.

Copy link

juliangruber commented on test/util.js in 040c682 Apr 15, 2013

why doesn't .. work?

@dominictarr

This comment has been minimized.

Copy link
Owner Author

dominictarr commented on 040c682 Apr 15, 2013

breaks in browserify... at least it did... maybe it's fixed now? I got into the habit of doing that so I could run my tests on ci.testling.com

This comment has been minimized.

Copy link

juliangruber replied Apr 15, 2013

ok I see

@juliangruber

This comment has been minimized.

Copy link

juliangruber commented on test/nested-batch.js in 40dff89 Apr 21, 2013

why's that necessary?

This comment has been minimized.

Copy link
Owner Author

dominictarr replied Apr 21, 2013

it's possible to do a batch across multiple sublevels,
notice that it's db.sublevel('foo').batch([{.... prefix: db.sublevel('bar').prefix()}])

this is an essential feature, necessary to for things like triggers, and indexes.

This comment has been minimized.

Copy link

juliangruber replied Apr 21, 2013

what about

{key:'b', value:'2', type: 'put', sublevel: db.sublevel('bar')}

Less leaky IMO

This comment has been minimized.

Copy link
Owner Author

dominictarr replied Apr 22, 2013

Needs to be a string so that it can be sent over the wire...
Am contemplating adding code to the manifest, (manifest.js instead).
Which could be used to handle cases like this...

Which can handle things like this, but making this compromise got it out the door.

@juliangruber

This comment has been minimized.

Copy link

juliangruber commented on lib/server.js in 46a3d89 Apr 21, 2013

w00ps

juliangruber added a commit that referenced this pull request Apr 21, 2013

Merge pull request #8 from dominictarr/master
support sublevel and plugin extensions

@juliangruber juliangruber merged commit a7cba5f into juliangruber:master Apr 21, 2013

1 check passed

default The Travis build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.