JavaScript Driver #256

vrde · 2016-05-09T09:38:22Z

Abstract

BigchainDB misses an official implementation for the JavaScript Driver.
Release 0.3 introduces cryptoconditions and a new format for creating and signing transactions. Since this won't change soon, it's a good time to develop a JS Driver.

Goals

The goal is to have a package developers can use both for client (browser) and server (Node.js) applications.
This should be feasible (and have acceptable performance in a browser) since the JavaScript implementation of cryptoconditions uses TweetNaCl, a self-contained, small crypto library that can run in the browser.

This project follows the ascribe JavaScript Style Guide and will be stored in the BigchainDB repository, under the drivers/javascript directory.

Challenges

When running the Driver from the browser, we must consider the following challenges:

store the private key of the identity in a safe place(might be generated on then fly given a passphrase? Should we store it in localstorage?)
enable CORS in the API Server, to allow cross-origin access

Architecture

While most of the code will be shared between the browser and the Node.js implementation, there will be few platform specific functions, in particular the one to load keys.

Examples

Here are some examples on how the interface will look like.

/**
 * In this scenario the two actors Alice and Bob are managed
 * in the code, or "shared memory", but they use the BigchainDB
 * Federation to communicate.
 *
 * In a real application Alice and Bob will be using different devices.
 */


// Import the BigchainDB driver
const bigchaindb = require('bigchaindb');
const config = {
    nodes: [{
        endpoint: 'https://bigchaindb.example.org:9984/',
        pubkey: '<pubkey>'
    }, {
        endpoint: 'https://bigchaindb.some-other-organization.org:9984/',
        pubkey: '<pubkey>'
    }]
};


////////////////////////////////////////////////////
// Use case:                                      //
// Create and Transfer an asset to another entity //
////////////////////////////////////////////////////

// Generate a keypair, will return a mapping like:
// { pubkey: '<pubkey>', privkey: '<privkey>' }
var aliceKeypair = bigchaindb.generateKeypair();
var bobKeypair = bigchaindb.generateKeypair();

// Return a session object to create and transfer assets
var aliceSession = bigchaindb.session(aliceKeypair, config);
var bobSession = bigchaindb.session(bobKeypair, config);


// Create an asset
var txId = session.create({
    title: 'Shmui is a beautiful cat',
    type: 'photo',
    location: 'Berlin',
    uri: '/ipfs/QmYFtAk2d97znWsCLpYXYPQoj8k7eNziuTdg1UoRKe6Ymr'
});


// Transfer an asset. This will create a new cryptocondition
// that can be unlocked only by the matching private key
aliceSession.transfer(txId, bobKeypair.pubkey);

// List all the assets own by Bob. Will return the asset created
// by Alice.
bobSession.listAssets();

// List all the assets own by Bob matching a specific query.
// Will return the asset created by Alice
bobSession.listAssets({type: 'photo'});



////////////////////////////////////////////////////
// Use case:                                      //
// Create and hashlock an asset                   //
////////////////////////////////////////////////////

var carlyKeypair = bigchaindb.generateKeypair();
var carlySession = bigchaindb.session(bobKeypair, config);

// Create an asset
var txId = session.create({
    title: 'Shmui with flowers',
    type: 'photo',
    location: 'Berlin',
    uri: '/ipfs/QmVBRtxw9v56ptyKs54bqyopsZ4zerPZJYoAjVdtuppXys'
});

// Lock down the asset using a specific message
aliceSession.lock(txId, 'Shmui likes flowers');

// Trying to unlock the asset using a wrong message returns `null`
bobSession.unlock(txId, 'This is wrong');

// Unlock the asset using the right message unlocks the asset
carlySession.unlock(txId, 'Shmui likes flowers');

// Will return the last unlocked asset
carlySession.listAssets();


// Will add more use cases soon

Roadmap

Deadline May 12th TBD

There has been more discussion than expected (which is a good thing, actually). I just put TBD on the deadline, but will update it soon.

generate an ED25519 key pair
backend to load keys from the filesystem (for Node.js server applications)
CREATE operations for a single owner and TRANSFER operations between single entities
retrieval of a specific transaction id
retrieval of own assets

Later

support CREATE operations for multiple owners and TRANSFER between multiple entities
backend to load keys for browser applications (TBD)
filter transactions (global and owned)
hash locks conditions
threshold conditions

The text was updated successfully, but these errors were encountered:

ttmc · 2016-05-09T10:00:13Z

Looks like a good start. I would remove the checkboxes from the Later section so that this issue is "done" (5 out of 5) once the first five checkboxes get checked. The later things can go in future "project" issues with their own checkboxes.

vrde · 2016-05-09T10:01:38Z

@ttmc yes, that's a really good point. I'll remove the checkboxes and put a normal list, so I can copy paste it to the next project 💃

TimDaub · 2016-05-09T12:52:35Z

OK so here are a few things:

I'd separate crypto from retrieving and posting information to a server
- Why do we allow for the generation of key pairs? Isn't there already tons of libraries that let you do this? Also, when actually creating and passing this key to the bigchaindb client. What's really happening there?
- From other "blockchainesque" "ledger" drivers, there is always a concept of creating and signing. Your example implies that transfers are signed implicitly. Not sure if I like this. I always thought of a transaction as something that can be constructed on the side and then when it is ready, be signed and broadcasted to the network. I'd separate creation and signature of transaction.
- Why do I need to know a BigchainDB node's public key?
- Why can I define multiple nodes? Do I need to care to which node I send something?
What would the work-flow with require('bigchaindb') look like? Callbacks? Promises? Error handling? I assume all communication is async, right?
Invocation of db drivers. I looked at other nodejs db drivers but couldn't find a coherent pattern :/

Mongojs

var mongojs = require('mongojs')
var db = mongojs(connectionString, [collections])

node-postgres

var pg = require('pg');
pg.connect(connectionString, callback)

node-mysql

var mysql      = require('mysql');
var connection = mysql.createConnection({
...
});

connection.connect();

redis-node

var redis = require("redis"),
    client = redis.createClient();

couchdb

const NodeCouchDb = require('node-couchdb');

// node-couchdb instance with default options
const couch = new NodeCouchDb();

sohkai · 2016-05-09T14:01:01Z

It might be nice to have separate repos for different drivers, or even a drivers repo, rather than keeping it all in this repo. Just a thought.

vrde · 2016-05-09T14:15:37Z

Why do we allow for the generation of key pairs? Isn't there already tons of libraries that let you do this? Also, when actually creating and passing this key to the bigchaindb client. What's really happening there?

I want to wrap some basic features, like generating a key pair, to make it easy to create entities and exchange assets between them. If you already have a key pair, you can still use it.

From other "blockchainesque" "ledger" drivers, there is always a concept of creating and signing. Your example implies that transfers are signed implicitly. Not sure if I like this. I always thought of a transaction as something that can be constructed on the side and then when it is ready, be signed and broadcasted to the network. I'd separate creation and signature of transaction.

The idea is to keep the driver as easy as possible. Right now the few functions I'm thinking about are create and transfer, and are like actions, so they have an immediate effect. What's a use case of decoupling the creation of a transaction and the signing? The only idea that pops up in my mind is: a third party creates a transaction and a user signs it with the client (and this can be a separated function).
Now that you make me think, might be a good idea to separate the act of creating something with the act to talk to the federation.

Why do I need to know a BigchainDB node's public key?

The driver might need to verify some transactions made by the nodes in the federation. Anyway, I'm not 100% sure of this use case. @r-marques any thoughts on this?

What would the work-flow with require('bigchaindb') look like? Callbacks? Promises? Error handling? I assume all communication is async, right?

Right, it was in my head but not in the proposal 😅. Promises for sure. I'll update the proposal now (do you suggest using Q or should I rely on ECMAScript 2015 promises?)

Invocation of db drivers. I looked at other nodejs db drivers but couldn't find a coherent pattern :/

Wow, thanks a lot for the examples. In this case we don't have a wire protocol, but just http requests, so you don't really "connect" to BigchainDB. This might change in the future if we decide to have a different way to talk to the federation.

vrde · 2016-05-09T14:23:09Z

It might be nice to have separate repos for different drivers, or even a drivers repo, rather than keeping it all in this repo. Just a thought.

IMO it's a bit simpler to keep stuff in the same repo to manage issues and revisions.

vrde · 2016-05-09T15:00:06Z

@TimDaub related to promises, I was thinking something like:

var keypair = bigchaindb.generateKeypair();
var alice = bigchaindb.session(keypair, config);

alice.create({asset: 'whatever'})
    .then(response => {
        console.log(`Transaction "${response.transaction.id}" has been submitted.`);
    })
    .catch(err => {
        console.log(`There was an error while processing the transaction: "${err}"`);
    });

There is another problem anyway. The lifecycle of a transaction is:

                               .----------------------------------------------.
                               |            BigchainDB Federation             |
                               |                                              |
                               |              vote    .------------------.    |
                               |             .------->|      valid       |    |
                               |             |        '------------------'    |
                               |             |                                |
                               |             |                                |
.------------------.     push  |   .------------------.                       |
|      ready       |-------------->|    undecided     |                       |
'------------------'           |   '------------------'                       |
                               |             |                                |
                               |             |                                |
                               |             |        .------------------.    |
                               |             '------->|     invalid      |    |
                               |              vote    '------------------'    |
                               |                                              |
                               |                                              |
                               '----------------------------------------------'

Right now the promise returns when the transaction has been accepted by the Federation. We need another mechanism to notify the user when the transaction moves from undecided to valid or invalid. This can be generalized in the future using a websocket for asynchronous communications.

@r-marques any thoughts on that?

diminator · 2016-05-10T07:40:51Z

It might be nice to have separate repos for different drivers, or even a drivers repo, rather than keeping it all in this repo. Just a thought.
IMO it's a bit simpler to keep stuff in the same repo to manage issues and revisions.

In favour of having at least a drivers repo.

diminator · 2016-05-10T07:44:42Z

The goal is to have a package developers can use both for client (browser) and server (Node.js) applications.

Also have a look at the interledger project:
https://github.com/interledger/
https://github.com/interledger/specs/blob/gh-pages/0001-interledger-architecture.md

Some specifically interesting repo's
https://github.com/interledger/five-bells-wallet
https://github.com/interledger/five-bells-ledger
https://github.com/interledger/five-bells-connector

diminator · 2016-05-10T07:46:38Z

var txId = session.create({
...
aliceSession.lock(txId, 'Shmui likes flowers');

We'll have to figure out what will be the final syntax - but I guess it will be resembling the python client language?

ttmc · 2016-05-10T07:51:27Z

The Python driver code will probably share some code with the BigchainDB server code, so having them in separate repos would be a pain. Maybe the Python driver is a special case though, and all other drivers can go in another repo (or set of repos).

RethinkDB has all their supported drivers in a subdirectory named rethinkdb/drivers/ e.g. rethinkdb/drivers/javascript.

https://github.com/rethinkdb/rethinkdb/tree/next/drivers

TimDaub · 2016-05-10T08:23:09Z

What's a use case of decoupling the creation of a transaction and the signing? The only idea that pops up in my mind is: a third party creates a transaction and a user signs it with the client (and this can be a separated function).

👍

I can see two main use cases for this driver:

Use it in a nodejs backend to store your users' meta data decentralized in a bigchaindb federation
Use it in a JavaScript frontend to store your meta data decentralized in a bigchaindb federation

In use case one, all your business logic is residing in your backend. If something needs to be executed, certain conditions need to be valued to not make your data inconsistent. What you'd like to do in that case is create a transaction, push it to the client. The client signs it on his computer by generating his privatekey on the fly and sends it back to the server. Transaction gets pushed to the federation by the backend. SPOOL would be an example of this use case.

In use case two, all your business logic is residing in the frontend. A backend might not even be needed.
Contrary to use case one, here it's very likely that your driver never has to handle more than one keypair per application instance. A transaction is created, signed and pushed to the network all in the frontend. No real need to separate creation and signature.

Right, it was in my head but not in the proposal . Promises for sure. I'll update the proposal now (do you suggest using Q or should I rely on ECMAScript 2015 promises?)

Little bit out of the loop, but all the cool kids from the block use https://github.com/petkaantonov/bluebird right now, I heard.

Wow, thanks a lot for the examples. In this case we don't have a wire protocol, but just http requests, so you don't really "connect" to BigchainDB.

Does it really matter?
All input and output in Javascript needs to be handled async. Whether I'm using pg-node or fetch to retrieve some data from the outside, I'll have to use some kind of async paradigm in my code.
I'm trying to say: It's not the same, but we could make it feel the same.
The question for me is though: What do we really want to implement?

a rest api client that feels like a database driver
a rest api client that feeld like a rest api client
(are there more options?)

There is another problem anyway. The lifecycle of a transaction is:
Right now the promise returns when the transaction has been accepted by the Federation. We need another mechanism to notify the user when the transaction moves from undecided to valid or invalid. This can be generalized in the future using a websocket for asynchronous communications.

Great point! All a federation node could do right now, is to send a 202 Accepted.
A federation node could however expose some pub-sub interface to listen for specific events that are interesting to the client.

Something like:

Notify me about all transactions that relate to one of my public keys

As a side note, Ethereum has a really cool concept of notifying the outside world about stuff (not sure how applicable this is in our case):
https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI#events

Also have a look at the interledger project:
https://github.com/interledger/
https://github.com/interledger/specs/blob/gh-pages/0001-interledger-architecture.md

Great point. What is the relation of this client and cryptoconditions? Does it allow me to do all the things, cryptoconditions (I'm speaking of their JS implementation) let me?

sohkai · 2016-05-10T09:42:31Z

Right, it was in my head but not in the proposal . Promises for sure. I'll update the proposal now (do you suggest using Q or should I rely on ECMAScript 2015 promises?)

Unless we really find ourselves needing the extra features from the other promise libraries, I'd prefer using ES2015 promises. This way you won't force your choice on other's projects, and if they really want to use one of the other libraries, they can always wrap the promises we return with their library of choice or, with some configuration, override the Promise global so that our driver would also internally use and return the same type of Promises.

vrde · 2016-05-10T10:05:39Z

https://github.com/interledger/specs/blob/gh-pages/0001-interledger-architecture.md

@diminator: thanks, this is pretty interesting!

We'll have to figure out what will be the final syntax - but I guess it will be resembling the python client language?

@diminator: for this first iteration I want to focus on few use cases, once they are there we can decide if we like the API and iterate from that :)

The Python driver code will probably share some code with the BigchainDB server code, so having them in separate repos would be a pain. Maybe the Python driver is a special case though, and all other drivers can go in another repo (or set of repos).

@troy: should not be a problem anyway to keep it in a different repo anyway, since the dependencies are defined in setup.py

I can see two main use cases for this driver: [...]

@TimDaub: Good use cases, thanks!

Does it really matter?

@TimDaub: IMO yes. A connection is something that might need a pool (of connections), or can be lost, etc :)

A federation node could however expose some pub-sub interface to listen for specific events that are interesting to the client.

@TimDaub: Yup, the Federation Nodes can expose a pubsub API. The Driver can subscribe to specific events to it's own key or more.

Great point. What is the relation of this client and cryptoconditions? Does it allow me to do all the things, cryptoconditions (I'm speaking of their JS implementation) let me?

@TimDaub: the Driver is just a handy tool to connect to the Federation and push some transactions. The Driver ease the creation of transactions for the common cases. If the user has more specific needs, they will always be able to create cryptoconditions by hand /cc @diminator

Unless we really find ourselves needing the extra features from the other promise libraries, I'd prefer using ES2015 promises.

@sohkai: do I need to use a shim? Maybe we can talk a second about that.

TimDaub · 2017-02-17T10:14:13Z

For everyone external looking at this ticket:

@sohkai has developed a minimal viable JS library to build bigchaindb transactions here: https://github.com/sohkai/js-bigchaindb-quickstart

EDIT

We're now officially supporting a javascript driver here: https://github.com/bigchaindb/js-bigchaindb-driver

kremalicious · 2017-05-17T09:50:01Z

this issue here is the first result coming up when I google for bigchaindb nodejs. A user interested in that would miss the fact we seem to have a Nodejs client here: https://github.com/bigchaindb/js-bigchaindb-driver (hard to tell cause Node.js is NOT EVEN MENTIONED ONCE in the readme, name or description)

TimDaub · 2017-05-17T18:13:06Z

Good point. @kremalicious Diwcoverability and SEO is important. You've created a ticket on the driver. Great.

Also, I'll close this ticket as I largely consider it done (we should/can still take it as a starting point for the js-bigchaindb-driver).

vrde self-assigned this May 9, 2016

vrde added the project label May 9, 2016

vrde changed the title ~~JavaScript client~~ JavaScript Driver May 9, 2016

trentmc added 4) UX docs cmd api driver and removed 0. Project in future labels May 13, 2016

ttmc added the Project-Issue label May 20, 2016

ttmc added when: In the future and removed when: In progress labels Jun 6, 2016

This was referenced Aug 11, 2016

Define v1.1 Python Client API #542

Closed

Define BichainDB API v1.1 Client API bigchaindb/bigchaindb-driver#8

Closed

sohkai self-assigned this Oct 10, 2016

ttmc removed the when: Later label Nov 4, 2016

kremalicious unassigned sohkai May 17, 2017

TimDaub closed this as completed May 17, 2017

kremalicious mentioned this issue May 17, 2017

This repo is undiscoverable and confusing bigchaindb/js-bigchaindb-driver#9

Closed

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JavaScript Driver #256

JavaScript Driver #256

vrde commented May 9, 2016 •

edited

Loading

ttmc commented May 9, 2016

vrde commented May 9, 2016

TimDaub commented May 9, 2016 •

edited

Loading

sohkai commented May 9, 2016

vrde commented May 9, 2016

vrde commented May 9, 2016

vrde commented May 9, 2016

diminator commented May 10, 2016

diminator commented May 10, 2016

diminator commented May 10, 2016

ttmc commented May 10, 2016

TimDaub commented May 10, 2016 •

edited

Loading

sohkai commented May 10, 2016

vrde commented May 10, 2016

TimDaub commented Feb 17, 2017 •

edited

Loading

kremalicious commented May 17, 2017 •

edited

Loading

TimDaub commented May 17, 2017

JavaScript Driver #256

JavaScript Driver #256

Comments

vrde commented May 9, 2016 • edited Loading

Abstract

Goals

Challenges

Architecture

Examples

Roadmap

Deadline May 12th TBD

Later

ttmc commented May 9, 2016

vrde commented May 9, 2016

TimDaub commented May 9, 2016 • edited Loading

sohkai commented May 9, 2016

vrde commented May 9, 2016

vrde commented May 9, 2016

vrde commented May 9, 2016

diminator commented May 10, 2016

diminator commented May 10, 2016

diminator commented May 10, 2016

ttmc commented May 10, 2016

TimDaub commented May 10, 2016 • edited Loading

sohkai commented May 10, 2016

vrde commented May 10, 2016

TimDaub commented Feb 17, 2017 • edited Loading

EDIT

We're now officially supporting a javascript driver here: https://github.com/bigchaindb/js-bigchaindb-driver

kremalicious commented May 17, 2017 • edited Loading

TimDaub commented May 17, 2017

vrde commented May 9, 2016 •

edited

Loading

TimDaub commented May 9, 2016 •

edited

Loading

TimDaub commented May 10, 2016 •

edited

Loading

TimDaub commented Feb 17, 2017 •

edited

Loading

kremalicious commented May 17, 2017 •

edited

Loading