add api

[juliangruber]

spawned by discussion in #495, should we create a high level javascript api to the dat network?

I think +1, a module called dat-api or something, separate from this repo, so

1. you don't install all the logging and cli related stuff when installing dat-api
2. the cli can be simplified a lot

[joehand]

+1 for having a library, and keeping it separate from the cli. While it's pretty straightforward to use hyperdrive and related modules to DIY Dat, it's not trivial.

Having a dedicated JS library will allow us to make a tested library that developers can use immediately. With a DIY Dat stuff, developers will have a bit more leg work and may have to add tests for dat-related code.

I've (kinda) already started this in the logger-module branch.

[joehand]

+1 for having a library, and keeping it separate from the cli. While it's pretty straightforward to use hyperdrive and related modules to DIY Dat, it's not trivial.

Having a dedicated JS library will allow us to make a tested library that developers can use immediately. With a DIY Dat stuff, developers will have a bit more leg work and may have to add tests for dat-related code.

I've (kinda) already started this in the logger-module branch.

[lukeburns]

+1
A minimal high-level library might look something like...

var api = require('dat-api')
var dat = api()

dat.share(path, function (err, key) {
  if (!err) console.log('key', key)
})

dat.download(key, path, function (err) {
  if (!err) console.log('saved to', path)
})

[lukeburns]

+1
A minimal high-level library might look something like...

var api = require('dat-api')
var dat = api()

dat.share(path, function (err, key) {
  if (!err) console.log('key', key)
})

dat.download(key, path, function (err) {
  if (!err) console.log('saved to', path)
})

[juliangruber]

we can probably start with a 1:1 mapping of cli commands to js methods

[juliangruber]

we can probably start with a 1:1 mapping of cli commands to js methods

[maxogden]

paging @mafintosh (who has relevant opinions)

[maxogden]

paging @mafintosh (who has relevant opinions)

[mafintosh]

I've been arguing 👎 for having a official Dat js api.

There is a bunch of trade-offs / configuration involved in using the tool, and imo it's more sustainable to do a modular approach where we have a series of small modules that make out the "Dat framework".

Since Dat doesn't have a clear "one thing it does" it invites bike shedding agreeing on APIs and we'd be indirectly blessing some specific use cases. For example, booting docker containers over Dat, requires a very different approach than simple cloning a normal data. Writing a chat app of top of Dat has similar differences.

What I propose instead is a more radical massively modular approach where we publish a series of decoupled modules that allows you to compose a dat stack in ~20 lines of code. This forces us to decouple the stack as well. Whenever something is too complicated to do, make a module that makes it easier.

The hyperdrive-archive-swarm module is a great example of this as it reduced the complexity of setting up a network swarm to hyperdriveArchiveSwarm(archive). If adding a bunch of files is too complicated a similar approach would help make that easier.

For this approach to work we need to have very good documentation and examples that show how you compose different use cases and explains how everything fits together. It also hurts discoverability compared to having a Dat framework module.

[mafintosh]

I've been arguing 👎 for having a official Dat js api.

There is a bunch of trade-offs / configuration involved in using the tool, and imo it's more sustainable to do a modular approach where we have a series of small modules that make out the "Dat framework".

Since Dat doesn't have a clear "one thing it does" it invites bike shedding agreeing on APIs and we'd be indirectly blessing some specific use cases. For example, booting docker containers over Dat, requires a very different approach than simple cloning a normal data. Writing a chat app of top of Dat has similar differences.

What I propose instead is a more radical massively modular approach where we publish a series of decoupled modules that allows you to compose a dat stack in ~20 lines of code. This forces us to decouple the stack as well. Whenever something is too complicated to do, make a module that makes it easier.

The hyperdrive-archive-swarm module is a great example of this as it reduced the complexity of setting up a network swarm to hyperdriveArchiveSwarm(archive). If adding a bunch of files is too complicated a similar approach would help make that easier.

For this approach to work we need to have very good documentation and examples that show how you compose different use cases and explains how everything fits together. It also hurts discoverability compared to having a Dat framework module.

[maxogden]

So if we decide that "Dat" is a very narrow subset of the possibility space that is the hyperdrive/hypercore/discovery-channel stack, then we can have the 'dat-api' module only do that one narrow subset.

On the other hand I agree with the quote above: "While it's pretty straightforward to use hyperdrive and related modules to DIY Dat, it's not trivial.".

I guess it comes down to: do we wanna push one use case and make that the flagship easy to use thing, or do we want to not commit to one use case, and make it more advanced to build your own framework.

IMO a good ecosystem has both, so I don't think they're mutually exclusive (see choo vs yo-yo vs bel for example)

[maxogden]

So if we decide that "Dat" is a very narrow subset of the possibility space that is the hyperdrive/hypercore/discovery-channel stack, then we can have the 'dat-api' module only do that one narrow subset.

On the other hand I agree with the quote above: "While it's pretty straightforward to use hyperdrive and related modules to DIY Dat, it's not trivial.".

I guess it comes down to: do we wanna push one use case and make that the flagship easy to use thing, or do we want to not commit to one use case, and make it more advanced to build your own framework.

IMO a good ecosystem has both, so I don't think they're mutually exclusive (see choo vs yo-yo vs bel for example)

[karissa]

I'm fine with having a JS api, but if it exists it should expose only the same exact functions (and options) available in the cli.

[karissa]

I'm fine with having a JS api, but if it exists it should expose only the same exact functions (and options) available in the cli.

[maxogden]

good discussion everyone, if someone wants to pursue this sounds like they can start by experimenting with a separate dat-api module outside this repo

[maxogden]

good discussion everyone, if someone wants to pursue this sounds like they can start by experimenting with a separate dat-api module outside this repo

[joehand]

Started working on that here: https://github.com/joehand/dat-js.

[joehand]

Started working on that here: https://github.com/joehand/dat-js.