Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The big refactor! #200

Merged
merged 37 commits into from
Feb 12, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
012c327
part I of the big refactor!
daviddias Feb 7, 2018
5040d41
foundation for browser tsts
daviddias Feb 7, 2018
df9ffa0
SLAAAAY the 🐉🐉🐉
daviddias Feb 9, 2018
caca0e2
add-retrieve
daviddias Feb 9, 2018
d72ab8a
test: api.node
daviddias Feb 9, 2018
06f70a2
docs: trim the readme
daviddias Feb 9, 2018
2582dbf
moar refactor
daviddias Feb 9, 2018
9f6632c
test: remote endpoint tests
daviddias Feb 9, 2018
56c618e
start spinning browser tests again
daviddias Feb 9, 2018
b0da61a
I am finally happy with the structure of this project
daviddias Feb 9, 2018
a9f6031
fix tests in node as well as resolve a couple of todos
dryajov Feb 9, 2018
7c149a1
feat: implemented the version method for factory-in-proc
dryajov Feb 9, 2018
664d8ed
fix: `daemon exec path should match type` test
dryajov Feb 9, 2018
e476440
lint
dryajov Feb 9, 2018
aab54af
doc
dryajov Feb 9, 2018
fc49b3b
fix: get proc version from package.json
dryajov Feb 9, 2018
c8083df
test: browser tests are working again
daviddias Feb 10, 2018
8ff1976
test: more browser testing
daviddias Feb 10, 2018
2a00842
feat: added `version` endpoint
dryajov Feb 10, 2018
37194d3
experiment
dryajov Feb 10, 2018
7ee051f
feat: add tmpDir endpoint to be able to generate tmp repos in browser
dryajov Feb 10, 2018
97ab8ed
chore
daviddias Feb 12, 2018
581945e
test: fix spawn options tests
daviddias Feb 12, 2018
4d87386
dont delete addrs
daviddias Feb 12, 2018
8cc19eb
docs: polish the readme
daviddias Feb 12, 2018
12ef363
chore
daviddias Feb 12, 2018
63f5243
test: all running
daviddias Feb 12, 2018
5738822
test: all passing locally
daviddias Feb 12, 2018
bea0caf
bump timeouts
daviddias Feb 12, 2018
1463b98
test: add prefix back to repo name
daviddias Feb 12, 2018
19a4133
chore
daviddias Feb 12, 2018
ce7b68c
chore: timeout dance
daviddias Feb 12, 2018
d2f8680
fix: tmp dir
daviddias Feb 12, 2018
d9ada2e
real timeout
daviddias Feb 12, 2018
dd20ef7
feat: do not depend on any locally initalized daemon
daviddias Feb 12, 2018
30c25c6
chore
daviddias Feb 12, 2018
f9d3eb4
now yes
daviddias Feb 12, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .aegir.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

const createServer = require('./src').createServer

const server = createServer()
const server = createServer() // using defaults

module.exports = {
karma: {
files: [{
Expand Down
187 changes: 115 additions & 72 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,66 +1,45 @@
# ipfsd-ctl
# ipfsd-ctl, the IPFS Factory.

[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io)
[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://protocol.ai)
[![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/)
[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
[![standard-readme compliant](https://img.shields.io/badge/standard--readme-OK-green.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme)
[![standard-readme](https://img.shields.io/badge/standard--readme-OK-green.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme)
[![Coverage Status](https://coveralls.io/repos/github/ipfs/js-ipfsd-ctl/badge.svg?branch=master)](https://coveralls.io/github/ipfs/js-ipfsd-ctl?branch=master)
[![Travis CI](https://travis-ci.org/ipfs/js-ipfsd-ctl.svg?branch=master)](https://travis-ci.org/ipfs/js-ipfsd-ctl)
[![Circle CI](https://circleci.com/gh/ipfs/js-ipfsd-ctl.svg?style=svg)](https://circleci.com/gh/ipfs/js-ipfsd-ctl)
[![Appveyor CI](https://ci.appveyor.com/api/projects/status/4p9r12ch0jtthnha?svg=true)](https://ci.appveyor.com/project/wubalubadubdub/js-ipfsd-ctl-a9ywu)
[![Dependency Status](https://david-dm.org/ipfs/js-ipfsd-ctl.svg?style=flat-square)](https://david-dm.org/ipfs/js-ipfsd-ctl) [![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/feross/standard)
[![Dependency Status](https://david-dm.org/ipfs/js-ipfsd-ctl.svg?style=flat-square)](https://david-dm.org/ipfs/js-ipfsd-ctl)
[![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/feross/standard)

> Control an IPFS daemon using JavaScript in Node.js or in the Browser.

```
+-----+
| H |
| T |
+-----------------------------+ | T |
| Node.js | +-----------------------+ | P | +-----------------------------+
| | | | | | | BROWSER |
| +-----------------------+ | | IPFS Daemon | | S | | |
| | Local Daemon Ctrl | | | | | E | | +----------------------+ |
| | +------- -------- R -----|---- Remote Daemon Ctrl | |
| +-----------------------+ | +-----|-----------|-----+ | V | | | | |
| | | | | E | | +----------------------+ |
| +-----------------------+ | | | | R | | |
| | IPFS API | | | | +-----+ | +----------------------+ |
| | -------------+ | | | IPFS API | |
| +-----------------------+ | +-----------------------|---- | |
| | | +----------------------+ |
+-----------------------------+ +-----------------------------+
```
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a new diagram.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the current diagram missing?

> Spawn IPFS daemons using JavaScript!

## Table of Contents

- [Install](#install)
- [Usage](#usage)
- [API](#api)
- [Packaging](#packaging)
- [Contribute](#contribute)
- [License](#license)

## Install

Install:
```sh
npm install --save ipfsd-ctl
```

## Usage

IPFS daemons are already easy to start and stop, but this module is here to do it from JavaScript itself.

### Spawn an IPFS daemon from Node.js
**Spawn an IPFS daemon from Node.js**

```js
// Start a disposable node, and get access to the api
// print the node id, and stop the temporary daemon

const DaemonFactory = require('ipfsd-ctl')
const df = DaemonFactory.create()
const IPFSFactory = require('ipfsd-ctl')
const f = IPFSFactory.create()

df.spawn(function (err, ipfsd) {
f.spawn(function (err, ipfsd) {
if (err) { throw err }

ipfsd.api.id(function (err, id) {
Expand All @@ -72,22 +51,22 @@ df.spawn(function (err, ipfsd) {
})
```

### Spawn an IPFS daemon from the Browser using the provided remote endpoint
**Spawn an IPFS daemon from the Browser using the provided remote endpoint**

```js
// Start a remote disposable node, and get access to the api
// print the node id, and stop the temporary daemon

const DaemonFactory = require('ipfsd-ctl')
const IPFSFactory = require('ipfsd-ctl')

const port = 9999
const server = DaemonFactory.createServer(port)
const df = DaemonFactory.create({ remote: true, port: port })
const server = IPFSFactory.createServer(port)
const f = IPFSFactory.create({ remote: true, port: port })

server.start((err) => {
if (err) { throw err }

df.spawn((err, ipfsd) => {
f.spawn((err, ipfsd) => {
if (err) { throw err }

ipfsd.api.id(function (err, id) {
Expand All @@ -102,61 +81,90 @@ server.start((err) => {

## Disposable vs non Disposable nodes

`ipfsd-ctl` can create two types of node controllers, `disposable` and `non-disposable`. A disposable node will be created on a temporary repo which will be optionally initialized and started (the default), as well cleaned up on process exit. A non-disposable node on the other hand, requires the user to initialize and start the node, as well as stop and cleanup after wards. Additionally, a non-disposable will allow you to pass a custom repo using the `repoPath` option, if the `repoPath` is not defined, it will use the default repo for the node type (`$HOME/.ipfs` or `$HOME/.jsipfs`). The `repoPath` parameter is ignored for disposable nodes, as there is a risk of deleting a live repo.
`ipfsd-ctl` can spawn `disposable` and `non-disposable` daemons.

## IPFS executables
- `disposable`- Creates on a temporary repo which will be optionally initialized and started (the default), as well cleaned up on process exit. Great for tests.
- `non-disposable` - Requires the user to initialize and start the node, as well as stop and cleanup after wards. Additionally, a non-disposable will allow you to pass a custom repo using the `repoPath` option, if the `repoPath` is not defined, it will use the default repo for the node type (`$HOME/.ipfs` or `$HOME/.jsipfs`). The `repoPath` parameter is ignored for disposable nodes, as there is a risk of deleting a live repo.

`ipfsd-ctl` no longer installs go-ipfs nor js-ipfs dependencies, instead it expects them to be provided by the parent project. In order to be able to use both go and js daemons, please make sure that your project includes these two npm packages as dependencies.
## Batteries not included. Bring your own IPFS executable.

- `ipfs` - the js-ipfs implementation
- `go-ipfs-dep` - the packaged go-ipfs implementation
Install one or both of the following modules:

## API
- `ipfs` - `> npm i ipfs` - If you want to spawn js-ipfs nodes and/or daemons.
- `go-ipfs-dep` - `> npm i go-ipfs-dep` - If you want to spwan go-ipfs daemons.

### Daemon Factory Class

#### `DaemonFactory` - `const df = DaemonFactory.create([options])`

`DaemonFactory.create([options])` returns an object that will expose the `df.spawn` method

- `options` - an optional object with the following properties
- `remote` bool - indicates if the factory should spawn local or remote nodes. By default, local nodes are spawned in Node.js and remote nodes are spawned in Browser environments.
- `port` number - the port number to use for the remote factory. It should match the port on which `DaemonFactory.server` was started. Defaults to 9999.
- `type` - the daemon type to create with this factory. See the section bellow for the supported types
- `exec` - path to the desired IPFS executable to spawn, otherwise `ipfsd-ctl` will try to locate the correct one based on the `type`. In the case of `proc` type, exec is required and expects an IPFS coderef.

`ipfsd-ctl` allows spawning different IPFS implementations, such as:
## API

- **`go`** - calling `DaemonFactory.create({type: 'go'})` will spawn a `go-ipfs` daemon.
- **`js`** - calling `DaemonFactory.create({type: 'js'})` will spawn a `js-ipfs` daemon.
- **`proc`** - calling `DaemonFactory.create({type: 'proc', exec: require('ipfs') })` will spawn an `in process js-ipfs node` using the provided code reference that implements the core IPFS API. Note that, `exec` option to `df.spawn()` is required if `type: 'proc'` is used.

#### DaemonFactory endpoint for remote spawning - `const server = `DaemonFactory.createServer([options]) `
### `IPFSFactory` - `const f = IPFSFactory.create([options])`

`DaemonFactory.createServer` create an instance of the bundled REST API used by the remote controller.
`IPFSFactory.create([options])` returns an object that will expose the `df.spawn` method

- exposes `start` and `stop` methods to start and stop the http server endpoint.
- `options` - optional object with:
- `remote` bool - use remote endpoint to spawn the nodes.
- `port` number - remote endpoint point. Defaults to 9999.
- `exec` - IPFS executable path. `ipfsd-ctl` will attempt to locate it by default. If you desire to spawn js-ipfs instances in the same process, pass the ref to the module instead (e.g `exec: require('ipfs')`)
- `type` - the daemon type, see below the options
- `go` - spawn go-ipfs daemon
- `js` - spawn js-ipfs daemon
- `proc` - spawn in-process js-ipfs instance. Needs to be called also with exec. Example: `DaemonFactory.create({type: 'proc', exec: require('ipfs') })`.

#### Spawn a new daemon with `df.spawn`
**example:** See [Usage](#usage)

Spawn either a js-ipfs or go-ipfs daemon
#### Spawn a daemon with `f.spawn([options], callback)`

`df.spawn([options], callback)`
Spawn the daemon

`options` is an optional object the following properties:
- `options` is an optional object the following properties:
- `init` bool (default true) - should the node be initialized
- `start` bool (default true) - should the node be started
- `repoPath` string - the repository path to use for this node, ignored if node is disposable
- `disposable` bool (default true) - a new repo is created and initialized for each invocation, as well as cleaned up automatically once the process exits
- `args` - array of cmd line arguments to be passed to ipfs daemon
- `config` - ipfs configuration options

`callback` - is a function with the signature `function (err, ipfsd)` where:
- `callback` - is a function with the signature `function (err, ipfsd)` where:
- `err` - is the error set if spawning the node is unsuccessful
- `ipfsd` - is the daemon controller instance:
- `api` - a property of `ipfsd`, an instance of [ipfs-api](https://github.com/ipfs/js-ipfs-api) attached to the newly created ipfs node

### IPFS Daemon Controller (`ipfsd`)
**example:** See [Usage](#usage)

#### Get daemon version with `f.version(callback)`

Get the version without spawning a daemon

- `callback` - is a function with the signature `function(err, version)`, where version might be one of the following:
- if `type` is 'go' a version string like `ipfs version <version number>`
- if `type` is 'js' a version string like `js-ipfs version: <version number>`
- if `type` is 'proc' an object with the following properties:
- version - the ipfs version
- repo - the repo version
- commit - the commit hash for this version

### Remote endpoint - `const server = `IPFSFactory.createServer([options]) `

`IPFSFactory.createServer` starts a IPFSFactory endpoint.

**example:**
```
const IPFSFactory = require('ipfsd-ctl')

const server = IPFSFactory.createServer({ port: 12345 })

server.start((err) => {
if (err) { throw err }

console.log('endpoint is running')

server.stop((err) => {
if (err) { throw err }

console.log('endpoint has stopped')
})
})
```

### IPFS Daemon Controller - `ipfsd`

The IPFS daemon controller (`ipfsd`) allows you to interact with the spawned IPFS daemon.

Expand Down Expand Up @@ -246,13 +254,13 @@ Get the version of ipfs

`callback` is a function with the signature `function(err, version)`

### IPFS Client (`ipfsd.api`)
### IPFS HTTP Client - `ipfsd.api`

An instance of [ipfs-api](https://github.com/ipfs/js-ipfs-api#api) that is used to interact with the daemon.

This instance is returned for each successfully started IPFS daemon, when either `df.spawn({start: true})` (the default) is called, or `ipfsd.start()` is invoked in the case of nodes that were spawned with `df.spawn({start: false})`.
This instance is returned for each successfully started IPFS daemon, when either `df.spawn({start: true})` (the default) is called, or `ipfsd.start()` is invoked in the case of nodes that were spawned with `df.spawn({start: false})`.

### Packaging
## Packaging

`ipfsd-ctl` can be packaged in Electron applications, but the ipfs binary has to be excluded from asar (Electron Archives).
[read more about unpack files from asar](https://electron.atom.io/docs/tutorial/application-packaging/#adding-unpacked-files-in-asar-archive).
Expand All @@ -265,6 +273,41 @@ electron-packager ./ --asar.unpackDir=node_modules/go-ipfs-dep

See [electron asar example](https://github.com/ipfs/js-ipfsd-ctl/tree/master/examples/electron-asar/)

## Development

Project structure:

```
src
├── defaults
│   ├── config.json
│   └── options.json
├── endpoint # endpoint to support remote spawning
│   ├── routes.js
│   └── server.js
├── factory-client.js # IPFS Factories: client (remote), daemon (go or js) and in-proc (js)
├── factory-daemon.js
├── factory-in-proc.js
├── index.js
├── ipfsd-client.js # ipfsd (Daemon Controller): client (remote), daemon (go or js), in-proc (js)
├── ipfsd-daemon.js
├── ipfsd-in-proc.js
└── utils # Utils used by the Factories and Daemon Controllers
├── configure-node.js
├── exec.js
├── find-ipfs-executable.js
├── flatten.js
├── parse-config.js
├── repo
│   ├── create-browser.js
│   └── create-nodejs.js
├── run.js
├── set-config-value.js
└── tmp-dir.js

4 directories, 21 files
```
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm finally happy with the code structure :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here for reference:

src
├── defaults
│   ├── config.json
│   └── options.json
├── endpoint                    # endpoint to support remote spawning
│   ├── routes.js
│   └── server.js
├── factory-client.js           # IPFS Factories: client (remote), daemon (go or js) and in-proc (js)
├── factory-daemon.js
├── factory-in-proc.js
├── index.js
├── ipfsd-client.js             # ipfsd (Daemon Controller): client (remote), daemon (go or js), in-proc (js)
├── ipfsd-daemon.js
├── ipfsd-in-proc.js
└── utils                       # Utils used by the Factories and Daemon Controllers
    ├── configure-node.js
    ├── exec.js
    ├── find-ipfs-executable.js
    ├── flatten.js
    ├── parse-config.js
    ├── repo
    │   ├── create-browser.js
    │   └── create-nodejs.js
    ├── run.js
    ├── set-config-value.js
    └── tmp-dir.js

4 directories, 21 files


## Contribute

Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfsd-ctl/issues)!
Expand Down
29 changes: 18 additions & 11 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,31 +1,38 @@
{
"name": "ipfsd-ctl",
"version": "0.27.3",
"description": "simple controls for an ipfs node",
"description": "Spawn IPFS Daemons, JS or Go",
"main": "src/index.js",
"scripts": {
"lint": "aegir lint",
"coverage": "COVERAGE=true aegir coverage --timeout 50000",
"docs": "aegir docs",
"build": "aegir build",
"test": "aegir test -t node -t browser --no-cors",
"test:node": "aegir test -t node",
"test:browser": "aegir test -t browser --no-cors",
"docs": "aegir docs",
"release": "aegir release -t node",
"release-minor": "aegir release --type minor -t node",
"release-major": "aegir release --type major -t node",
"release": "aegir release",
"release-minor": "aegir release --type minor",
"release-major": "aegir release --type major",
"coverage": "COVERAGE=true aegir coverage --timeout 50000",
"coverage-publish": "aegir coverage -u"
},
"browser": {
"./src/utils/create-repo-nodejs.js": "./src/utils/create-repo-browser.js",
"./src/daemon-node.js": false,
"./src/remote-node/routes.js": false,
"./src/exec.js": false,
"hapi": false,
"glob": false,
"fs": false,
"joi": false,
"stream": "readable-stream",
"http": "stream-http"
"http": "stream-http",
"./src/utils/repo/create-nodejs.js": "./src/utils/repo/create-browser.js",
"./src/utils/exec.js": false,
"./src/utils/find-ipfs-executable.js": false,
"./src/utils/tmp-dir.js": "./src/utils/tmp-dir-browser.js",
"./src/utils/run.js": false,
"./src/factory-daemon.js": false,
"./src/ipfsd-daemon.js": false,
"./src/endpoint/server.js": false,
"./src/endpoint/routes.js": false,
"./test/utils/df-config-nodejs.js": "./test/utils/df-config-browser.js"
},
"engines": {
"node": ">=6.0.0",
Expand Down
Loading