Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

Commit

Permalink
feat: add libp2p generator config option with example
Browse files Browse the repository at this point in the history
  • Loading branch information
jacobheun committed Jul 27, 2018
1 parent 1f63e8c commit deab67d
Show file tree
Hide file tree
Showing 8 changed files with 434 additions and 42 deletions.
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,12 @@ Creates and returns an instance of an IPFS node. Use the `options` argument to s

- `config` (object) Modify the default IPFS node config. Find the Node.js defaults at [`src/core/runtime/config-nodejs.js`](https://github.com/ipfs/js-ipfs/tree/master/src/core/runtime/config-nodejs.js) and the browser defaults at [`src/core/runtime/config-browser.js`](https://github.com/ipfs/js-ipfs/tree/master/src/core/runtime/config-browser.js). This object will be *merged* with the default config; it will not replace it.

- `libp2p` (object) add custom modules to the libp2p stack of your node
- `libp2p` (object or function(ipfs, config)) add custom modules to the libp2p stack of your node

The libp2p option allows you to build your libp2p node by configuration, or via a generator. If you are looking to just modify the below options, using the object format is the quickest way to get the default features of libp2p. If you need to create a more customized libp2p node, such as with custom transports or peer/content routers that need some of the ipfs data on startup, a generator is a great way to achieve this.

You can see the generator in action in the [custom libp2p example](examples/custom-libp2p).

- `modules` (object):
- `transport` (Array<[libp2p.Transport](https://github.com/libp2p/interface-transport)>): An array of Libp2p transport classes/instances to use _instead_ of the defaults. See [libp2p/interface-transport](https://github.com/libp2p/interface-transport) for details.
- `peerDiscovery` (Array<[libp2p.PeerDiscovery](https://github.com/libp2p/interface-peer-discovery)>): An array of Libp2p peer discovery classes/instances to use _instead_ of the defaults. See [libp2p/peer-discovery](https://github.com/libp2p/interface-peer-discovery) for details. If passing a class, configuration can be passed using the config section below under the key corresponding to you module's unique `tag` (a static property on the class)
Expand Down Expand Up @@ -360,10 +365,10 @@ The core API is grouped into several areas:
- [`ipfs.files.addPullStream([options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesaddpullstream)
- [`ipfs.files.addReadableStream([options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesaddreadablestream)
- [`ipfs.files.cat(ipfsPath, [options], [callback])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filescat). Alias to `ipfs.cat`.
- [`ipfs.files.catPullStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filescatpullstream)
- [`ipfs.files.catPullStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filescatpullstream)
- [`ipfs.files.catReadableStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filescatreadablestream)
- [`ipfs.files.get(ipfsPath, [options], [callback])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesget). Alias to `ipfs.get`.
- [`ipfs.files.getPullStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesgetpullstream)
- [`ipfs.files.getPullStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesgetpullstream)
- [`ipfs.files.getReadableStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesgetreadablestream)
- [`ipfs.ls(ipfsPath, [callback])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#ls)
- [`ipfs.lsPullStream(ipfsPath)`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#lspullstream)
Expand Down
18 changes: 18 additions & 0 deletions examples/custom-libp2p/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Customizing the libp2p node

This example shows you how to make full use of the ipfs configuration to create a libp2p generator function. As IPFS applications become more complex, their needs for a custom libp2p node also grow. Instead of fighting with configuration options, you can use your own libp2p generator to get exactly what you need. This example shows you how.

## Run this example

Running this example should result in metrics being logged out to the console every few seconds.

```
> npm install
> npm start
```

## Play with the configuration!

With the metrics for peers and bandwidth stats being logged out, try playing around with the nodes configuration to see what kind of metrics you can get. How many peers are you getting? What does your bandwidth look like?

This is also a good opportunity to explore the various stats that ipfs offers! Not seeing a statistic you think would be useful? We'd love to have you [contribute](https://github.com/ipfs/js-ipfs/blob/master/CONTRIBUTING.md)!
121 changes: 121 additions & 0 deletions examples/custom-libp2p/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
'use strict'

const Libp2p = require('libp2p')
const IPFS = require('ipfs')
const TCP = require('libp2p-tcp')
const MulticastDNS = require('libp2p-mdns')
const WebSocketStar = require('libp2p-websocket-star')
const Bootstrap = require('libp2p-railing')
const SPDY = require('libp2p-spdy')
const KadDHT = require('libp2p-kad-dht')
const MPLEX = require('libp2p-mplex')
const SECIO = require('libp2p-secio')
const assert = require('assert')

/**
* This is the generator we will use to generate our fully customized libp2p node.
*
* @param {*} _ipfsNode The ipfs node. This houses the PeerInfo and PeerBook that modules may need
* @param {*} _ipfsConfig The config that is fetched from the ipfs-repo
* @returns {Libp2p} Our new libp2p node
*/
const libp2pGenerator = (_ipfsNode, _ipfsConfig) => {
// Set convenience variables to clearly showcase some of the useful things that are available
const peerInfo = _ipfsNode._peerInfo
const peerBook = _ipfsNode._peerBook
const bootstrapList = _ipfsConfig.Bootstrap

// Create our WebSocketStar transport and give it our PeerId, straight from the ipfs node
const wsstar = new WebSocketStar({
id: peerInfo.id
})

// Build and return our libp2p node
return new Libp2p({
peerInfo,
peerBook,
// Lets limit the connection managers peers and have it check peer health less frequently
connectionManager: {
maxPeers: 25,
pollInterval: 5000
},
modules: {
transport: [
TCP,
wsstar
],
streamMuxer: [
MPLEX,
SPDY
],
connEncryption: [
SECIO
],
peerDiscovery: [
MulticastDNS,
Bootstrap,
wsstar.discovery
],
dht: KadDHT
},
config: {
peerDiscovery: {
mdns: {
interval: 10000,
enabled: true
},
bootstrap: {
interval: 10000,
enabled: true,
list: bootstrapList
}
},
// Turn on relay with hop active so we can connect to more peers
relay: {
enabled: true,
hop: {
enabled: true,
active: true
}
},
dht: {
kBucketSize: 20
},
EXPERIMENTAL: {
dht: true,
pubsub: true
}
}
})
}

// Now that we have our custom generator, let's start up the ipfs node!
const node = new IPFS({
libp2p: libp2pGenerator
})

// Listen for the node to start, so we can log out some metrics
node.once('start', (err) => {
assert.ifError(err, 'Should startup without issue')

// Lets log out the number of peers we have every 2 seconds
setInterval(() => {
node.swarm.peers((err, peers) => {
if (err) {
console.log('An error occurred trying to check our peers:', err)
process.exit(1)
}
console.log(`The node now has ${peers.length} peers.`)
})
}, 2000)

// Log out the bandwidth stats every 4 seconds so we can see how our configuration is doing
setInterval(() => {
node.stats.bw((err, stats) => {
if (err) {
console.log('An error occurred trying to check our stats:', err)
}
console.log(`\nBandwidth Stats: ${JSON.stringify(stats, null, 2)}\n`)
})
}, 4000)
})
23 changes: 23 additions & 0 deletions examples/custom-libp2p/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"name": "custom-libp2p",
"version": "0.1.0",
"description": "Customizing your libp2p node",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "node index.js"
},
"license": "MIT",
"dependencies": {
"ipfs": "file:../../",
"libp2p": "~0.22.0",
"libp2p-kad-dht": "~0.10.1",
"libp2p-mdns": "~0.12.0",
"libp2p-mplex": "~0.8.0",
"libp2p-railing": "~0.9.2",
"libp2p-secio": "~0.10.0",
"libp2p-spdy": "~0.12.1",
"libp2p-tcp": "~0.12.0",
"libp2p-websocket-star": "~0.8.1"
}
}
82 changes: 46 additions & 36 deletions src/core/components/libp2p.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,49 +16,59 @@ module.exports = function libp2p (self) {
return callback(err)
}

const libp2pDefaults = {
peerInfo: self._peerInfo,
peerBook: self._peerInfoBook,
config: {
peerDiscovery: {
mdns: {
enabled: get(self._options, 'config.Discovery.MDNS.Enabled',
get(config, 'Discovery.MDNS.Enabled', true))
const defaultGenerator = (_ipfs, _config) => {
const libp2pDefaults = {
peerInfo: _ipfs._peerInfo,
peerBook: _ipfs._peerInfoBook,
config: {
peerDiscovery: {
mdns: {
enabled: get(_ipfs._options, 'config.Discovery.MDNS.Enabled',
get(_config, 'Discovery.MDNS.Enabled', true))
},
webRTCStar: {
enabled: get(_ipfs._options, 'config.Discovery.webRTCStar.Enabled',
get(_config, 'Discovery.webRTCStar.Enabled', true))
},
bootstrap: {
list: get(_ipfs._options, 'config.Bootstrap',
get(_config, 'Bootstrap', []))
}
},
webRTCStar: {
enabled: get(self._options, 'config.Discovery.webRTCStar.Enabled',
get(config, 'Discovery.webRTCStar.Enabled', true))
relay: {
enabled: get(_ipfs._options, 'relay.enabled',
get(_config, 'relay.enabled', false)),
hop: {
enabled: get(_ipfs._options, 'relay.hop.enabled',
get(_config, 'relay.hop.enabled', false)),
active: get(_ipfs._options, 'relay.hop.active',
get(_config, 'relay.hop.active', false))
}
},
bootstrap: {
list: get(self._options, 'config.Bootstrap',
get(config, 'Bootstrap', []))
EXPERIMENTAL: {
dht: get(_ipfs._options, 'EXPERIMENTAL.dht', false),
pubsub: get(_ipfs._options, 'EXPERIMENTAL.pubsub', false)
}
},
relay: {
enabled: get(self._options, 'relay.enabled',
get(config, 'relay.enabled', false)),
hop: {
enabled: get(self._options, 'relay.hop.enabled',
get(config, 'relay.hop.enabled', false)),
active: get(self._options, 'relay.hop.active',
get(config, 'relay.hop.active', false))
}
},
EXPERIMENTAL: {
dht: get(self._options, 'EXPERIMENTAL.dht', false),
pubsub: get(self._options, 'EXPERIMENTAL.pubsub', false)
}
},
connectionManager: get(self._options, 'connectionManager',
get(config, 'connectionManager', {}))
connectionManager: get(_ipfs._options, 'connectionManager',
get(_config, 'connectionManager', {}))
}

const libp2pOptions = defaultsDeep(
get(self._options, 'libp2p', {}),
libp2pDefaults
)

return new Node(libp2pOptions)
}

const libp2pOptions = defaultsDeep(
get(self._options, 'libp2p', {}),
libp2pDefaults
)
// Always create libp2p via a generator
let libp2pGenerator = get(self._options, 'libp2p', null)
if (typeof libp2pGenerator !== 'function') {
libp2pGenerator = defaultGenerator
}

self._libp2pNode = new Node(libp2pOptions)
self._libp2pNode = libp2pGenerator(self, config)

self._libp2pNode.on('peer:discovery', (peerInfo) => {
const dial = () => {
Expand Down
9 changes: 6 additions & 3 deletions src/core/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,12 @@ const schema = Joi.object().keys({
}).allow(null),
Bootstrap: Joi.array().items(Joi.multiaddr().IPFS().options({ convert: false }))
}).allow(null),
libp2p: Joi.object().keys({
modules: Joi.object().allow(null) // TODO: schemas for libp2p modules?
}).allow(null)
libp2p: Joi.alternatives().try(
Joi.func(),
Joi.object().keys({
modules: Joi.object().allow(null) // TODO: schemas for libp2p modules?
})
).allow(null)
}).options({ allowUnknown: true })

module.exports.validate = (config) => Joi.attempt(config, schema)
1 change: 1 addition & 0 deletions test/core/config.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ describe('config', () => {
{ libp2p: { modules: null } },
{ libp2p: { modules: undefined } },
{ libp2p: { unknown: 'value' } },
{ libp2p: () => {} },
{ libp2p: null },
{ libp2p: undefined }
]
Expand Down
Loading

0 comments on commit deab67d

Please sign in to comment.