Skip to content

Commit

Permalink
feat: implement cluster client (#1)
Browse files Browse the repository at this point in the history
  • Loading branch information
gxcsoccer committed Dec 22, 2016
1 parent 1fd47d3 commit 0299b68
Show file tree
Hide file tree
Showing 32 changed files with 2,662 additions and 38 deletions.
4 changes: 4 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
test/fixtures
examples/**/app/public
logs
run
3 changes: 3 additions & 0 deletions .eslintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"extends": "eslint-config-egg"
}
24 changes: 24 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<!--
Thank you for your pull request. Please review below requirements.
Bug fixes and new features should include tests and possibly benchmarks.
Contributors guide: https://github.com/eggjs/egg/blob/master/CONTRIBUTING.md
感谢您贡献代码。请确认下列 checklist 的完成情况。
Bug 修复和新功能必须包含测试,必要时请附上性能测试。
Contributors guide: https://github.com/eggjs/egg/blob/master/CONTRIBUTING.md
-->

##### Checklist
<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

- [ ] `npm test` passes
- [ ] tests and/or benchmarks are included
- [ ] documentation is changed or added
- [ ] commit message follows commit guidelines

##### Affected core subsystem(s)
<!-- Provide affected core subsystem(s). -->


##### Description of change
<!-- Provide a description of the change below this comment. -->
54 changes: 18 additions & 36 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,37 +1,19 @@
# Logs
logs
*.log
npm-debug.log*

# Runtime data
pids
*.pid
*.seed

# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov

# Coverage directory used by tools like istanbul
coverage

# nyc test coverage
.nyc_output

# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt

# node-waf configuration
.lock-wscript

# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release

# Dependency directories
node_modules
jspm_packages

# Optional npm cache directory
.npm

# Optional REPL history
.node_repl_history
coverage
*.log
npm-debug.log
.logs
logs
*.swp
run
*-run
.idea
.DS_Store
.tmp

.*
!.github
!.eslintignore
!.eslintrc
!.gitignore
!.travis.yml
12 changes: 12 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
sudo: false
language: node_js
node_js:
- '4'
- '6'
- '7'
install:
- npm i npminstall && npminstall
script:
- npm run ci
after_script:
- npminstall codecov && codecov
212 changes: 212 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,214 @@
# cluster-client
Sharing Connection among Multi-Process Nodejs

[![NPM version][npm-image]][npm-url]
[![build status][travis-image]][travis-url]
[![Test coverage][codecov-image]][codecov-url]
[![David deps][david-image]][david-url]
[![Known Vulnerabilities][snyk-image]][snyk-url]
[![npm download][download-image]][download-url]

[npm-image]: https://img.shields.io/npm/v/cluster-client.svg?style=flat-square
[npm-url]: https://npmjs.org/package/cluster-client
[travis-image]: https://img.shields.io/travis/node-modules/cluster-client.svg?style=flat-square
[travis-url]: https://travis-ci.org/node-modules/cluster-client
[codecov-image]: https://codecov.io/gh/node-modules/cluster-client/branch/master/graph/badge.svg
[codecov-url]: https://codecov.io/gh/node-modules/cluster-client
[david-image]: https://img.shields.io/david/node-modules/cluster-client.svg?style=flat-square
[david-url]: https://david-dm.org/node-modules/cluster-client
[snyk-image]: https://snyk.io/test/npm/cluster-client/badge.svg?style=flat-square
[snyk-url]: https://snyk.io/test/npm/cluster-client
[download-image]: https://img.shields.io/npm/dm/cluster-client.svg?style=flat-square
[download-url]: https://npmjs.org/package/cluster-client

As we know, each Node.js process runs in a single thread. Usually, we split a single process into multiple processes to take advantage of multi-core systems. On the other hand, it brings more system overhead, sush as maintaining more TCP connections between servers.

This module is designed to share connections among multi-process Nodejs.

## Theory

- Inspired by [Leader/Follower pattern](http://www.cs.wustl.edu/~schmidt/PDF/lf.pdf).
- Allow ONLY one process "the Leader" to communicate with server. Other processes "the Followers" act as "Proxy" client, and forward all requests to Leader.
- The Leader is selected by "Port Competition". Every process try to listen on a certain port (for example 7777), but ONLY one can occupy the port, then it becomes the Leader, the others become Followers.
- TCP socket connections are maintained between Leader and Followers. And I design a simple communication protocol to exchange data between them.
- If old Leader dies, one of processes will be selected as the new Leader.

## Diagram

normal (without using cluster client)
```js
+--------+ +--------+
| Client | | Client | ...
+--------+ +--------+
| \ / |
| \ / |
| / \ |
| / \ |
+--------+ +--------+
| Server | | Server | ...
+--------+ +--------+

```

using cluster-client
```js
+-------+
| start |
+---+---+
|
+--------+---------+
__| port competition |__
win / +------------------+ \ lose
/ \
+--------+ tcp conn +----------+
| Leader |<---------------->| Follower |
+--------+ +----------+
|
+--------+
| Client |
+--------+
| \
| \
| \
| \
+--------+ +--------+
| Server | | Server | ...
+--------+ +--------+

```

## Protocol

- Packet structure
```js
0 1 2 4 12
+-------+-------+---------------+---------------------------------------------------------------+
|version|req/res| reserved | request id |
+-------------------------------+-------------------------------+-------------------------------+
| timeout | connection object length | application object length |
+-------------------------------+---------------------------------------------------------------+
| conn object (JSON format) ... | app object |
+-----------------------------------------------------------+ |
| ... |
+-----------------------------------------------------------------------------------------------+
```
- Protocol Type
- Register Channel
- Subscribe/Publish
- Invoke
- Sequence diagram

```js
+----------+ +---------------+ +---------+
| Follower | | local server | | Leader |
+----------+ +---------------+ +---------+
| register channel | assign to |
+ -----------------------> | --------------------> |
| | |
| subscribe |
+ ------------------------------------------------> |
| subscribe result |
| <------------------------------------------------ +
| |
| invoke |
+ ------------------------------------------------> |
| invoke result |
| <------------------------------------------------ +
| |
```

## Install

```bash
$ npm install cluster-client --save
```

Node.js >= 4.0.0 required

## Usage

```js
'use strict';

const co = require('co');
const Base = require('sdk-base');
const cluster = require('cluster-client');

/**
* Client Example
*/
class YourClient extends Base {
constructor(options) {
super(options);

this.options = options;
this.ready(true);
}

subscribe(reg, listener) {
// subscribe logic
}

publish(reg) {
// publish logic
}

* getData(id) {
// invoke api
}

getDataCallback(id, cb) {
// ...
}

getDataPromise(id) {
// ...
}
}

// create some client instances, but only one instance will connect to server
const client_1 = cluster(YourClient)
.delegate('getData')
.delegate('getDataCallback')
.delegate('getDataPromise')
.create({ foo: 'bar' });
const client_2 = cluster(YourClient)
.delegate('getData')
.delegate('getDataCallback')
.delegate('getDataPromise')
.create({ foo: 'bar' });
const client_3 = cluster(YourClient)
.delegate('getData')
.delegate('getDataCallback')
.delegate('getDataPromise')
.create({ foo: 'bar' });;

// subscribe information
client_1.subsribe('some thing', result => console.log(result));
client_2.subsribe('some thing', result => console.log(result));
client_3.subsribe('some thing', result => console.log(result));

// publish data
client_2.publish('some data');

// invoke method
client_3.getDataCallback('some thing', (err, val) => console.log(val));
client_2.getPromise('some thing').then(val => console.log(val));

co(function*() {
const ret = yield client_1.getData('some thing');
console.log(ret);
}).catch(err => console.error(err));
```

## API

- `delegate(from, to)`:
create delegate method, `from` is the method name your want to create, and `to` have 3 possible values: subscribe, publish, and invoke, the default value is invoke
- `override(name, value)`:
override one property
- `create(…)`
create the client instance


[MIT](LICENSE)
16 changes: 16 additions & 0 deletions appveyor.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
environment:
matrix:
- nodejs_version: '4'
- nodejs_version: '6'
- nodejs_version: '7'

install:
- ps: Install-Product node $env:nodejs_version
- npm i npminstall && node_modules\.bin\npminstall

test_script:
- node --version
- npm --version
- npm run ci

build: off
Loading

0 comments on commit 0299b68

Please sign in to comment.