-
Notifications
You must be signed in to change notification settings - Fork 15
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
32 changed files
with
2,662 additions
and
38 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
test/fixtures | ||
examples/**/app/public | ||
logs | ||
run |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
{ | ||
"extends": "eslint-config-egg" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
<!-- | ||
Thank you for your pull request. Please review below requirements. | ||
Bug fixes and new features should include tests and possibly benchmarks. | ||
Contributors guide: https://github.com/eggjs/egg/blob/master/CONTRIBUTING.md | ||
感谢您贡献代码。请确认下列 checklist 的完成情况。 | ||
Bug 修复和新功能必须包含测试,必要时请附上性能测试。 | ||
Contributors guide: https://github.com/eggjs/egg/blob/master/CONTRIBUTING.md | ||
--> | ||
|
||
##### Checklist | ||
<!-- Remove items that do not apply. For completed items, change [ ] to [x]. --> | ||
|
||
- [ ] `npm test` passes | ||
- [ ] tests and/or benchmarks are included | ||
- [ ] documentation is changed or added | ||
- [ ] commit message follows commit guidelines | ||
|
||
##### Affected core subsystem(s) | ||
<!-- Provide affected core subsystem(s). --> | ||
|
||
|
||
##### Description of change | ||
<!-- Provide a description of the change below this comment. --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,37 +1,19 @@ | ||
# Logs | ||
logs | ||
*.log | ||
npm-debug.log* | ||
|
||
# Runtime data | ||
pids | ||
*.pid | ||
*.seed | ||
|
||
# Directory for instrumented libs generated by jscoverage/JSCover | ||
lib-cov | ||
|
||
# Coverage directory used by tools like istanbul | ||
coverage | ||
|
||
# nyc test coverage | ||
.nyc_output | ||
|
||
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files) | ||
.grunt | ||
|
||
# node-waf configuration | ||
.lock-wscript | ||
|
||
# Compiled binary addons (http://nodejs.org/api/addons.html) | ||
build/Release | ||
|
||
# Dependency directories | ||
node_modules | ||
jspm_packages | ||
|
||
# Optional npm cache directory | ||
.npm | ||
|
||
# Optional REPL history | ||
.node_repl_history | ||
coverage | ||
*.log | ||
npm-debug.log | ||
.logs | ||
logs | ||
*.swp | ||
run | ||
*-run | ||
.idea | ||
.DS_Store | ||
.tmp | ||
|
||
.* | ||
!.github | ||
!.eslintignore | ||
!.eslintrc | ||
!.gitignore | ||
!.travis.yml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
sudo: false | ||
language: node_js | ||
node_js: | ||
- '4' | ||
- '6' | ||
- '7' | ||
install: | ||
- npm i npminstall && npminstall | ||
script: | ||
- npm run ci | ||
after_script: | ||
- npminstall codecov && codecov |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,214 @@ | ||
# cluster-client | ||
Sharing Connection among Multi-Process Nodejs | ||
|
||
[![NPM version][npm-image]][npm-url] | ||
[![build status][travis-image]][travis-url] | ||
[![Test coverage][codecov-image]][codecov-url] | ||
[![David deps][david-image]][david-url] | ||
[![Known Vulnerabilities][snyk-image]][snyk-url] | ||
[![npm download][download-image]][download-url] | ||
|
||
[npm-image]: https://img.shields.io/npm/v/cluster-client.svg?style=flat-square | ||
[npm-url]: https://npmjs.org/package/cluster-client | ||
[travis-image]: https://img.shields.io/travis/node-modules/cluster-client.svg?style=flat-square | ||
[travis-url]: https://travis-ci.org/node-modules/cluster-client | ||
[codecov-image]: https://codecov.io/gh/node-modules/cluster-client/branch/master/graph/badge.svg | ||
[codecov-url]: https://codecov.io/gh/node-modules/cluster-client | ||
[david-image]: https://img.shields.io/david/node-modules/cluster-client.svg?style=flat-square | ||
[david-url]: https://david-dm.org/node-modules/cluster-client | ||
[snyk-image]: https://snyk.io/test/npm/cluster-client/badge.svg?style=flat-square | ||
[snyk-url]: https://snyk.io/test/npm/cluster-client | ||
[download-image]: https://img.shields.io/npm/dm/cluster-client.svg?style=flat-square | ||
[download-url]: https://npmjs.org/package/cluster-client | ||
|
||
As we know, each Node.js process runs in a single thread. Usually, we split a single process into multiple processes to take advantage of multi-core systems. On the other hand, it brings more system overhead, sush as maintaining more TCP connections between servers. | ||
|
||
This module is designed to share connections among multi-process Nodejs. | ||
|
||
## Theory | ||
|
||
- Inspired by [Leader/Follower pattern](http://www.cs.wustl.edu/~schmidt/PDF/lf.pdf). | ||
- Allow ONLY one process "the Leader" to communicate with server. Other processes "the Followers" act as "Proxy" client, and forward all requests to Leader. | ||
- The Leader is selected by "Port Competition". Every process try to listen on a certain port (for example 7777), but ONLY one can occupy the port, then it becomes the Leader, the others become Followers. | ||
- TCP socket connections are maintained between Leader and Followers. And I design a simple communication protocol to exchange data between them. | ||
- If old Leader dies, one of processes will be selected as the new Leader. | ||
|
||
## Diagram | ||
|
||
normal (without using cluster client) | ||
```js | ||
+--------+ +--------+ | ||
| Client | | Client | ... | ||
+--------+ +--------+ | ||
| \ / | | ||
| \ / | | ||
| / \ | | ||
| / \ | | ||
+--------+ +--------+ | ||
| Server | | Server | ... | ||
+--------+ +--------+ | ||
|
||
``` | ||
|
||
using cluster-client | ||
```js | ||
+-------+ | ||
| start | | ||
+---+---+ | ||
| | ||
+--------+---------+ | ||
__| port competition |__ | ||
win / +------------------+ \ lose | ||
/ \ | ||
+--------+ tcp conn +----------+ | ||
| Leader |<---------------->| Follower | | ||
+--------+ +----------+ | ||
| | ||
+--------+ | ||
| Client | | ||
+--------+ | ||
| \ | ||
| \ | ||
| \ | ||
| \ | ||
+--------+ +--------+ | ||
| Server | | Server | ... | ||
+--------+ +--------+ | ||
|
||
``` | ||
|
||
## Protocol | ||
|
||
- Packet structure | ||
```js | ||
0 1 2 4 12 | ||
+-------+-------+---------------+---------------------------------------------------------------+ | ||
|version|req/res| reserved | request id | | ||
+-------------------------------+-------------------------------+-------------------------------+ | ||
| timeout | connection object length | application object length | | ||
+-------------------------------+---------------------------------------------------------------+ | ||
| conn object (JSON format) ... | app object | | ||
+-----------------------------------------------------------+ | | ||
| ... | | ||
+-----------------------------------------------------------------------------------------------+ | ||
``` | ||
- Protocol Type | ||
- Register Channel | ||
- Subscribe/Publish | ||
- Invoke | ||
- Sequence diagram | ||
|
||
```js | ||
+----------+ +---------------+ +---------+ | ||
| Follower | | local server | | Leader | | ||
+----------+ +---------------+ +---------+ | ||
| register channel | assign to | | ||
+ -----------------------> | --------------------> | | ||
| | | | ||
| subscribe | | ||
+ ------------------------------------------------> | | ||
| subscribe result | | ||
| <------------------------------------------------ + | ||
| | | ||
| invoke | | ||
+ ------------------------------------------------> | | ||
| invoke result | | ||
| <------------------------------------------------ + | ||
| | | ||
``` | ||
|
||
## Install | ||
|
||
```bash | ||
$ npm install cluster-client --save | ||
``` | ||
|
||
Node.js >= 4.0.0 required | ||
|
||
## Usage | ||
|
||
```js | ||
'use strict'; | ||
|
||
const co = require('co'); | ||
const Base = require('sdk-base'); | ||
const cluster = require('cluster-client'); | ||
|
||
/** | ||
* Client Example | ||
*/ | ||
class YourClient extends Base { | ||
constructor(options) { | ||
super(options); | ||
|
||
this.options = options; | ||
this.ready(true); | ||
} | ||
|
||
subscribe(reg, listener) { | ||
// subscribe logic | ||
} | ||
|
||
publish(reg) { | ||
// publish logic | ||
} | ||
|
||
* getData(id) { | ||
// invoke api | ||
} | ||
|
||
getDataCallback(id, cb) { | ||
// ... | ||
} | ||
|
||
getDataPromise(id) { | ||
// ... | ||
} | ||
} | ||
|
||
// create some client instances, but only one instance will connect to server | ||
const client_1 = cluster(YourClient) | ||
.delegate('getData') | ||
.delegate('getDataCallback') | ||
.delegate('getDataPromise') | ||
.create({ foo: 'bar' }); | ||
const client_2 = cluster(YourClient) | ||
.delegate('getData') | ||
.delegate('getDataCallback') | ||
.delegate('getDataPromise') | ||
.create({ foo: 'bar' }); | ||
const client_3 = cluster(YourClient) | ||
.delegate('getData') | ||
.delegate('getDataCallback') | ||
.delegate('getDataPromise') | ||
.create({ foo: 'bar' });; | ||
|
||
// subscribe information | ||
client_1.subsribe('some thing', result => console.log(result)); | ||
client_2.subsribe('some thing', result => console.log(result)); | ||
client_3.subsribe('some thing', result => console.log(result)); | ||
|
||
// publish data | ||
client_2.publish('some data'); | ||
|
||
// invoke method | ||
client_3.getDataCallback('some thing', (err, val) => console.log(val)); | ||
client_2.getPromise('some thing').then(val => console.log(val)); | ||
|
||
co(function*() { | ||
const ret = yield client_1.getData('some thing'); | ||
console.log(ret); | ||
}).catch(err => console.error(err)); | ||
``` | ||
|
||
## API | ||
|
||
- `delegate(from, to)`: | ||
create delegate method, `from` is the method name your want to create, and `to` have 3 possible values: subscribe, publish, and invoke, the default value is invoke | ||
- `override(name, value)`: | ||
override one property | ||
- `create(…)` | ||
create the client instance | ||
|
||
|
||
[MIT](LICENSE) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
environment: | ||
matrix: | ||
- nodejs_version: '4' | ||
- nodejs_version: '6' | ||
- nodejs_version: '7' | ||
|
||
install: | ||
- ps: Install-Product node $env:nodejs_version | ||
- npm i npminstall && node_modules\.bin\npminstall | ||
|
||
test_script: | ||
- node --version | ||
- npm --version | ||
- npm run ci | ||
|
||
build: off |
Oops, something went wrong.