Skip to content

Commit

Permalink
Merge 094db17 into c7fbaeb
Browse files Browse the repository at this point in the history
  • Loading branch information
yunnysunny committed Nov 16, 2019
2 parents c7fbaeb + 094db17 commit f3ee805
Show file tree
Hide file tree
Showing 16 changed files with 44 additions and 139 deletions.
6 changes: 6 additions & 0 deletions changelog.md
@@ -1,3 +1,9 @@
# v1.0.0
## Breaking Changes

1. The parameter of `subReis` has been renamed to `subRedis`.
2. Removing all code associated with lru.

# v0.6.2
## Remove

Expand Down
4 changes: 2 additions & 2 deletions docs/api.md
Expand Up @@ -153,11 +153,11 @@ The callback function, which will be called when data is cleared.
| [expireTime] | <code>Number</code> | <code>0</code> | Expiration time in second, default is 0, which will be not expired, but when you set the paramter of `option.crontabStr`, it will be cleared at some time still. |
| redisKeyPrefix | <code>String</code> | | The prefix of redis key to save session data |
| reids | <code>Object</code> | | The redis client used to save session data |
| [subReis] | <code>Object</code> | | The subscribe redis client to receive delete operation form other node.js process, it's useful when you start node in cluster mode. |
| [subRedis] | <code>Object</code> | | The subscribe redis client to receive delete operation form other node.js process, it's useful when you start node in cluster mode. |
| [crontabStr] | <code>String</code> | | Crontab string, use for clearing the memeory cache. |
| [maxSize] | <code>Number</code> | <code>0</code> | The max size of the cache in memory, default is 0, which will not limit the size of cache in memory. When it passed as -1, the cache in memory will be disabled. |
| [useLru] | <code>Boolean</code> | <code>false</code> | Whether to use LRU algorithm to delete the elder elements. It only takes effect when `option.maxSize` is greater than zero. |
| [clusteId] | <code>String</code> | | An id of current process, when not set, it will use random string. When do the operation of delete or update, SessionToken will publish a message, which is started with a perfix of current clusterId, to redis. Then all the processes will receive the message and read the clusterId of the message to check whether it from self. But when the subReis is not set, the `clusterId` is useless now. |
| [clusteId] | <code>String</code> | | An id of current process, when not set, it will use random string. When do the operation of delete or update, SessionToken will publish a message, which is started with a perfix of current clusterId, to redis. Then all the processes will receive the message and read the clusterId of the message to check whether it from self. But when the subRedis is not set, the `clusterId` is useless now. |
| [subscribeCallback] | [<code>SubscribeCallback</code>](#SubscribeCallback) | | The callback function , which will be triggered when new message form redis subscription. |
| [cacheWriteCallback] | [<code>CacheWriteCallback</code>](#CacheWriteCallback) | | |
| [cacheClearCallback] | [<code>CacheClearCallback</code>](#CacheClearCallback) | | |
Expand Down
53 changes: 13 additions & 40 deletions lib/SessionToken.js
@@ -1,7 +1,6 @@
const crypto = require('crypto');
const slogger = require('node-slogger');
const async = require('neo-async');
// const LRUList = require('native-linked-list').LRUList;
const cronNode = require('node-cron');

const REDIS_KEY_SUB_SESSION_TOKEN = 'pubSubSessionToken:';
Expand Down Expand Up @@ -50,10 +49,10 @@ class TokenCacheItem {
* @param {Number} [expireTime=0] Expiration time in second, default is 0, which will be not expired, but when you set the paramter of `option.crontabStr`, it will be cleared at some time still.
* @param {String} redisKeyPrefix The prefix of redis key to save session data
* @param {Object} reids The redis client used to save session data
* @param {Object=} subReis The subscribe redis client to receive delete operation form other node.js process, it's useful when you start node in cluster mode.
* @param {Object=} subRedis The subscribe redis client to receive delete operation form other node.js process, it's useful when you start node in cluster mode.
* @param {String=} crontabStr Crontab string, use for clearing the memeory cache.
* @param {Number} [maxSize=0] The max size of the cache in memory, default is 0, which will not limit the size of cache in memory. When it passed as -1, the cache in memory will be disabled.
* @param {String=} clusteId An id of current process, when not set, it will use random string. When do the operation of delete or update, SessionToken will publish a message, which is started with a perfix of current clusterId, to redis. Then all the processes will receive the message and read the clusterId of the message to check whether it from self. But when the subReis is not set, the `clusterId` is useless now.
* @param {String=} clusteId An id of current process, when not set, it will use random string. When do the operation of delete or update, SessionToken will publish a message, which is started with a perfix of current clusterId, to redis. Then all the processes will receive the message and read the clusterId of the message to check whether it from self. But when the subRedis is not set, the `clusterId` is useless now.
* @param {SubscribeCallback=} subscribeCallback The callback function , which will be triggered when new message form redis subscription.
* @param {CacheWriteCallback=} cacheWriteCallback
* @param {CacheClearCallback=} cacheClearCallback
Expand All @@ -79,7 +78,7 @@ class TokenCacheItem {
* @returns {SessionToken}
* @constructor
*/
function SessionToken({expireTime, redisKeyPrefix, redis, subReis,crontabStr,maxSize,clusteId,subscribeCallback,cacheWriteCallback,cacheClearCallback,showMemSizeInterval,idleCheckInterval,idleCheckPerCount,memLifecycleRatio}) {
function SessionToken({expireTime, redisKeyPrefix, redis, subRedis,crontabStr,maxSize,clusteId,subscribeCallback,cacheWriteCallback,cacheClearCallback,showMemSizeInterval,idleCheckInterval,idleCheckPerCount,memLifecycleRatio}) {
this.expireTime = expireTime;
this._memLifecycleRatio = Math.min(parseFloat(memLifecycleRatio) || 1, 1);
this.redisKeyPrefix = redisKeyPrefix;
Expand All @@ -89,15 +88,10 @@ function SessionToken({expireTime, redisKeyPrefix, redis, subReis,crontabStr,max
if (crontabStr) {
this._addClearHandler(crontabStr);
}
this.subReis = subReis;
this.subRedis = subRedis;
this.pubSubName = REDIS_KEY_SUB_SESSION_TOKEN + redisKeyPrefix;
this.maxSize = Number(maxSize) || 0;
this.useLru = false;
this._lruList = null;
// if (this.useLru && this.maxSize) {
// this._lruList = new LRUList(this.maxSize);
// this._addToLru('1th');
// }

this._subscribePromise = null;
this._subscribeSuccess = false;
this.clusterId = clusteId || crypto.randomBytes(RAND_BYTES).toString('hex');
Expand Down Expand Up @@ -150,9 +144,6 @@ SessionToken.prototype._idleCheck = function() {
if (value.expire <= now) {
_data.delete(token);
//slogger.trace(token + 'is expired',_data.size);
if (_this.useLru) {
this._lruList.remove(token);
}
}
}
},this._idleCheckInterval);
Expand Down Expand Up @@ -200,16 +191,16 @@ SessionToken.prototype._parseMessage = function(originalMessage) {
};

SessionToken.prototype._subRedisMessage = function() {
if (!this.subReis) {
if (!this.subRedis) {
return;
}
this._subscribePromise = this.subReis.subscribe(this.pubSubName);
this._subscribePromise = this.subRedis.subscribe(this.pubSubName);
// const data = this.data;
const _this = this;
if (typeof(this._subscribeCallback) !== 'function') {
this._subscribeCallback = function() {};
}
this.subReis.on('message',function(channel, message) {
this.subRedis.on('message',function(channel, message) {
const result = _this._parseMessage( message );
const clusterNow = result.clusterId;
if (_this.clusterId === clusterNow) {
Expand Down Expand Up @@ -261,28 +252,19 @@ SessionToken.prototype._saveTokenToMem = function(token,value,expireTime=0) {
this._historyDataSize++;
};

SessionToken.prototype._addToLru = function(token) {
const _this = this;
return this._lruList.addOne(token,function(isTailRemoved,removedTailValue) {
if (isTailRemoved) {
_this.data.delete(removedTailValue);
}
});
};


SessionToken.prototype._addCache = function(token,value,expireTime=0) {
if (this.maxSize === -1) {//the cache in memeory is disabled
return;
}
if (this.maxSize > 0) {//with a fixed size limit
if (this.useLru) {
this._addToLru(token);
}
if (this.data.size < this.maxSize || this.useLru) {

if (this.data.size < this.maxSize) {
return this._saveTokenToMem(token,value,expireTime);
}

slogger.debug('maxsize overflow and not use lru algorithm', this.data.size , this.maxSize);
slogger.debug('maxsize overflow', this.data.size , this.maxSize);
} else {//without limit
this._saveTokenToMem(token,value,expireTime);
}
Expand Down Expand Up @@ -431,9 +413,7 @@ SessionToken.prototype.get = function (token, callback) {
return callback(null,item.session);
}
_self.data.delete(token);
if (_self.useLru) {
_self._lruList.remove(token);
}

return callback(null,false);
},
function (sessionValue,callback) {
Expand Down Expand Up @@ -464,9 +444,6 @@ SessionToken.prototype.get = function (token, callback) {
// console.log('get session',value);
if (hitCache) {//the data is get from memory
callback(null, value,hitCache);
if (this.useLru) {
this._addToLru(token);
}
return ;
}//console.log(`cache miss:${seq}`);
// if (_self.expireTime > 0) {
Expand Down Expand Up @@ -506,10 +483,6 @@ SessionToken.prototype.delete = function(token,callback) {
return callback('delete token associated data failed');
}
_this.data.delete(token);

if (_this.useLru) {
_this._lruList.remove(token);
}

if (Number(reply) === 1) {
_this._pubRedisMessage(OPERATION_DEL+','+token);
Expand Down
2 changes: 1 addition & 1 deletion package.json
@@ -1,6 +1,6 @@
{
"name": "session-token",
"version": "0.6.2",
"version": "1.0.0",
"description": "A library to create token in redis easily.",
"main": "index.js",
"scripts": {
Expand Down
2 changes: 1 addition & 1 deletion readme.md
Expand Up @@ -36,7 +36,7 @@ const sessionToken = new SessionToken({
expireTime:7200,//the time of seconds before the session data expired
redisKeyPrefix:'myprefix:mytoken:',//the redis key's prefix
redis:redisClient,//the redis client used to save session data
subReis:redisSub,//The subscribe redis client to receive delete operation
subRedis:redisSub,//The subscribe redis client to receive delete operation
maxSize:1000000,// The max size of the cache in memory.
});
```
Expand Down
2 changes: 1 addition & 1 deletion test/basic_test.js
Expand Up @@ -7,7 +7,7 @@ const sessionToken = new SessionToken({
expireTime:7200,//the time of seconds before the session data expired
redisKeyPrefix:'myprefix:mytoken:',//the redis key's prefix
redis:redisClient,//the redis client object
subReis:redisSub
subRedis:redisSub
});
const VALUE = {name:'sunny',id:1};
const VALUE_UPDATE = {name:'sunny_new',id:1};
Expand Down
74 changes: 0 additions & 74 deletions test/bench_lru_test.js

This file was deleted.

22 changes: 11 additions & 11 deletions test/bench_none_lru_test.js → test/bench_test.js
Expand Up @@ -11,45 +11,45 @@ const sessionTokenWithoutLru = new SessionToken({
expireTime:7200,//the time of seconds before the session data expired
redisKeyPrefix:'myprefix:mynolrutoken:',//the redis key's prefix
redis:redisClient,//the redis client object
subReis:redisSub,
subRedis:redisSub,
maxSize:MAX_SIZE,
//showMemSizeInterval:10*1000
});

const VALUE = {name:'sunny',id:1};
const LOOP_SIZE = 102400;
const LruToken = new Array(LOOP_SIZE);
const arrayToken = new Array(LOOP_SIZE);
const GET_LOOP_SIZE = LOOP_SIZE / 10;


describe('none lru benchmark test',function() {
describe('benchmark test',function() {
before(function() {
slogger.init({level:'warn'});
});

it('bench for none lur',function(done) {
it('bench for generate',function(done) {
async.times(LOOP_SIZE,function(n,next) {
sessionTokenWithoutLru.generate(VALUE,function(err,tokenViaCreate) {//save session
if (err) {
return next(err);
}
LruToken[n] = tokenViaCreate;
arrayToken[n] = tokenViaCreate;
next();
});
},done);
});
it('get test without lru',function(done) {
it('get test',function(done) {
async.times(GET_LOOP_SIZE,function(n,next) {
sessionTokenWithoutLru.get(LruToken[GET_LOOP_SIZE-1-n],next);
sessionTokenWithoutLru.get(arrayToken[GET_LOOP_SIZE-1-n],next);
},done);
});
it('get test without lru again',function(done) {
it('get test again',function(done) {
async.times(GET_LOOP_SIZE,function(n,next) {
sessionTokenWithoutLru.get(LruToken[n],next);
sessionTokenWithoutLru.get(arrayToken[n],next);
},done);
});
it('remove all data create via none lru session token',function(done) {
async.each(LruToken,function(token,next) {
it('remove all data',function(done) {
async.each(arrayToken,function(token,next) {
sessionTokenWithoutLru.delete(token,next);
},done);
});
Expand Down
2 changes: 1 addition & 1 deletion test/cluster/cluster_test.js
Expand Up @@ -12,7 +12,7 @@ const sessionToken = new SessionToken({
expireTime:7200,//the time of seconds before the session data expired
redisKeyPrefix:'mycluster:mytoken:',//the redis key's prefix
redis:redisClient,//the redis client object
subReis:redisSub
subRedis:redisSub
});
console.log(process.execArgv);

Expand Down
2 changes: 1 addition & 1 deletion test/cluster/worker.js
Expand Up @@ -9,7 +9,7 @@ const sessionToken = new SessionToken({
expireTime:7200,//the time of seconds before the session data expired
redisKeyPrefix:'mycluster:mytoken:',//the redis key's prefix
redis:redisClient,//the redis client object
subReis:redisSub,
subRedis:redisSub,
subscribeCallback:function(operation,token,value) {
console.log('subscribe',operation,token,value,'data after subscription :',sessionToken.data);
},
Expand Down
2 changes: 1 addition & 1 deletion test/disabled_memory_test.js
Expand Up @@ -7,7 +7,7 @@ const sessionToken = new SessionToken({
expireTime:7200,//the time of seconds before the session data expired
redisKeyPrefix:'disabled_mem:mytoken:',//the redis key's prefix
redis:redisClient,//the redis client object
subReis:redisSub,
subRedis:redisSub,
maxSize:-1
});
const VALUE = {name:'sunny',id:1};
Expand Down
2 changes: 1 addition & 1 deletion test/fixed_size_test.js
Expand Up @@ -9,7 +9,7 @@ const sessionToken = new SessionToken({
expireTime:7200,//the time of seconds before the session data expired
redisKeyPrefix:'myprefix:mytoken:',//the redis key's prefix
redis:redisClient,//the redis client object
subReis:redisSub,
subRedis:redisSub,
maxSize:MAX_SIZE,
// useLru:true
});
Expand Down
4 changes: 2 additions & 2 deletions test/idle_check_test.js
Expand Up @@ -15,7 +15,7 @@ const sessionTokenWithLru = new SessionToken({
expireTime:1,//the time of seconds before the session data expired
redisKeyPrefix:'myprefix:idletoken:',//the redis key's prefix
redis:redisClient,//the redis client object
subReis:redisSub,
subRedis:redisSub,
maxSize:MAX_SIZE,
// useLru:true,
idleCheckInterval:1000,
Expand All @@ -26,7 +26,7 @@ const sessionTokenWithLru = new SessionToken({
// const GET_LOOP_SIZE = LOOP_SIZE / 10;


describe.only('idle check test',function() {
describe('idle check test',function() {
before(function() {
slogger.init({level:'debug'});
});
Expand Down
2 changes: 1 addition & 1 deletion test/mem_lifecycle_test.js
Expand Up @@ -14,7 +14,7 @@ const sessionToken = new SessionToken({
expireTime:REDIS_EXPIRE,//the time of seconds before the session data expired
redisKeyPrefix:'myprefix:ratiotoken:',//the redis key's prefix
redis:redisClient,//the redis client object
subReis:redisSub,
subRedis:redisSub,
maxSize:MAX_SIZE,
idleCheckInterval:1000,
memLifecycleRatio:LIFE_RATIO
Expand Down
2 changes: 1 addition & 1 deletion test/no_expired_test.js
Expand Up @@ -9,7 +9,7 @@ const sessionToken = new SessionToken({
expireTime:0,//the time of seconds before the session data expired
redisKeyPrefix:'long:mytoken:',//the redis key's prefix
redis:redisClient,//the redis client object
subReis:redisSub,
subRedis:redisSub,
maxSize:MAX_SIZE,
// useLru:true
});
Expand Down

0 comments on commit f3ee805

Please sign in to comment.