Memory leak? #804
Comments
|
Tested in node 4.4.7 and node 6.3.1, the result is similar. |
|
Facing the same issue. any help? |
|
Looks like the clients array is still full, could you guys log |
|
|
|
i've done some tests overnight.
'use strict';
const WS = require('ws');
let counter = 0;
(function create() {
const ws = new WS('ws://10.11.241.244:8081');
const data = 'somedata';
ws.on('error', err => console.error(err));
ws.on('open', open);
function open() {
ws.send(data);
process.stdout.write('.');
if (++counter < 5000) {
create();
}
}
})();
'use strict';
const WebSocketServer = require('ws').Server;
const wss = new WebSocketServer({ port: 8081 });
let connections = 0;
let messages = 0;
wss.on('connection', function connection(ws) {
connections++;
ws.on('message', function incoming(message) {
messages++;
});
ws.on('close', function () {
connections--;
})
ws.send('something');
});
(function report() {
const usage = process.memoryUsage();
for (let key of Object.keys(usage)) {
usage[key] = Math.round(usage[key] / 1024 / 1024) + 'MB';
}
console.log(new Date());
console.log(' MEM:', usage);
console.log(' STATS:', { connections, messages, clients: wss.clients.length });
console.log('');
setTimeout(report, 5000);
})();process
before each test server was restarted and code was modified (only ws.send commented or uncommented on client or/and server side) results20k connections; no messages (ws.send commented on both client and server) 20k connections; 20k incoming messages (client -> server); 1 message per connection 20k connections; 20k outgoing messages (server -> client); 1 message per connection 20k connections; 20k incoming and 20k outgoing messages; 1in/1out message per connection seems like something doggy is going on in |
|
also, if I run a code which:
'use strict';
const WS = require('ws');
(function create() {
const ws = new WS('ws://10.11.241.244:8081');
const data = 'somedata';
ws.on('error', err => console.error(err));
ws.on('open', open);
function open() {
ws.send(data);
process.stdout.write('.');
create();
}
setTimeout(function () {
ws.close();
}, 30000);
})();the memory (on both sides!) grows indefinitely until process is killed by os https://jpst.it/MzKj |
|
I faced same issue here. This code has issue: and This code has NO issue: The leaked memory is not in GC Collection... really weird... |
|
I have managed to reduce leakage by setting 'perMessageDeflate: false'. I've dumped rss mem and found a weird endless recursive call in perMessageDeflate->zlib->_onTimeout->_onTimeout->... method (I don't remember now if these are exact method names, but at least you have a guidance). Didn't had time to dig deeper into zlib implementation. |
|
I can confirm and reproduce the issue as described by @3490. |
|
this seems like a pretty major issue... is there an ETA for a fix? or is this is an issue in node proper? |
|
@leebenson I agree but I think there is not much we can do about it as this is an external issue. |
|
Maybe we should just disable permessage-deflate by default. |
|
Even if disable permessage-deflate ,there still Memory leak, just less. for example var WebSocket = require('ws'); open the server ,it cost memory usage is 20 MB |
|
@binginto on the first run this is normal, you have to wait for it "stabilize". Run it multiple times, if memory does not grow every time, then it works as expected. |
|
I dont know what means 'stabilize',is the 60MB memory will be clean by GC after a period of time? Now, i found the Situation is if it "stabilize",if connnet and close severtime. the memory does not grow every time.just oppuied 60MB at first time and it not decrease in 10 min. And it grows with the number of the clients. for example,it 20K clients connected and closed. it oppuied 60MB and will not clean by gc. if 5k clients, it just it oppuied 20MB and will not clean by gc. and if 50K clients ,it oppuied 140MB and not decrease in 10 min. And I dont know Is this Situation works as expected and it isnt Memory leak? |
|
@binginto here is an example: server.js'use strict';
const WebSocket = require('ws');
const wss = new WebSocket.Server({
perMessageDeflate: false,
port: 3000
});
wss.on('connection', (ws) => ws.send('something'));
const report = () => {
gc();
const rss = process.memoryUsage().rss / 1024 / 1024;
console.log('clients: %d, rss: %d', wss.clients.size, rss);
};
setInterval(report, 30000);
report();client.js'use strict';
const WebSocket = require('ws');
let cnt = 0;
(function create() {
const ws = new WebSocket('ws://localhost:3000');
ws.on('error', err => console.error(err));
ws.on('open', () => {
ws.send('something');
if (++cnt !== 20000) create();
});
})();This is what I get by starting and killing the process that runs the client multiple times: As you can see memory usage does not grow every time and is stable at ~180 max / ~60 min MiB. |
|
@lpinca why if a new client connect to server and then closed it .it still oppuied some space ? If the server cant clean it any more . Now ,it cause a problem , there are many new client connnect to the server and then close. This action cost some memory, as time when by , the server will lost all memory and have to restart . Is it true ? if it is true ,the memory leak still exist , is it right? |
No, once it is GC'ed the retained memory is freed. You can also verify this by taking heap snapshots.
No, unless proven otherwise, there is no leak. P.S. if you run the above example on Linux with permessage-deflate enabled ( |
|
@lpinca server var WebSocket = require('ws');
var wss = new WebSocket.Server({
perMessageDeflate: false,
port:8001
});
wss.on('connection', function connection(ws) {
ws.on('message', function incoming(message) {
console.log('received: %s', message);
});
ws.send('something');
});client var WebSocket = require('ws');
function cerateclient(){
var ws = new WebSocket("ws://localhost:8001/");
ws.onopen = function() {
ws.send('connnect');
};
ws.onmessage = function(evt) {
console.log(evt.data);
};
}
for(var i =0 ; i < 200000 || i == 20000; i++){
cerateclient();
}why after 20000 times connect and close ? Then wait serval hours ,many times of gc cleaned no used memory. The rss still will not back to the original level, and it still larger than the original level .Also, the |
|
It won't go back to the startup level (~20 MiB), this is how it works. |
|
Partially fixed by #1204, closing. |
my code:
When 20000 clients connected, memory usage is 2.864 GB.
Then close all clients,
After a few minute, memory usage is 2.188GB.
And, after 20000 new clients connected, memory usage is 4.137 GB.
It looks like that memory did not free.
The text was updated successfully, but these errors were encountered: