Memory leak in 55

[gnorbsl]

I noticed out of memory errors on our testserver and found out that one script is filling the heap. I got some heapdumps and it looks like web3 is spamming strings.

I don't exactly know how to properly debug memory leaks but it looks like web3 is the cause:

Any advise how to proceed would be appreciated.

[gnorbsl]

After a bit of digging this seems to be connected to getTransaction. I subscribe to pending transactions and call getTransaction() on every hash. This seems to slowly fill up the memory.

[nivida]

Could you please add the related code and details about your environment (versions etc.)?

[gnorbsl]

 web3.eth.subscribe('pendingTransactions', function (error, transactionHash) {
}).on('data', transactionHash => {
    web3.eth.getTransaction(transactionHash)
        .then(transaction => {//dostuff })     
        .catch(error => console.log(error))
})

node: v8.11.2
web3 55
Debian Jessie

[sirpy]

I also think i'm experiencing a memory leak on the server, every time a TX happens.
I dont subscribe to anything simply execute it and i use promises to handle the response ie catch/then

return this.web3.eth.sendTransaction({
            from: this.address,
            to: address,
            value: toTop,
            gas: 100000,
            gasPrice: Web3.utils.toWei('1', 'gwei')
          })

Every TX there's a memory spike it then goes down a little but still adds 15-30mb.
Gonna try to reproduce it on a local server now

[sshelton76]

@nivida This is the culprit I think
https://github.com/ethereum/web3.js/blob/1.0/packages/web3-core-subscriptions/lib/subscriptions/AbstractSubscription.js

You are using "isFunction" from lodash. Lodash in general tends to leak in webpack.
Replace that with...

if (callback instanceof Function) {
       dosomething
    }

And I bet this problem goes away. Don't have time to check it, but importing from lodash into something designed for webpack is a well known source of memory leaks in my experience. My solution has been to avoid using both lodash and webpack like the plague, but YMMV. Meantime, this is something you should check since it's non-obvious.

[sshelton76]

@gnorbsl Don't mix callbacks and promises, you'll encounter all kinds of issues and this is only the start. I recommend you use one or the other, but not both. My strong preference is for promises since @nivida wants to ditch callbacks ASAP.

[nivida]

@sshelton76 Yes :)

I've optimized the websocket handling and subscriptions handling for having no memory leak and I'm currently checking the whole lib. The current conclusion is that EventEmitters are definitely the wrong way to handle subscriptions. I will drop a comment here with a bigger conclusion about the tests and the improvements I've done.

[nivida]

I've tested Web3 with the pendingTransactions subscription and calling of getTransaction.

const web3 = new Web3(new Web3.providers.WebsocketProvider('wss://mainnet.infura.io/ws/v3/APP_ID'));

web3.eth.subscribe('pendingTransactions').on('data', (hash) => {
  web3.eth.getTransaction(hash);
});

The changes in the referenced PR improved the memory consumption of Web3 a lot. The heap stack is still slightly increasing over time. I've compared the increase with the beta.55 architecture and could see a small improvement in comparison to the architecture of beta.37. The reason for the high memory usage of Web3 is because the Web3 object contains all the sub-modules and all of these are loaded into the memory. I couldn't find fixable closure related memory leaks or other leaks. The current module structure we have in Web3 is definitely a reason for the small increase we have but this can only be fixed in V2.0.

I've used the deoptigate tool to detect the potential de-optimizing code and couldn't find any bad code in the Web3.js library. I've used the chrome dev tools for measuring the memory usage and I was running the above example in parallel with different Web3 versions for detecting potential bad changes.

An important side note here is that optimized code is using more memory than non-optimized code. This behavior is because the saved machine code in the memory is bigger than the interpreter code.
source

Version 2.0 of Web3 will export the methods as single methods without modules which will definitely improve the "base" memory consumption of this library. It's also planned to remove the EventEmitter dependency and to just use Observables and Promises. For further details will I publish soon a Medium blog post with the next steps of the project.

[gnorbsl]

@nivida when can I expect a release for that? I have the feeling this is crashing our server if we dont restert the script regularily, since the script is using up all the memory at some point.

[snurfer0]

I'm still facing this issue in 2023, did you fix it?