CancelToken causes memory leak if re-used for multiple requests

[barryspearce]

Obtaining a cancel token (as per the Axios documentation on github) and re-using it across many requests results in a major memory leak as the payload for each request is not freed.

I am transferring large files - from 3 GB to 100GB using a chunked protocol and 20MB requests. The memory fills and remains until the cancel token reference is removed. Using the same code but not setting the cancel token in the config resolved the issue.

Creating a cancel token for each and every single request individually and removing the reference to the cancel token achieves the expected behaviour and frees the request payload memory.

Simple use of chrome dev tools and memory profile shows the cancel token as holding the memory, also reflect in process stats using top.

This is not using the global axios but an instance.

Previously reported in #1118 but it is unclear if this was properly resolved.

Axios 0.19.2
OS Linux
Browsers: Firefox 76.0, Opera 68.0.3618.63, Chromium 81.0.4044.129

[jasonsaayman]

Hi,

I think #1118 is possibly the wrong issue? Other than that I think we should have a look at this.

Thanks

[barryspearce]

You are right Jason - Sorry it was #1181

[gegana]

I recently encountered this issue as well on my React Native application. Our application downloads pages of data sequentially because it is an offline first app. Our findings indicate that as long as the cancel token source (even if you construct a new one each time) lives within a scope of an object or function that is long lived, the GC will have trouble releasing the memory handles from axios request & response.

With cancel token, notice the cliff towards the end, that is because the React component re-rendered.

Without cancel token, notice how GC is doing its job.

Axios 0.19.2
OS: iOS 13.6, Android 6 to 10
React Native: 0.61.5

(Please note that on React Native, the HTTP responses are stored as Blobs and cached on native side. In the version of React Native 0.61.5, the community fixed the issue where those Blobs in native land were not getting released even though the equivalent object in Javascript has been GCed. So you can see that fix doing its job well in the second screenshot but also explains the high memory consumption when its not collected because the amount of memory being cached is more than doubled for say a JSON response).

[yizhengfeng-jj]

Obtaining a cancel token (as per the Axios documentation on github) and re-using it across many requests results in a major memory leak as the payload for each request is not freed.

I am transferring large files - from 3 GB to 100GB using a chunked protocol and 20MB requests. The memory fills and remains until the cancel token reference is removed. Using the same code but not setting the cancel token in the config resolved the issue.

Creating a cancel token for each and every single request individually and removing the reference to the cancel token achieves the expected behaviour and frees the request payload memory.

Simple use of chrome dev tools and memory profile shows the cancel token as holding the memory, also reflect in process stats using top.

This is not using the global axios but an instance.

Previously reported in #1118 but it is unclear if this was properly resolved.

Axios 0.19.2
OS Linux
Browsers: Firefox 76.0, Opera 68.0.3618.63, Chromium 81.0.4044.129

Hello, according to your method seems to be useless, is my writing wrong, hope to get a reply, thank you very much

// request
config.cancelToken = new cancelToken(cancel => {
  source[request] = cancel;
})

// resonse
interceptors.response.use(response => {
 source[request] = null
})

[barryspearce]

?? I am not sure what you mean by "seems to be useless"?

If I understand your code you are cancelling the token immediately after a single request. However, for a file upload of 2000+ requests (I am talking multi-GB files here) I used the following algorithm.

Obtain cancel token
n * requests using the SAME cancel token
Clear the cancel token

This has the advantage of not having to allocate cancel tokens on every request. However, this causes a memory leak. Instead to avoid this every single request has to cancel the token and then allocate a new token.

No interceptors are involved.

[yizhengfeng-jj]

?? 我不确定“似乎没用”是什么意思？

如果我了解您的代码，您将在单次请求后立即取消令牌。但是，对于2000多个请求的文件上传（我在这里说的是多GB的文件），我使用了以下算法。

使用相同的取消令牌获取取消令牌n *请求
清除取消令牌

这具有不必在每个请求上分配取消令牌的优点。但是，这会导致内存泄漏。为了避免这种情况，每个请求都必须取消令牌，然后分配新令牌。

不涉及拦截器。

Thank you very much for your reply. I may not express it clearly. I saw you in your comments saying: Remove the cancellation token for every request and create a new cancellation token. Now the problem is, the pseudo code that I understand to remove the cancellation token for each request is as follows

source[request] = null; 

But after I set it this way, there are still memory leaks. You can write a pseudo code to explain how Help remove the cancellation token for every request and create a new cancellation token?

[r3c]

This issue also causes leak of the entire memory used for request & response when assigning a default CancelToken to config defaults, which makes this API quite error-prone:

instance.defaults.cancelToken = new axios.CancelToken((canceler) => { /* anything */ });

// Now every request sent through `instance` will leak memory because of the unique token (unless they override it)

It seems @DigitalBrainJS's #3305 nicely fixes the issue. Is there any chance to get it accepted & released? 🙏