memory leak in fetch()

[vampirefrog]

Version

v19.9.0

Platform

Linux www 5.10.0-26-amd64 #1 SMP Debian 5.10.197-1 (2023-09-29) x86_64 GNU/Linux

Subsystem

fetch/undici

What steps will reproduce the bug?

Simply run this code and watch the memory usage increase without decreasing. The code itself shouldn't hold references to anything.

(async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()

How often does it reproduce? Is there a required condition?

Every time

What is the expected behavior? Why is that the expected behavior?

Memory usage should stay at the same value after a fetch() call, because there are no references to it.

What do you see instead?

Memory usage increasing:

Fetched 19583 bytes, RSS is 80588800
Fetched 19592 bytes, RSS is 81358848
Fetched 19550 bytes, RSS is 82419712
Fetched 19579 bytes, RSS is 82632704
Fetched 19597 bytes, RSS is 83132416
Fetched 19563 bytes, RSS is 83390464
Fetched 19581 bytes, RSS is 83636224
Fetched 19523 bytes, RSS is 83705856
Fetched 19556 bytes, RSS is 83988480
Fetched 19582 bytes, RSS is 83988480
Fetched 19543 bytes, RSS is 83988480
Fetched 19625 bytes, RSS is 84475904
Fetched 19533 bytes, RSS is 84606976
Fetched 19595 bytes, RSS is 84836352
Fetched 20015 bytes, RSS is 84836352
Fetched 19552 bytes, RSS is 85291008
Fetched 19572 bytes, RSS is 85516288
Fetched 19588 bytes, RSS is 85770240
Fetched 19646 bytes, RSS is 85770240
Fetched 19585 bytes, RSS is 85245952

Additional information

Discussion started in #51429 which seems to be similar or a duplicate of #46435.

[marco-ippolito]

Maybe transfer the issue in undici?
cc @nodejs/undici

[metcoder95]

Think so? cc @KhafraDev

[mcollina]

Does that reproduce on undici main?

[metcoder95]

Yeah, I can confirm that

[vampirefrog]

You want me to re-file it in the other repo?

[vampirefrog]

After a bit more inspection it seems that the memory does indeed go down after a few seconds, but it never goes down to what it was before the fetches. In other words, if you run the same one-liner above multiple times, and wait a number of seconds in between runs.

Welcome to Node.js v19.9.0.
Type ".help" for more information.
> (async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 23,
  [Symbol(trigger_async_id_symbol)]: 5
}
> Fetched 19580 bytes, RSS is 76619776
Fetched 19553 bytes, RSS is 76894208
Fetched 19560 bytes, RSS is 79294464
Fetched 19636 bytes, RSS is 80613376
Fetched 19536 bytes, RSS is 81383424
Fetched 19609 bytes, RSS is 79568896
Fetched 19584 bytes, RSS is 80084992
Fetched 19568 bytes, RSS is 80330752
Fetched 19619 bytes, RSS is 80560128
Fetched 19636 bytes, RSS is 80642048
Fetched 19512 bytes, RSS is 80887808
Fetched 19527 bytes, RSS is 81682432
Fetched 19561 bytes, RSS is 82202624
Fetched 19596 bytes, RSS is 82739200
Fetched 19644 bytes, RSS is 83001344
Fetched 19570 bytes, RSS is 83259392
Fetched 19566 bytes, RSS is 83505152
Fetched 19662 bytes, RSS is 83505152
Fetched 19633 bytes, RSS is 83505152
Fetched 19633 bytes, RSS is 83763200
> 
> 
> (async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 17396,
  [Symbol(trigger_async_id_symbol)]: 5
}
> Fetched 19577 bytes, RSS is 84676608
Fetched 19656 bytes, RSS is 82186240
Fetched 19602 bytes, RSS is 81530880
Fetched 19554 bytes, RSS is 82026496
Fetched 19576 bytes, RSS is 82247680
Fetched 19579 bytes, RSS is 82370560
Fetched 19581 bytes, RSS is 83111936
Fetched 19605 bytes, RSS is 83316736
Fetched 19595 bytes, RSS is 84127744
Fetched 19640 bytes, RSS is 84664320
Fetched 19627 bytes, RSS is 84922368
Fetched 19626 bytes, RSS is 85999616
Fetched 19654 bytes, RSS is 86257664
Fetched 19650 bytes, RSS is 86798336
Fetched 19615 bytes, RSS is 87068672
Fetched 19614 bytes, RSS is 87068672
Fetched 19629 bytes, RSS is 87588864
Fetched 19629 bytes, RSS is 87855104
Fetched 19575 bytes, RSS is 87855104
Fetched 19600 bytes, RSS is 88121344

> 
> 
> (async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 35336,
  [Symbol(trigger_async_id_symbol)]: 5
}
> Fetched 19656 bytes, RSS is 86163456
Fetched 19623 bytes, RSS is 85905408
Fetched 19628 bytes, RSS is 86347776
Fetched 19563 bytes, RSS is 85876736
Fetched 19549 bytes, RSS is 86192128
Fetched 19624 bytes, RSS is 86622208
Fetched 19589 bytes, RSS is 86851584
Fetched 19626 bytes, RSS is 86806528
Fetched 19645 bytes, RSS is 87011328
Fetched 19646 bytes, RSS is 87822336
Fetched 19648 bytes, RSS is 88363008
Fetched 19685 bytes, RSS is 88625152
Fetched 19690 bytes, RSS is 88891392
Fetched 19623 bytes, RSS is 89862144
Fetched 19569 bytes, RSS is 90390528
Fetched 19598 bytes, RSS is 90660864
Fetched 19561 bytes, RSS is 90660864
Fetched 19588 bytes, RSS is 90660864
Fetched 19615 bytes, RSS is 90927104
Fetched 19622 bytes, RSS is 90927104

> 
> (async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 51869,
  [Symbol(trigger_async_id_symbol)]: 5
}
> Fetched 19541 bytes, RSS is 87912448
Fetched 19512 bytes, RSS is 88801280
Fetched 19601 bytes, RSS is 88166400
Fetched 19615 bytes, RSS is 88395776
Fetched 19574 bytes, RSS is 88735744
Fetched 19538 bytes, RSS is 88408064
Fetched 19653 bytes, RSS is 88580096
Fetched 19658 bytes, RSS is 88465408
Fetched 19613 bytes, RSS is 89546752
Fetched 19652 bytes, RSS is 89817088
Fetched 19585 bytes, RSS is 90079232
Fetched 19592 bytes, RSS is 90619904
Fetched 19619 bytes, RSS is 90886144
Fetched 19651 bytes, RSS is 91422720
Fetched 19581 bytes, RSS is 91422720
Fetched 19622 bytes, RSS is 91680768
Fetched 19643 bytes, RSS is 91680768
Fetched 19542 bytes, RSS is 91934720
Fetched 19632 bytes, RSS is 91934720
Fetched 19606 bytes, RSS is 91934720

> 
> (async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 71255,
  [Symbol(trigger_async_id_symbol)]: 5
}
> Fetched 19618 bytes, RSS is 92164096
Fetched 19501 bytes, RSS is 89681920
Fetched 19651 bytes, RSS is 89407488
Fetched 19626 bytes, RSS is 89501696
Fetched 19578 bytes, RSS is 89391104
Fetched 19612 bytes, RSS is 89333760
Fetched 19630 bytes, RSS is 89812992
Fetched 19565 bytes, RSS is 90304512
Fetched 19591 bytes, RSS is 91115520
Fetched 19606 bytes, RSS is 91652096
Fetched 19658 bytes, RSS is 92463104
Fetched 19597 bytes, RSS is 92729344
Fetched 19613 bytes, RSS is 93540352
Fetched 19561 bytes, RSS is 93810688
Fetched 19607 bytes, RSS is 93810688
Fetched 19651 bytes, RSS is 94076928
Fetched 19613 bytes, RSS is 94076928
Fetched 19595 bytes, RSS is 94076928
Fetched 19588 bytes, RSS is 94605312
Fetched 19596 bytes, RSS is 94605312

But it seems to be fixed in v21.5.0, or at least greatly reduced:

Welcome to Node.js v21.5.0.
Type ".help" for more information.
> (async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 25,
  [Symbol(trigger_async_id_symbol)]: 6
}
> Fetched 19612 bytes, RSS is 71208960
Fetched 19654 bytes, RSS is 65150976
Fetched 19618 bytes, RSS is 65957888
Fetched 19546 bytes, RSS is 66838528
Fetched 19637 bytes, RSS is 67485696
Fetched 19602 bytes, RSS is 67751936
Fetched 19594 bytes, RSS is 68022272
Fetched 19622 bytes, RSS is 68022272
Fetched 19567 bytes, RSS is 68833280
Fetched 19637 bytes, RSS is 69103616
Fetched 19555 bytes, RSS is 70553600
Fetched 19601 bytes, RSS is 71364608
Fetched 19502 bytes, RSS is 71364608
Fetched 19590 bytes, RSS is 72445952
Fetched 19665 bytes, RSS is 72986624
Fetched 19603 bytes, RSS is 73256960
Fetched 19566 bytes, RSS is 73256960
> Fetched 19588 bytes, RSS is 73527296
> Fetched 19545 bytes, RSS is 73527296
> Fetched 19615 bytes, RSS is 73797632
> 
> 
> (async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 19311,
  [Symbol(trigger_async_id_symbol)]: 6
}
> Fetched 19575 bytes, RSS is 74665984
Fetched 19622 bytes, RSS is 74665984
Fetched 19636 bytes, RSS is 74665984
Fetched 19631 bytes, RSS is 68681728
Fetched 19620 bytes, RSS is 69758976
Fetched 19591 bytes, RSS is 68784128
Fetched 19601 bytes, RSS is 68145152
Fetched 19561 bytes, RSS is 68100096
Fetched 19588 bytes, RSS is 68911104
Fetched 19557 bytes, RSS is 68116480
Fetched 19589 bytes, RSS is 68231168
Fetched 19611 bytes, RSS is 68907008
Fetched 19557 bytes, RSS is 69500928
Fetched 19576 bytes, RSS is 70852608
Fetched 19594 bytes, RSS is 71122944
Fetched 19583 bytes, RSS is 71663616
Fetched 19610 bytes, RSS is 71663616
Fetched 19573 bytes, RSS is 71933952
Fetched 19576 bytes, RSS is 71933952
Fetched 19554 bytes, RSS is 71933952

> (async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 37785,
  [Symbol(trigger_async_id_symbol)]: 6
}
> Fetched 19633 bytes, RSS is 69656576
Fetched 19575 bytes, RSS is 69689344
Fetched 19584 bytes, RSS is 68931584
Fetched 19621 bytes, RSS is 69705728
Fetched 19609 bytes, RSS is 66514944
Fetched 19609 bytes, RSS is 66547712
Fetched 19619 bytes, RSS is 65097728
Fetched 19628 bytes, RSS is 65908736
Fetched 19600 bytes, RSS is 66633728
Fetched 19504 bytes, RSS is 67174400
Fetched 19563 bytes, RSS is 67715072
Fetched 19657 bytes, RSS is 67715072
Fetched 19618 bytes, RSS is 68526080
Fetched 19628 bytes, RSS is 68796416
Fetched 19633 bytes, RSS is 69607424
Fetched 19617 bytes, RSS is 69607424
Fetched 19630 bytes, RSS is 69877760
Fetched 19581 bytes, RSS is 70418432
Fetched 19550 bytes, RSS is 71491584
Fetched 19555 bytes, RSS is 71491584

> (async () => { for (let i = 0; i < 20; i++) { await fetch("https://google.com").then(r => r.text()).then(r => console.log('Fetched', r.length, 'bytes, RSS is', process.memoryUsage().rss)); } })()
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 56514,
  [Symbol(trigger_async_id_symbol)]: 6
}
> Fetched 19623 bytes, RSS is 71761920
Fetched 19616 bytes, RSS is 68161536
Fetched 19648 bytes, RSS is 69513216
Fetched 19615 bytes, RSS is 69017600
Fetched 19572 bytes, RSS is 68681728
Fetched 19587 bytes, RSS is 68587520
Fetched 19558 bytes, RSS is 68726784
Fetched 19601 bytes, RSS is 69120000
Fetched 19617 bytes, RSS is 69156864
Fetched 19652 bytes, RSS is 69103616
Fetched 19596 bytes, RSS is 69169152
Fetched 19621 bytes, RSS is 70250496
Fetched 19635 bytes, RSS is 71061504
Fetched 19666 bytes, RSS is 71061504
Fetched 19677 bytes, RSS is 71331840
Fetched 19529 bytes, RSS is 72413184
Fetched 19575 bytes, RSS is 72953856
Fetched 19646 bytes, RSS is 73764864
Fetched 19605 bytes, RSS is 73764864
Fetched 19612 bytes, RSS is 73764864

[marco-ippolito]

(from undici main)
At some point memory gets garbage collected:

const { request } = require('..');
(async () => {
    console.log(process.memoryUsage().rss);
    for (let i = 0; i < 50; i++) {
        const { body } = await request('http://example.org');
        await body.text();
        console.log(process.memoryUsage().rss);
    }
    setTimeout(() => {
        console.log(process.memoryUsage().rss);
    }, 3000);   
})();

76120064
76120064
76120064
76120064
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76922880
76939264
76955648
76955648
76955648
73940992

[mcollina]

So this seems already fixed in main (it has the latest undici landed).
This will need backports to v18 and v20.

[metcoder95]

Hmm, I might be reproduce wrongly the. Sorry for the wrong callout