memory leak

[trim21]

Type: Bug Report

• Try to follow the update procedure described in the README and try again before opening this issue.

Your Environment

• Version used: 7aec1e2
• Environment name and version:
  • Node.js version node --version: 18.16.0
  • npm version npm --version
  • Web browser name and version
• Operating system and version: windows/linux
• Torrent client and version: rtorrent

Summary

memory leak in event source handler

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce

request dis-connect "/activity-stream" many time (about 1000)
you will find memory leaked, and web UI stop working

import httpx
import sseclient
import tqdm


def stream():
    with httpx.stream("GET", "http://127.0.0.1:3000/api/activity-stream") as s:
        yield from s.iter_bytes()


def main():
    events = {
        "TORRENT_LIST_DIFF_CHANGE",
        "CLIENT_CONNECTIVITY_STATUS_CHANGE",
        "DISK_USAGE_CHANGE",
        "TAXONOMY_FULL_UPDATE",
        "TAXONOMY_DIFF_CHANGE",
        "TRANSFER_HISTORY_FULL_UPDATE",
        "TRANSFER_SUMMARY_FULL_UPDATE",
        "NOTIFICATION_COUNT_CHANGE",
        "TORRENT_LIST_FULL_UPDATE",
    }
    client = sseclient.SSEClient(stream())
    for _event in client.events():
        events.discard(_event.event)
        if events:
            return


if __name__ == "__main__":
    with tqdm.tqdm() as bar:
        while True:
            main()
            bar.update()

Context

[trim21]

very likely this is caused by rtorrent request queue. there maybe a bug. I replace it with p-queue, and everythink looks fine.

[filippo-orru]

I'm on master and have memory leak problems. When running the flood development server, it quickly starts consuming upwards of 16 GB of memory. After some time, it either crashes or my Mac prompts me to close other applications, as I'm out of memory.
This is the graph after closing flood (node):

[trim21]

I'm on master and have memory leak problems. When running the flood development server, it quickly starts consuming upwards of 16 GB of memory. After some time, it either crashes or my Mac prompts me to close other applications, as I'm out of memory. This is the graph after closing flood (node):

what bt client are you using?

[filippo-orru]

@trim21 I'm running rtorrent using the following command:

docker run -it --rm -v ./volumes/config:/config -v ./volumes/data:/Users/ffactory/data --user 1000:1001 -p 0.0.0.0:6881:5000 -e HOME=/config jesec/rtorrent

edit:
Node v20.0.0
npm 9.8.1

[trim21]

@trim21 I'm running rtorrent using the following command:

docker run -it --rm -v ./volumes/config:/config -v ./volumes/data:/Users/ffactory/data --user 1000:1001 -p 0.0.0.0:6881:5000 -e HOME=/config jesec/rtorrent

edit: Node v20.0.0 npm 9.8.1

looks like there are more than one memory leak bug...

[trim21]

it quickly starts consuming upwards of 16 GB of memory

how many torrents do you have in rtorrent?

and "it quickly starts consuming upwards of 16 GB of memory", how quickly?

[filippo-orru]

@trim21 the number of threads keeps rising (currently 2.9k)

Within 5 seconds of starting the server, it's already at 4 GB, after total 30 seconds it's at 9 GB and the number of threads approaches 1k.

After killing both npm run start:development:server and npm run start:development:client it takes more than 20 seconds for the node process to fully close. Used memory and number of threads quickly decline during that time.

Edit: I should note, I'm using a MacBook Pro with Apple silicon. On my VPS (Ubuntu 20, x86) I'm using flood with zero problems.

[trim21]

@trim21 the number of threads keeps rising (currently 2.9k)

Within 5 seconds of starting the server, it's already at 4 GB, after total 30 seconds it's at 9 GB and the number of threads approaches 1k.

After killing both npm run start:development:server and npm run start:development:client it takes more than 20 seconds for the node process to fully close. Used memory and number of threads quickly decline during that time.

Edit: I should note, I'm using a MacBook Pro with Apple silicon. On my VPS (Ubuntu 20, x86) I'm using flood with zero problems.

I don't have Apple silicon, I can't help with this...

[filippo-orru]

@trim21
I don't have Apple silicon, I can't help with this...

I'm happy to help investigate this if you have any clues. 🙂

[trim21]

@trim21 the number of threads keeps rising (currently 2.9k)

Within 5 seconds of starting the server, it's already at 4 GB, after total 30 seconds it's at 9 GB and the number of threads approaches 1k.

After killing both npm run start:development:server and npm run start:development:client it takes more than 20 seconds for the node process to fully close. Used memory and number of threads quickly decline during that time.

Edit: I should note, I'm using a MacBook Pro with Apple silicon. On my VPS (Ubuntu 20, x86) I'm using flood with zero problems.

does this happened if you are not using developing startup?

for example: npm run build and node dist/index.js.

There maybe some incompatible between apple silicon and tsx(esbuild)

[filippo-orru]

does this happened if you are not using developing startup?

for example: npm run build and node dist/index.js.

There maybe some incompatible between apple silicon and tsx(esbuild)

Running npm run build and node dist/index.js works without any problems.

[trim21]

does this happened if you are not using developing startup?
for example: npm run build and node dist/index.js.
There maybe some incompatible between apple silicon and tsx(esbuild)

Running npm run build and node dist/index.js works without any problems.

Then I think this is caused by tsx/esbuild... and I have no clue about how to fix it 😥

please just use production build.

[filippo-orru]

@trim21 okay, I'll do that. Thanks for your efforts.