v4.4.5 x64 - Insane memory usage (Windows)

[FiTADiNE]

qBittorrent & operating system versions

qBittorrent: v4.4.5 x64
OS: Windows 10 x64 21H2 10.0.19044
Qt: 6.3.0
Libtorrent: 1.2.17.0
32 GB RAM

What is the problem?

Insane memory leak. 5k torrents. In previous versions there was excessive consumption and chrome/telegram/media(vids, streams) crashes, even with a memory usage limit.

Steps to reproduce

1. Start qB
2. Wait
3. ???
4. Profit.

Additional context

No memory leak with all seeding torrents turned off.

Log(s) & preferences file(s)

No response

[glassez]

In previous versions there was excessive consumption and chrome/telegram/media(vids, streams) crashes, even with a memory usage limit.

Why do you report about previous versions?

[FiTADiNE]

Why do you report about previous versions?

Just for additional information. Main report about last version.

[adem4ik]

Seems like a duplicate of many other issues reported this year. It is connected with libtorrent v2 - arvidn/libtorrent#6667

Speaking of the current topic, it is better to switch to the client version with libtorrent v1. And since 4.4.5 it is default build. You can get under qBittorrent Windows x64 link from https://www.fosshub.com/qBittorrent.html

After installing it you'll see something like this in About dialog:

[FiTADiNE]

@adem4ik, sorry, there was a mistake in libtorrent ver. My current version is 1.2.17.0 already.

[adem4ik]

@FiTADiNE in this case you can limit RAM usage by setting up Tools>Options>Advanced>Disk cache (Кэш диска). -1 means unlimited/auto. Only build with libtorrent v1 has such option.

[FiTADiNE]

@adem4ik, probably, in prev releases there was RAM limit, but now it's disk cache limit. Omg... Thanks, i'll try it.

[FiTADiNE]

@adem4ik, still the same.
@ElectricityMachine, just windows dark theme.

[pulbitz]

4.4.5(libtorrent1+qt5) x64 uses 300-400 MB more memory than the disk cache size. 4.3.9 did not.

[FiTADiNE]

@pulbitz, best way is downgrading to 4.3.9?

[pulbitz]

@pulbitz, best way is downgrading to 4.3.9?

I think it's a bit odd to use around 300-400 MB more memory than the disk cache size I specified. That's all. I'm still using 4.4.5 (libtorrent1+qt5) as there are no other inconveniences.

[glassez]

4.4.5(libtorrent1+qt5) x64 uses 300-400 MB more memory than the disk cache size.

Do you think qBittorrent uses memory only for the disk cache?

[pulbitz]

I usually use the disk cache size set to 4096 MB.
Below is the result of measuring memory usage in Process Explorer.
And I tested it by setting it to 1024 MB.

[FiTADiNE]

Working Set limit 1024MB.

25% of 32GB RAM used after 15 minutes. (Work around application stuttering on Windows not fixed [fixed in 4.4.4]).

Disk Cache limit 1024MB.

68% of 32GB RAM used after 15 minutes and growing. (Peak Working set cuz i've tried 2048MB Disk Cache).

[stalkerok]

FiTADiNE, decide already which version of qbt you are using and on which libtorrent.

[FiTADiNE]

@stalkerok, already decided in 1st post.

[glassez]

Working Set limit 1024MB.

25% of 32GB RAM used after 15 minutes.

There are less than 1GB of RAM used on this screenshot.
The similar contradiction is in your second screenshot and related comment.

[FiTADiNE]

@glassez, yes, cuz it's older version on libtorrent 2 with Working set limit parameter and w/o memory leak. Here you can see 2 screenshots: 1066MB qB usage with total 88% of RAM (28.16GB) and 21% total of RAM (6.72GB) w/o qB.

Today i've tested older version, and last on Qt5. 4.4.5 has memory leak with libtorrent 1.2.17.0 on any Qt. 4.4.3.1 on libtorrent 2.0.6.0 doesn't have memory leak, but has problem with "work around application stuttering on Windows" which was fixed in 4.4.4.

Now i'm testing 4.4.5 RC_2 with libtorrent 2.0.7.0

[stalkerok]

@FiTADiNE, have you tried lowering the memory priority?

[glassez]

1066MB qB usage with total 88% of RAM (28.16GB)

Unfortunately, your screenshot does not provide any information about what actually uses all this memory.

4.4.5 has memory leak with libtorrent 1.2.17.0 on any Qt

There can be no memory usage that is not taken into account by the operating system (unless of course there is a memory leak in the operating system itself). Even if there is a memory leak in some application program, the operating system knows that this memory is occupied by this program.

Of course, this memory can be used by the operating system itself for something related to the operation of qBittorrent (for example, for I/O caching). Therefore, it would be much more useful if you presented a more accurate source of memory consumption.
Besides, it would be better to see reports in a more structured form (since you decided to compare data on the work of several qBittorrent versions).

[FiTADiNE]

@stalkerok, no, should i? And why?

@glassez, lolwhat? 1st screen: qB works, 88% of 32GB RAM. Second: qB closed, 21% of 32GB RAM. What kind of screenshot do you need? Maybe i should record video?
UPD: i'll be glad if you tell me, how can i present more accurate source of memory consumption and structured form of reports.

[stalkerok]

@FiTADiNE, disable the OS cache and set the memory usage to the lowest level, this will reduce the memory consumption.

[glassez]

1st screen: qB works, 88% of 32GB RAM. Second: qB closed, 21% of 32GB RAM. What kind of screenshot do you need?

Where it is seen what exactly uses all these memory.

[glassez]

structured form of reports

Since you provide reports about different qBittorrent version don't forget to attach info about used qBittorrent, libtorrent, OS, etc.

[FiTADiNE]

Where it is seen what exactly uses all these memory.

Okay, where could i see it?

Since you provide reports about different qBittorrent version don't forget to attach info about used qBittorrent, libtorrent, OS, etc.

Already done (with attached screenshots and text information in posts).

v4.4.5 RC_2, Qt 6.3.0, libtorrent 2.0.7.0. 30% of RAM used. Works fine probably, but Private bytes...?

[ghost]

I do not understand what people expect the RAM usage to be with 5000 torrents.
If qBt has to constantly access and read files from disk, the OS is definitely caching some of them and even with a working set limit you may not have control over how the OS behaves. If there is a memory leak, you should run a profiler and collect data for the developers to look into it and see if there's indeed a leak or it's just the OS.

[Pentaphon]

I do not understand what people expect the RAM usage to be with 5000 torrents.

What do you expect it to be?

[FiTADiNE]

@summerqb, well, now i'm testing v4.4.5 Qt6 libtorrent 2.0.7.0 and i don't have any problems with RAM. 350MB usage with 1024MB Working set limit, and 21% usage of total RAM with no "Work around application stuttering on Windows".

[LarLei]

I run 2.6k torrents.

Just curious, does it mean 2.6k active (actually downloading and/or seeding) torrents?

I'd assume there is a difference in system resources (i.e. CPU, RAM) between active vs stalled torrents.

[sserdda-liamE]

I do not understand what people expect the RAM usage to be with 5000 torrents.

What do you expect it to be?

Anywhere between 16 to 32 GB. Also someone should not be seeding 5k torrents on a regular PC and expect the torrent client to not bog down the system. It should be done on a dedicated system with adequate CPU, RAM, Network capacity and proper storage medium to be able to handle the I/O demand.

Uh, this is µTorrent 2.2.1 with 2500 and ~20TB seeding at 20mbps. Takes 14 seconds to load. Has never bogged down anything:

I've always found it odd that so many people seem to think that torrenting is some high resource activity that requires a dedicated modern server, when µTorrent is basically invisible on a Pentium 3.

[Pentaphon]

I've always found it odd that so many people seem to think that torrenting is some high resource activity that requires a dedicated modern server, when µTorrent is basically invisible on a Pentium 3.

I don't get it either. They are the same people who recommend a Raspi for torrenting thousands of files which is already done just fine with other clients. The issue is that libtorrent still hasn't become as stable as uTorrent. libtorrent has so many memory leaks that people just assume torrenting even just hundreds of torrents takes several GBs of memory.

[glassez]

libtorrent has so many memory leaks that people just assume torrenting even just hundreds of torrents takes several GBs of memory.

A lot of people don't understand the meaning of the term memory leak at all, so they call it any large memory consumption.

[ghost]

Uh, this is µTorrent 2.2.1 with 2500 and ~20TB seeding at 20mbps. Takes 14 seconds to load. Has never bogged down anything

I wonder why so many people are switching from such wonderful piece of software to an inferior client like qBt. I also wonder why they even bother to comment here 😁

[sserdda-liamE]

Uh, this is µTorrent 2.2.1 with 2500 and ~20TB seeding at 20mbps. Takes 14 seconds to load. Has never bogged down anything

I wonder why so many people are switching from such wonderful piece of software to an inferior client like qBt.

Freetardism is a weird religion. It motivates those caught in its warped ideology to proselytize 24/7 for software based not on any analysis of quality, but merely on the legal licensing scheme it is released under.
As normal people do not proselytize software, this causes a minority of fanatics to dominate the volume of traffic. And your average person is biased to believe a suggestion he hears many times on the assumption that it would not be made if it didn't have any backing, as leading him astray would be antisocial behavior that your average person would avoid.
Freetardism not only has no internal prohibition against such antisocial behavior, it demands it in the name of the spread of Freetardism. Thus general population is thus convinced to move based on nothing but a false assumption that the behavior they witnessed was sociable.

I'm not sure how people manage to get mentally trapped in such a silly bubble as Freetardism. It seems to me that just an ounce of pragmatism is all that is required to pierce the surface film and see the bubble for what it is. But it seems like the common person has an inordinate desire to belong to a group, causing all lines of intellectual inquiry to be corrupted and coopted into curved lines in the name of fulfilling said desire.

I prefer straight lines. They align with my autistic desire to be objectively correct. They often lie in a messy heap, but there's an underlying, indescribable beauty to it.

https://www.youtube.com/watch?v=2sPDPr3S7Zg

[Pentaphon]

I wonder why so many people are switching from such wonderful piece of software to an inferior client like qBt. I also wonder why they even bother to comment here

Maybe they want to simply point out that qBit is actually far from stable and that uTorrent, despite its bad reputation, remains a stable, low-resource torrent client that is still a benchmark that qBittorrent (more accurately libtorrent) still needs much work to meet or exceed.

[sledgehammer999]

Uh, this is µTorrent 2.2.1 with 2500 and ~20TB seeding at 20mbps. Takes 14 seconds to load

And your average person is biased to believe a suggestion he hears

Your use case certainly isn't that of an average person's use case.
You're a power user. Power users should test and see what software best fits them. The average person can make do with almost any torrent client software.

That said, you're more than welcome to report bugs on qbt, help test and possibly provide fixes.
Ranting about it gets you nowhere.

[Pentaphon]

Ranting about it gets you nowhere.

To be fair, ranting about the other issues that libtorrent 2.x had finally made you guys decide to (rightfully) go back to libtorrent 1.2.x by default so user rants do actually sometimes get them somewhere that is beneficial to the userbase.

Ignoring what users say is what actually gets a project nowhere, since this project is for end users including yourself, after all.

[Seeker2]

The problem pointed out by FiTADiNE is more of a virtual memory leak than a physical memory leak.
I've pointed out this before:
qBitTorrent Virtual Memory Leak #7531

And I've also noted that physical memory usage by qBT's cache was ~30% higher than the cache size was set to, suggesting very poor memory use in indexing the cache and/or physical memory allocations for the cache larger than the cache's actual usage.
That among other reasons was why libtorrent v2.x was dropping its own internal cache.

[glassez]

The problem pointed out by FiTADiNE is more of a virtual memory leak than a physical memory leak.

#17680 (comment)

[Seeker2]

Is your contention that this is not a virtual memory leak or that my terminology is incorrect?

...Because it helps to clearly define the problem, if there is one, before continuing!

[glassez]

Is your contention that this is not a virtual memory leak or that my terminology is incorrect?

My contention that "A lot of people don't understand the meaning of the term memory leak at all". For them "high memory consumption" is equal to "memory leak" which is incorrect. "Memory leak" is a strictly defined problem. And it cannot be identified based only on symptoms in the form of high memory consumption (it may not cause such symptoms at all, while the program's error-free operation may). By using this term, you seem to be claiming that the cause has already been identified and it lies precisely in a "memory leak", and not in some other problem.

[Seeker2]

I claimed the problem the initial poster is reporting is a virtual memory leak NOT a standard physical memory leak or high physical memory usage.
Now "working as designed" can be the cause but what is being reported/claimed here are the symptoms.

I mentioned my earlier report of somewhat-higher-than-expected physical memory usage caused by qBT's file cache using 30% more physical memory than the max size of the cache...which is a side issue but perhaps related, since such things do increase virtual memory usage as well.

[glassez]

the problem the initial poster is reporting is a virtual memory leak NOT a standard physical memory leak or high physical memory usage.

As for the "virtual" memory leak, it is generally incorrect to use "virtual" in this context (when we are talking about an application program, considering it from the inside). Virtual memory is exactly what is available to the application program, it does not know anything about physical memory at all (in the sense that it does not use physical memory directly). It is the OS that decides which pages of the virtual memory of a process reside in physical memory at a given time.
In any case, you should not claim that this is a memory leak unless you have actually inspected the code (this is practically the only way to detect a memory leak).

[Seeker2]

Would triggering Windows bluescreens suffice?

[Pentaphon]

My contention that "A lot of people don't understand the meaning of the term memory leak at all". For them "high memory consumption" is equal to "memory leak" which is incorrect.

I have never seen people think of those as the same thing. When people report this issue, they know it is a specific problem. They do not think that qBittorrent uses too much memory by its nature and that memory consumption should be lowered. They are saying that qBittorrent suffers from memory leaks and they should be investigated and fixed.

[glassez]

They do not think that qBittorrent uses too much memory by its nature and that memory consumption should be lowered. They are saying that qBittorrent suffers from memory leaks and they should be investigated and fixed.

That's exactly what I'm talking about. You are confusing "the cause of the disease with its symptoms".
As I said above, you cannot know that this is a memory leak unless you have actually inspected the code since inspecting the code is practically the only way to detect a memory leak (static analysis can help with some of them).

[Pentaphon]

As I said above, you cannot know that this is a memory leak unless you have actually inspected the code since inspecting the code is practically the only way to detect a memory leak

Inspecting the code is the only way to find the exact cause of a memory leak, but the symptoms of memory leaks get spotted all the time by people who report most or even all their RAM being used for no reason by an exe. It is strange that people seem to think that it is normal for a torrent client to require up to 32GB of RAM for a few thousand torrents when other clients seem to do similar loads with much less than 32GB of RAM.

[glassez]

It is strange that people seem to think that it is normal for a torrent client to require up to 32GB of RAM for a few thousand torrents when other clients seem to do similar loads with much less than 32GB of RAM.

There may be some kind of problem or even several problems behind this (of course, there may be a memory leak among them, but you will not determine this from the outside).

Inspecting the code is the only way to find the exact cause of a memory leak, but the symptoms of memory leaks get spotted all the time by people who report most or even all their RAM being used for no reason by an exe.

https://en.m.wikipedia.org/wiki/Memory_leak

In computer science, a memory leak is a type of resource leak that occurs when a computer program incorrectly manages memory allocations[1] in a way that memory which is no longer needed is not released. A memory leak may also happen when an object is stored in memory but cannot be accessed by the running code (i.e. unreachable memory).[2] A memory leak has symptoms similar to a number of other problems and generally can only be diagnosed by a programmer with access to the program's source code.

Just for fun, I could add a couple of real memory leaks to the qBittorrent code, and you wouldn't even notice it.

As for the problem of high memory consumption, it is important first of all to define the boundaries of this very problem. And it's damn hard to do in an Issue like this, where a lot of contradictory information is mixed up, several versions of qBittorrent are mentioned, and most importantly different branches of libtorrent, which have fundamental differences in the disk I/O subsystem, which has a direct impact on memory consumption.

[Seeker2]

And your code could be "bulletproof" levels of correct and the compiler can still make a mess of it too, with runtime optimizations.
So good luck as a programmer finding problems that happen with qBitTorrent but don't happen with other BT clients.

Incidentally, the 2 GB virtual memory crash with qBT 32bit Windows versions I found years ago...supposedly didn't happen when run on a VM. ...didn't make that problem any less real or not caused by qBitTorrent.

[glassez]

And your code could be "bulletproof" levels of correct and the compiler can still make a mess of it too, with runtime optimizations.
So good luck as a programmer finding problems that happen with qBitTorrent but don't happen with other BT clients.

You can do it yourself if you want. Your prejudice that it is a "memory leak" confuses everyone, distracting from the search for more likely causes.
Unfortunately, this Issue is not suitable for continuing constructive work, as it has turned into a dump of inconsistent and contradictory reports, often not related to any one specific problem.

[Seeker2]

I called it a virtual memory leak.

[Cosmitz]

Can confirm, this is a relatively recent issue. I updated 4.5.0 after delaying a few versions (prolly 4.3.6 was what i was on), and while the exe itself doesn't show a lot of usage, i can come back to something like 15-25gb of memory used, more importantly DEFINITELY using windows swap file by that point, making the entire system super stuttery and sluggish. All of this goes back to normal, dropping to regular ram usage, after closing down qbittorent.

Had set disk cache to auto -1 with DiskOS use on, set it to 4096 right now, will report in if that changes anything. All i'm seeing is that qbittorent now shows the increased ram usage in the exe to what i set as Disk Cache, instead of the 1gb or something it showed before.

[ghost]

@sserdda-liamE couldn't be more correct. I genuinely cannot comprehend how this excuse of a program is still around. it has the same painfully annoying issues since ages ago that keep being ignored by everyone involved in qbit. and then you have insufferable people like glassez that instead of helping will close issues and tell you to shut up. what's wrong with these guys? it can only be understood as pure insanity at this point. but that's not new. people keep reinventing the wheel in here, but instead of actually making a better wheel, they'll create a square, sandpaper-covered piece of solid tungsten that is outperformed by 10 years old software and pretend it is all right with it.

[Pentaphon]

We won't be able to pinpoint the cause of this until #8664 is implemented. We need to make this issue a priority for Windows users.

We need to be able to install qBittorrent-nox as a headless, barebones service on Windows so we can see if the headless version of qBittorrent is responsible or the full desktop version of qBittorrent is responsible for the excessive RAM usage a lot of people see. I very much suspect that qBittorrent-nox running as a service by itself won't have the same excessive RAM usage issue on Windows machines.

Transmission does this on Windows by installing a headless version of Transmission called "transmission-daemon" as a background service on Windows and qBittorrent should do the same with -nox.

[DNS]

@Pentaphon Excessive RAM usage is caused by disk cache (which is actually a good thing).
You want to use most of the RAM available, otherwise why buy 32GB RAM in the first place.

[Pentaphon]

Excessive RAM usage is caused by disk cache (which is actually a good thing).

If that were true, why doesn't every torrent client or program use more than 1 GB of your available ram?

[DNS]

@Pentaphon rtorrent (without user pre-defined settings) can use more than 1.4 GB of RAM on Linux.

The rtorrent developers are no fool. The are smart, that's why they do it like that.

Without disk cache, your disk will be dead earlier than you expected.