Pre-allocate disk space for all files doesn't work

[x8009]

Please provide the following information

libtorrent version (or branch):
1.0.11.0
platform/architecture:
Windows 10 x64 1703
compiler and compiler version:
Qbittorent 3.3.13 QT: 5.8.0 Boost : 1.63.0
please describe what symptom you see, what you would expect to see instead and
how to reproduce it.
you can read all about it in here
https://qbforums.shiki.hu/index.php/topic,5063.0.html

[ssiloti]

Pre-allocation works for me with RC_1_1. Are you sure that libtorrent version is correct? I thought that qBittorrent was shipping with libtorrent 1.1 now.

[x8009]

that's the version it says on the newest qbittorrent 3.3.13. how can we update it with the newest libtorrent? what can we do right now?

[Seeker2]

Disk space is only pre-allocated when files start dl'ing qbittorrent/qBittorrent#6195
...and mentioned here: qbittorrent/qBittorrent#6421

...and probably a bug in how this is supposed to work:
Make the "Preallocate" option behave in a sane manner qbittorrent/qBittorrent#1072

[x8009]

Hey, what those links actually means?

when starting to download with pre allocate and downloading everything selected it writes zeroes means the harddisk is 100% usage before it actually write the data. that's the issue I meant. I don't want it to write zeroes with pre allocate. how to fix the zero writing with pre allocate? enabled?

[Seeker2]

It's broken and there doesn't seem to be any way to pre-allocate without zero-filling.

[arvidn]

This was fixed by #2077 right?

[ssiloti]

I wouldn't expect #2077 to fix this. That PR is just an unrelated bug I happened to notice while investigating this issue.

[ssiloti]

I haven't been able to find a bug here, or at least no clearly "wrong" behavior. I've tested RC_1_1 and RC_1_0 and for me they both allocate files as intended.

Libtorrent only pre-allocates files on first write instead of on torrent add. There are reasons one may want either behavior so there's a trade-off here. Allocating on first-write seem perfectly reasonable to me.

I also see complaints about performance of pre-allocation, but that's a limitation of Windows which libtorrent can't do anything about. For security reasons Windows does not allow allocating uninitialized space in the filesystem without special privileges. Regular users lack that privilege so even though libtorrent tries to allocate non-zeroed space it typically fails.

[x8009]

I just don't want it to write zeros before. it seems that sometimes it works fine, and sometimes it doesn't. it depends on the torrents ? Utorrent and Qbittorrent needs to run as Administrator so it will have access.

For now it doesn't seem to write zeroes for me. I am not sure what I change, but it's like something I can't figure. Usually Utorrent and Qbittorrent write zeroes when the problem exists. but sometimes one of them write zeros and one not.

Writing a 40 Gb files or many RAR files + 1 ISO file is really harddisk killer if you will do it a lot of times, instead fragmantion you will get broken harddisk. even when I have Enterprise harddisks . WD Gold Datacenter Hard Drive 6 TB 128 MB cache, Seagate 5TB Enterprise Capacity 128 MB (ST5000NM0024).

[arvidn]

I suppose we could add a performance_alert to indicate when the attempt to SetValidData() fails or succeeds. Normally it's not expected to succeed though, so the default should probably be to indicate success.

[x8009]

I suppose we could add a performance_alert to indicate when the attempt to SetValidData() fails or succeeds. Normally it's not expected to succeed though, so the default should probably be to indicate success.

would that help ?

For the moment Qbittorrent download properly with Pre-Allocate and no zeroes writes. it starts to download and write data at the same amount of download speed / Write speed. so it doesn't stays 99% usage and download 400 kb, but it write 100-200 mb and download 300 mbit or so. if that explain it right.

[arvidn]

It would help pinpoint where things go wrong. if you run as administrator I would expect SetValidData() to succeed, and to not zero-fill the files.

[x8009]

It would help pinpoint where things go wrong. if you run as administrator I would expect SetValidData() to succeed, and to not zero-fill the files.

So how can you do it, and how can I post you this information ?

[Seeker2]

I'm running as an admin and my ramdrive failed to complete the file creation for a torrent in ~10-15 seconds. Zero-filling it apparently takes awhile even on the ramdrive. :(

[RadarNyan]

It's 2020 now, is there any update on this?

I've recently made the switch from the old µTorrent v2.2.1 to qBittorrent v4.2.1, and found that "Pre-allocate disk space for all files" doesn't work as it says. The files only get allocated when download begins. After some search I ended up here.

I couldn't wrap my head around the logic behind this. Say I'm downloading two torrents both have many individual files in them, because of not all files are being allocated at the beginning, the files from both torrents are mixed together in a random manner (well, in the order they first begin to download) which I can't think of a single instance that someone would want that to happen. Yes the individual files are not fragmented but the folders are! To access the files in the same folder (from one torrent) my HDD would have to go back and forth many times to find them all.

@ssiloti commented before:

Libtorrent only pre-allocates files on first write instead of on torrent add. There are reasons one may want either behavior so there's a trade-off here. Allocating on first-write seem perfectly reasonable to me.

Please explain to me how is this reasonable. And even then, you said "There are reasons one may want either behavior" then why can't I get the other behavior? I mean - the other behavior is like a common practice, every single download software I used before does that (allocate all files to-be-downloaded before hand)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[arvidn]

Please explain to me how is this reasonable.

Allocating the file up-front mitigates it being fragmented. Fragmentation is believed to primarily be an issue within files, not across files.

And even then, you said "There are reasons one may want either behavior" then why can't I get the other behavior?

It hasn't been implemented yet. patches are welcome.

the files from both torrents are mixed together in a random manner (well, in the order they first begin to download) which I can't think of a single instance that someone would want that to happen.

people that want to allocate disk space lazily, and thus use their space more efficiently, probably.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.