Error when downloading new torrents

[escape0707]

Qbittorrent 4.4rc1 built from AUR
Libtorrent 2.0.5 built from AUR
Archlinux 5.15.11
Ntfs3

When I'm downloading a torrent into a ntfs3 mounted external hdd drive, qbit always first create those files waiting to be downloaded, then it fails and spit a log with:

2021/12/03 15:47 - File error alert. Torrent: "xxx". File: "/run/media/jay/Seagate Basic/my-files/Downloads/Bittorrent/xxx/yyy.mkv". Reason: xxx file_open (/run/media/jay/Seagate Basic/my-files/Downloads/Bittorrent/yyy.mkv) error: resource temporarily unavailable

If I try resume several times, it will retry and fails then finally finish creating those files and continue to download.

When using the same setup to download to $HOME which is on the internal ssd and formatted as ext4, there is no problem.

I run into this problem even when downloading a torrent with only 9 files.

[arvidn]

I believe this is a bug (or missing feature) in ntfs-3g. libtorrent supports a disk I/O back-end that doesn't use memory mapped files (which is the default in 32 bit builds). I don't know if qbt lets you pick this though, but it could.

It would have to use the posix_disk_io_constructor. That might fix it. I'm suspecting the ntfs filesystem is having a hard time with memory mapped files.

[escape0707]

I'm using Linux kernel's new ntfs3 implementation, not the NTFS-3G one. I think NTFS-3G works, but I'm not sure. Should I try it out?

BTW, if the files are already created through qbt's several failed trials, there won't be any problems during subsequent downloading and uploading of this torrent.

The problem only happens when there are still files to be created.

[arvidn]

I see, I just assumed it was ntfs-3g you were using.

What's puzzling is that the man page for open() doesn't list EAGAIN as a possible error.

Looking over the code, there seems to be an error where the true operation field may get clobbered by file_open. Would it be possible for you to test this patch? #6617

If so, the error message may be a bit more helpful.

[arvidn]

it seems likely that it's the mmap() call that's failing with EAGAIN (man page).

EAGAIN The file has been locked, or too much memory has been
locked (see setrlimit(2)).

[escape0707]

New error log after rebuild with the patch:

2021/12/29 12:58 - File error alert. Torrent: "xxx". File: "/run/media/jay/Seagate Basic/my-files/Downloads/Bittorrent/xxx/yyy.mkv". Reason: xxx file_mmap (/run/media/jay/Seagate Basic/my-files/Downloads/Bittorrent/yyy.mkv) error: resource temporarily unavailable

[escape0707]

Should I provide any else info?

[escape0707]

This is still happening when using qBt 4.4.

[arvidn]

something in the ntfs filesystem driver is failing with EAGAIN:

The file has been locked, or too much memory has been
locked (see setrlimit(2)).

So, perhaps it's causing memory to be locked exceeding your ulimit. To me, it smells like a quality-of-implementation issue in the ntfs filesystem driver.

[database64128]

Besides the Resource temporarily unavailable errors when creating files, I'm also seeing system lockups during initial file writes. Stats in btm/htop would stop updating and qBittorrent would stop responding for an extended period of time, until a large chunk of zeros is written to disk. It seems to be partially caused by the fact that NTFS can't "store several unwritten areas": https://github.com/torvalds/linux/blob/719fce7539cd3e186598e2aed36325fe892150cf/fs/ntfs3/file.c#L690-L700.

I did a little experiment to compare ntfs3 to other file systems. Create an empty 10G file, seek to 8G and write 4K:

truncate -s 10G test
dd if=/dev/urandom of=test bs=4K count=1 seek=2M status=progress

On btrfs, both truncate and dd operations complete immediately and barely any data is written to disk.

1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000123942 s, 33.0 MB/s

On ntfs3, truncate completes immediately. dd took 35s. ntfs3 basically zeroed out the entire 8G region before the actual data.

4096 bytes (4.1 kB, 4.0 KiB) copied, 35 s, 0.1 kB/s
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 35.4918 s, 0.1 kB/s

During my experiment, the system did not lock up, stats in btm/htop did not stop updating. But it is clear that ntfs3 is not suitable for complex IO loads like downloading a torrent. I would recommend moving away from NTFS as soon as possible.

[escape0707]

I'm also seeing system lockups during initial file writes. Stats in btm/htop would stop updating and qBittorrent would stop responding for an extended period of time

Qbittorrent itself is locked up, too.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[database64128]

ntfs3 maintainer just sent a pull request with a bunch of fixes for 5.19. Keep this issue open for further updates.

[afeuerstein]

I can reproduce on 5.18.6-artix Artix Linux w/ open-rc and ntfs3.
Example:

An I/O error occurred for torrent 'torrent'.
Reason: file_mmap (/ntfs/torrent/) error: Resource temporarily unavailable

qBittorrent v4.4.3.1

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.