Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hash check fails for random blocks in complete torrent. #11971

Open
BiochemLouis opened this issue Feb 6, 2020 · 12 comments
Open

Hash check fails for random blocks in complete torrent. #11971

BiochemLouis opened this issue Feb 6, 2020 · 12 comments

Comments

@BiochemLouis
Copy link

@BiochemLouis BiochemLouis commented Feb 6, 2020

Please provide the following information

qBittorrent version and Operating System

Raspbian Lite GNU/Linux 10 (buster) kernel: 4.19.88-v7l+
qbittorrent-nox 4.1.5-1 armhf

If on linux, libtorrent-rasterbar and Qt version

libtorrent-rasterbar: 1.1.11-2 armhf
qt5: 5.11.3+dfsg1-1+rpi1armhf

AND

qBittorrent version and Operating System

Fedora release 31 (Thirty One) kernel: 5.4.15-200.fc31.x86_64
qbittorrent 4.2.1-1 x86_64

If on linux, libtorrent-rasterbar and Qt version

libtorrent-rasterbar: 1.1.13-4 x86_64
qt5: 5.13.2-2 x86_64

What is the problem

Completed torrent does not pass hash check on qbitorrent, reports several blocks missing.

What is the expected behavior

Hash check should detect the download is complete and start seeding.

Steps to reproduce

  1. Download a torrent file containing a somewhat large number (5k+) files (this one concerns 2019-nCov and is found on the-eye.eu).
  2. Complete torrent download using one client and stop transfer.
  3. Add torrent to qBittorrent, hash check fails for several blocks (seemingly random blocks but the same ones over several repeats and machines).

Nothing in the log besides added to download list.
Would have resulted in data loss if I had not made the files immutable before transfer.
Files on an NFS share so they are not changing between the machines.
Tested qBittorrent and transmission on both machines, hash check works in transmission.

Screenshot from 2020-02-06 01-25-27

Screenshot from 2020-02-06 01-24-38

@FranciscoPombal

This comment has been minimized.

Copy link
Member

@FranciscoPombal FranciscoPombal commented Feb 6, 2020

See this comment from a similar issue: #11901 (comment)

TL;DR: check your RAM.

@BiochemLouis

This comment has been minimized.

Copy link
Author

@BiochemLouis BiochemLouis commented Feb 6, 2020

See this comment from a similar issue: #11901 (comment)

TL;DR: check your RAM.

I'll do that now, but its odd it fails on the same blocks on two completely different machines (workstation and raspberry pi) while transmission works.

@FranciscoPombal

This comment has been minimized.

Copy link
Member

@FranciscoPombal FranciscoPombal commented Feb 6, 2020

See this comment from a similar issue: #11901 (comment)
TL;DR: check your RAM.

I'll do that now, but its odd it fails on the same blocks on two completely different machines (workstation and raspberry pi) while transmission works.

It is indeed odd that it fails on 2 different machines across 2 different versions even, but it is better to be safe than sorry. Also, I completely missed the part where you said it worked in transmission. Would you mind also trying with btcheck? btcheck -lvi torrent

@BiochemLouis

This comment has been minimized.

Copy link
Author

@BiochemLouis BiochemLouis commented Feb 6, 2020

I had to work this afternoon so I have not run memcheck yet.

For the rest, I tested btcheck on the Pi.

  • btcheck -lvi torrent lists all the files I expect
  • btcheck -i torrent returns Data integrity checked, all is ok.
@FranciscoPombal

This comment has been minimized.

Copy link
Member

@FranciscoPombal FranciscoPombal commented Feb 6, 2020

It's looking more and more like this could indeed be an issue on qBittorrent's side. Could the fact that the files are in an NFS share be playing a role?

Please report back the result of the memory check and also link here the torrent in question. I want to try to download it as well on a linux box to rule out the possibility that the problem lies with the torrent itself.

@BiochemLouis

This comment has been minimized.

Copy link
Author

@BiochemLouis BiochemLouis commented Feb 6, 2020

Just turned off the nfs-kernel-server and unexported it, on the Pi qbittorrent check fails, btcheck succeeds.

torrent: https://the-eye.eu/CoronaVirusPapers/coronaviruspapers.torrent

I haven't tried downloading to qbittorrent directly, since that's not my workflow for archives.
Either use another client or get the files directly from:

Just check the file tree in the torrent for organisation.

@BiochemLouis

This comment has been minimized.

Copy link
Author

@BiochemLouis BiochemLouis commented Feb 10, 2020

Just quickly confirming no issue with memory on my Fedora machine, I do not know how to test it on the Pi.

@FranciscoPombal

This comment has been minimized.

Copy link
Member

@FranciscoPombal FranciscoPombal commented Feb 10, 2020

Just downloaded the linked torrent and it rechecked just fine. Try downloading it to plain old local disk and rechecking it. Let's try to narrow this down to either the organization of the folders or a problem with the nfs.

@BiochemLouis

This comment has been minimized.

Copy link
Author

@BiochemLouis BiochemLouis commented Feb 10, 2020

I actually downloaded a fresh copy of the files on my Fedora machine this afternoon to check that, x86, no NFS, no external drive, just ext4 NVMe drive and in the Downloads folder.

Same blocks are failing after remove and re-add. Transmission and btcheck both check fine.

Screenshot from 2020-02-10 20-28-32

Screenshot from 2020-02-10 20-28-23

@FranciscoPombal

This comment has been minimized.

Copy link
Member

@FranciscoPombal FranciscoPombal commented Feb 13, 2020

The problem must be on your end. I tried removing and re-adding after completing the torrent. I aslo tried removing, restarting, and only then re-adding, and all was ok.

You mention you are downloading the files manually and only then adding the torrent to the client. Perhaps you are missing something?

Another idea: perhaps you are running into some limit, like max number of simultaneously open file descriptors. Check it with cat /proc/$PID/limits, where $PID is the PID you your qBittorrent process.

@BiochemLouis

This comment has been minimized.

Copy link
Author

@BiochemLouis BiochemLouis commented Feb 17, 2020

So, this is where I see the first difference between the Pi and my desktop.

    Pi
Max open files            1024                 524288               files
    Fedora
Max open files            524288               524288               files     

The soft limit of the Pi is under the number of files just in this torrent, bit I'm sure I don't go over the hard limit even with the other torrents.
On Fedora, I have just this one torrent and those 5k files are way under both the soft and hard limit.

I checked the same thing for btcheck, both report the same.

Max open files            1024                 524288               files     

Nothing else has changed.

Filesystem also has nothing to do with it, I have ext4 and xfs on the pi, ext4 and zfs on the desktop, all check out good in btcheck.

@FranciscoPombal

This comment has been minimized.

Copy link
Member

@FranciscoPombal FranciscoPombal commented Feb 18, 2020

I don't know what else it could be, frankly. If you gather any new information, let us know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.