set location OVERWRITES a COMPLETED torrent

[tehmasterer]

Please provide the following information

qBittorrent version and Operating System

4.0.4 Windows 10 (10.0.1xxxx)

What is the problem

Using set location on a PAUSED, COMPLETED torrent will OVERWRITE the completed file. I just lost multiple shows because I thought it was moved and seeding. NOPE it was download and overwriting the entire thing again.

What is the expected behavior

set location should move a completed torrent. simple as that. the whole point of set location is so you can easily organize torrents and not have to force recheck.

Steps to reproduce

pause torrent > set location > watch torrent start overwriting

Extra info(if any)

this is disastrous. no other client has this problem.

I would like to add one tv show started downloading 20%. I played the file and it was all still OK. So I removed the torrent from qbittorrent. added it again, selected location on the hard drive, and force recheck. IT WENT TO 20% again. I checked the file AND IT OVERWROTE it. It doesn't matter if you remove the torrent and add it again, if even 1% is downloaded it will OVERWRITE your file.

[sledgehammer999]

Your description is a bit confusing.

added it again, selected location on the hard drive, and force recheck. IT WENT TO 20% again. I checked the file AND IT OVERWROTE it

The file was 20%. You re-add it and it goes to 20%. But then you claim it overwrote it. How is that?

[tehmasterer]

I wanted to move the torrent, so I paused it and chose set location to another path on the drive. After I set location the torrent, which was completed weeks ago, started overwriting itself. I didn't notice it first but it got as far as 20%. I removed the torrent and added the link again to try and seed. My added torrents are always paused, so I set location, force recheck, and it went to 20%. I checked and played my file. It was overwritten with modification dates changed.

[Charles2904]

I have just used the "Set location" option to move a completed torrent with a single 40MB file from a hard drive to an SSD. It worked as I expected. It did exactly what uTorrent would have done. There was no indication that the torrent data was downloaded from the internet again.

Your description is still very unclear.

What exactly do you mean by "started overwriting itself"? When you say "overwrote" do you mean that qBittorrent has downloaded the file again? Do you mean that the torrent state changed to "Downloading"?

What exactly do you mean by " it got as far as 20%"? Do you mean that the torrent went to the "Downloading" state but only got as far as 20% complete? Or do you mean that when you do a recheck the result is that the torrent is indicated as only 20% done?

If you, or qBittorrent, move or copy a large file it is possible that you will see a Windows dialogue box showing the progress of the file move or copy. Is this what you are seeing? Are you mis-interpretting this as the file being downloaded from the internet again?

How big is the file and how long does it take to be "overwritten" as you describe it? If the file is large but the "overwriting" takes only seconds then the chances are that you are mis-interpretting a Windows message.

If you move, copy or modify a file in Windows then some or all of its time and date stamps will change. If you don't fully understand file date stamps in Windows then you shold not use them as an indication that the file has been downloaded from the internet again.

If you resolve this problem then please tell us how you fixed it !!!

[WeskerEnd]

Okay, this just happened to me, I lost 24GB.

Let me try to explain. I had downloaded this file before, and since then, I moved it to another folder.

Now that I did what I had to do with it, I wanted to keep seeding it, but since I moved it, qBitTorrent doesn't know where it is.

So I right-clicked the torrent name in qB, clicked "Set Location", and INSTEAD OF CHECKING the file and validating it, it simply started downloading the full torrent again as if there were no files in that folder, which means it overwrote my complete file with a new 0% file of the same name.

Everytime I did this with µTorrent, it saw that the files were already there and instead Checked the torrent, now I'll have to download 24GB again if I want to keep seeding it. There were no warnings or anything.

EDIT: qB v4.1.3 / W10

[BlockABoots]

Yep just had this happen to me!!!!!!!!!!!!!!

Lost 300GB worth of data!! Rather than the program checking to see if i already had the file it simply wiped the previous files and started downloading again. FUMMING!!!

[sledgehammer999]

@BlockABoots which qbt version are you using? And which OS?

I don't know how this keeps happening this was fixed in #7207 by @glassez in 2017!

[sledgehammer999]

@glassez I figured it out. From the docs in torrent_handle.hpp:

dont_replace always keeps the existing file in the target
directory, if there is one. The source files will still be removed in
that case. Note that it won't automatically re-check files. If an
incomplete torrent is moved into a directory with the complete files,
pause, move, force-recheck and resume. Without the re-checking, the
torrent will keep downloading and files in the new download directory
will be overwritten.

We never do pause-recheck-resume thus the file gets overwritten by downloading data. Do you have time to take another stab at it?

[glassez]

Strange, why didn't I notice this part of the documentation at the time?..

Do you have time to take another stab at it?

Unfortunately, I will be very busy in my real life in the next few months, so I will not be able to devote as much time to the project as before. Apparently, I won't even be able to review other PRs as intensively as before (@Chocobo1, please note it too).

[BlockABoots]

@BlockABoots which qbt version are you using? And which OS?

I don't know how this keeps happening this was fixed in #7207 by @glassez in 2017!

Im on version 4.1.5 and am using Windows 10

[sledgehammer999]

Strange, why didn't I notice this part of the documentation at the time?..

Out of curiosity I did a git-blame on the torrent_handle.hpp. And it isn't your fault. The documentation about that flag was clarified only ~10 months ago.

[tarcs999]

I have had this issue just now:

• Completed file was in my shiny new hard drive
• qB pointing to a very incomplete file on my old hard drive (a few MB in size of a 2GB file)
• Paused + rechecked + used "Set Location" to point to the complete file; instantly overwrote the complete file on my new hard drive with the incomplete file

qB version 4.1.6 built w/ Qt 5.12.2, Libtorrent 1.1.13.0, Boost 1.70.0

Is this unexpected behaviour post #7207 or am I misinterpreting how I should be using this command?

[Piccirello]

Duplicate of #127

[sledgehammer999]

@glassez May I remind you of this? Specifically see #8758 (comment)
Do you want to look into it since you did some related work in master lately?

[glassez]

Do you want to look into it since you did some related work in master lately?

I've been thinking about it lately...

[TomArrow]

I had two folders on my hard drive, unsure which one was the completed one. I pointed the torrent on the uncompleted one by accident. It checked to around 10%.

Then I pointed it to the completed one. Guess what happened? Qbit overwrote the completed files with the 10% files.

This is absolutely horrendous and unacceptable. These are files that I've waited two years for a seeder for! It's sheer luck that I still have a backup of them on my seedbox.

It's unbelievable that this doesn't get fixed. This should be priority number 1. Whoever programmed this had zero respect or consideration for the final user's data.

This issue was well-explained and actually got the "data-loss" tag to show how important it is but got closed because "muh duplicate" and that badly formulated very abstract issue stays up as the original.

Again, UN-FUCKING-BELIEVABLE.

[MontFox]

Just happened to me. Downloaded 60GB torrent on laptop to an external drive. Scanned it once on laptop, no issue. Plugged drive into desktop and pointed QBittorent to folder. Scanned it initially, then progress was back at 20%

[glassez]

Seems it is fixed in latest libtorrent. It rechecks after moving storage if there are some appropriate files exist in destination directory.

[Piccirello]

Is that change listed on the libtorrent changelog? I don't see one, but I'm likely overlooking it.

[glassez]

Is that change listed on the libtorrent changelog?

I don't know.
I accidentally came across this logic in the libtorrent code when I was looking at it for some reason recently. Then I ran some tests to check it.

[Piccirello]

Any chance you can recall what file/line it was? We can check the history from there.

[glassez]

https://github.com/arvidn/libtorrent/blob/ad83b1c0eb293b63c69f7879ca6ba2381369f77f/src/torrent.cpp#L7789

[Piccirello]

It doesn't appear this part of the logic is new. This is a pretty superficial search, but I don't have much to go off of.

libtorrent:RC_1_2 $ git blame -L 7789,-2 -L 7789,+2 src/torrent.cpp
60df41cd85 (arvidn        2015-11-29 20:31:57 -0500 7788)                       set_need_save_resume();
15ab8f387b (Arvid Norberg 2016-11-26 01:51:47 -0500 7789)                       if (status == status_t::need_full_check)
982a14c2e9 (Arvid Norberg 2013-05-09 02:50:16 +0000 7790)                               force_recheck();
libtorrent:RC_1_2 $ 
libtorrent:RC_1_2 $ grep -nrw src -e "need_full_check"
src/disk_io_thread.cpp:2576:		// or if error is set and return value is 'no_error' or 'need_full_check'
src/disk_io_thread.cpp:2597:				return status_t::need_full_check;
src/storage_utils.cpp:281:				if (ret == status_t::no_error) ret = status_t::need_full_check;
src/torrent.cpp:7783:			|| status == status_t::need_full_check)
src/torrent.cpp:7789:			if (status == status_t::need_full_check)
libtorrent:RC_1_2 $ 
libtorrent:RC_1_2 $ git blame -L 2597,-2 -L 2597,+2 src/disk_io_thread.cpp
ccd539f977 (arvidn        2016-11-11 22:01:18 -0500 2596)                               }
15ab8f387b (Arvid Norberg 2016-11-26 01:51:47 -0500 2597)                               return status_t::need_full_check;
ccd539f977 (arvidn        2016-11-11 22:01:18 -0500 2598)                       }
libtorrent:RC_1_2 $ 
libtorrent:RC_1_2 $ git blame -L 281,-2 -L 281,+2 src/storage_utils.cpp
ec37436d49 (Arvid Norberg 2017-01-17 08:02:44 -0500 280)                        {
ec37436d49 (Arvid Norberg 2017-01-17 08:02:44 -0500 281)                                if (ret == status_t::no_error) ret = status_t::need_full_check;
ec37436d49 (Arvid Norberg 2017-01-17 08:02:44 -0500 282)                                continue;

[sledgehammer999]

@glassez since this an old issue and you might have forgotten, when moving we should implement extra logic to handle the move's actions. (link: http://libtorrent.org/reference-Storage.html#move_flags_t)

[glassez]

we should implement extra logic to handle the move's actions

What "extra logic" do you mean exactly?

[sledgehammer999]

I think I have mentioned it here: #8758 (comment)

[glassez]

I think I have mentioned it here: #8758 (comment)

The fact is that when I started working on this, I found that it works correctly: when an existing file is found in the target folder, the torrent is automatically transferred to "checking" state.
At first I thought it was fixed recently. But as it turns out, it existed even when this "confusing" comment you're referring to was added.

Here the appropriate status is set and here it is handled and torrent resets its state to "checking".

@arvidn?

[arvidn]

is there some behavior in libtorrent here that's surprising or other than what's documented?
or is there some desired behavior that is not supported at all?

@glassez would you mind summarizing what you've found so far?

[glassez]

@glassez would you mind summarizing what you've found so far?

Sure. I will do it ASAP. I need to perform some additional tests before.

[FranciscoPombal]

@glassez is this still relevant?

[glassez]

@glassez would you mind summarizing what you've found so far?

We talking about the case when we move torrent storage and want to use existing files in target location.
In reality, we have two contradictory things here. On the one hand, there is the above-mentioned comment in the libtorrent code, according to which it requires some additional actions to prevent existing files from being reset to current download progress (which is illogical from the user's perspective, since it turns out that the "function" does not perform the stated work). On the other hand, here is real lbtorrent code that sets the necessary flag itself if the existing file was used during the move, and automatically activates the check after the move is complete (this is quite expected behavior).
I tried to test it. I moved unfinished torrents (both actively downloading and paused) in the location with complete files and it uses the existing files (after recheck) in most cases. Only once existing file was reset to original torrent progress. But I never be able to reproduce it again.
So I'm confused...

[FranciscoPombal]

Well, if anyone has problems or finds a way to reproduce such a problem in the latest version, please open a new issue report.

[Piccirello]

Why would we lock this? This is still very much an issue. I experienced this a few days ago when redownloading a torrent that hadn't been moved out of my downloads folder, and @glassez was able to confirm reproducing it once.

[FranciscoPombal]

@Piccirello sorry, I thought this was no longer reproducible. Do you happen to have more information?

[Piccirello]

I don't have anything too substantive, but re-adding a previously completed torrent resulted in a full re-download on 4.2.5 with libtorrent 1.2.6. The file had failed to move out of the download folder to the completed folder. I wanted to re-trigger the move to debug, so I re-added the torrent. Upon re-adding the torrent it never rechecked and instead started from 0. I didn't trying debugging the issue any further.

You might have more luck in #127.

[glassez]

@Piccirello

[glassez]

@Piccirello, you are probably affected by another bug (or several bugs).

[luzpaz]

is this ticket still relevant?

[TomArrow]

is this ticket still relevant?

Why wouldn't it be? Has there been a fix?

[hmpfkafka]

holy fucking shit i just lost ALL of the data i had on my hard drive due to this bug, how the actual FUCK is this still an issue??? more than 100 gigs of private tracker data just gone to the dump, i literally can't express how angry i am. why would it start downloading the files when it's pointed in the location where the files are already there, i'm so frustrated and confused, i'm actually done.

[TomArrow]

holy fucking shit i just lost ALL of the data i had on my hard drive due to this bug, how the actual FUCK is this still an issue??? more than 100 gigs of private tracker data just gone to the dump, i literally can't express how angry i am. why would it start downloading the files when it's pointed in the location where the files are already there, i'm so frustrated and confused, i'm actually done.

Sorry to hear mate. Doesn't seem they care.