Why does Transmission download .parts of files that I specified "not to download"?

[drkrol]

Hello. When I download a torrent, I download those files that I specified "not to download". Trying to solve this problem, I find records from 6-10 years ago with the exact same problem. I understand the problem is still not solved. Can you tell me if there is any way to make a script that removes unnecessary files?

I am using the console version on Ubuntu 20

[stefanos82]

If you don't mind have patience, this type of issues are about to be addressed as the project goes through a major code refactoring from C to C++; therefore, such bugs it would be easier to get resolved by volunteers who are willing to contribute.

I'm testing Transmission locally on my system as is my torrent client of choice; since the refactoring announcement on Hacker News, I came straight to help in any way possible.

So, have some faith and such peculiarities are going to get resolved sooner or later.

Cheers 👍

[drkrol]

If you don't mind have patience, this type of issues are about to be addressed

So, have some faith and such peculiarities are going to get resolved sooner or later.

Let's discuss again: are we talking about problems from 2007 that will be solved "soon"? There is a bug that for some reason has not been fixed for 14 years. Don't you think that's strange?

https://trac.transmissionbt.com/ticket/514

[stefanos82]

If you don't mind have patience, this type of issues are about to be addressed

So, have some faith and such peculiarities are going to get resolved sooner or later.

Let's discuss again: are we talking about problems from 2007 that will be solved "soon"? There is a bug that for some reason has not been fixed for 14 years. Don't you think that's strange?

https://trac.transmissionbt.com/ticket/514

I don't know whether I did not understand you correctly or not, but I have experimented a bit.

When you try to download a torrent or a manget link, it does not know in advance the number of files a link contains; therefore, it first needs to find the closest peers, connect with them, fetch the whole set of files, from 1 to any number of files it may contain, and then start downloading them.

After this procedure and right after the initialization of downloading the files, it lets you uncheck the files you don't want; thus the retaining on unnecessary files.

It has to respect the file structure you are supposed to download as a whole; it does not know your intentions nor that you want to download a single file out of, let's say 100 it may contain.

Update: I have tried the same procedure with uGet, a GUI front-end for aria2c and the behavior is exactly the same as Transmission; I cannot tell in advance the number of files a torrent or a manget link consists of in total so I can choose what I explicitly want to download.

[killemov]

The first phase you are talking about is getting the metadata. Use that wording. A file from the payload will only be created on disk when a piece is flushed from memory to disk. Some other torrent clients allocate all files before downloading.

[drkrol]

After this procedure and right after the initialization of downloading the files, it lets you uncheck the files you don't want; thus the retaining on unnecessary files.

I add a torrent file and immediately stop it
transmission-remote -a /var/host/my.torrent -S
Please note: not a single file has been downloaded yet.

Then I browse through the list of files with the command
transmission-remote -t 1 -f

For example, I get a list of 10 files. I don't want files 4 through 9 and I run the command
transmission-remote -t 1 -G 4-9

Then I start the download
transmission-remote -t 1 -s

The torrent starts and in the download folder I see files 4 and 5 in addition to files 0-3 for some reason.

I do not argue about the block system and that the files can not share and download separately, but why is it impossible to automatically delete these files after the end of the download? We know in advance which ones we need and which ones we don't.

[mraonea]

Transmission allocates the full size file to disk even though only a part of a piece of it would need to be downloaded, e.g. a 10GB file will take the whole 10GB of diskspace even though only a part of a 100kB piece from the beginning of the file would need to be downloaded.

So in effect Transmission wastes a whole 10GB of disk space just for zeroes for this particular file. Of course this adds up the more files only partially downloaded.

This could be fixed with sparse files ( https://docs.microsoft.com/en-us/windows/win32/fileio/sparse-files ) or some other solution.

[killemov]

This "fix" is just the usual assumption. Filesystems usually support sparse files.

[drkrol]

Transmission allocates the full size file to disk even though only a part of a piece of it would need to be downloaded, e.g. a 10GB file will take the whole 10GB of diskspace even though only a part of a 100kB piece from the beginning of the file would need to be downloaded.

What does this have to do with the allocation and the files that should have self-destructed?

Let's look at an example.

I add a torrent with the movie "Breaking Bad".
transmission-remote -a /var/host/4.torrent -S

I can see a list of files:

transmission-remote -t 5 -f
-The torrent file consists of 62 video files in .avi format. 

I can also get the total size of all the files:

transmission-remote -t 5 -i
Total size: 50.04 GB (50.04 GB wanted)

I only want to download season 2. That is files 7-19. Therefore I need to exclude files numbered 0-6 and 20-61. Let's do this:
transmission-remote -t 5 -G 0-6,20-61

Start the download
transmission-remote -t 5 -s

After the download is complete, look at

tree '/var/host/Breaking Bad'

/var/host/Breaking Bad'.
├── Season 01
│ └── BreakingBad.rus.S01E01.avi.part
├── Season 02
│ ├── BreakingBad.rus.S02E01.avi
│ ├── BreakingBad.rus.S02E02.avi
│ ├── BreakingBad.rus.S02E03.avi
│ ├── BreakingBad.rus.S02E04.avi
│ ├── BreakingBad.rus.S02E05.avi
│ ├── BreakingBad.rus.S02E06.avi
│ ├── BreakingBad.rus.S02E07.avi
│ ├── BreakingBad.rus.S02E08.avi
│ ├── BreakingBad.rus.S02E09.avi
│ ├── BreakingBad.rus.S02E10.avi
│ ├── BreakingBad.rus.S02E11.avi
│ ├── BreakingBad.rus.S02E12.avi
│ └── BreakingBad.rus.S02E13.avi
└── Season 03
    └── BreakingBad.rus.S03E01.avi.part

Let's check the size of the folder

du -sh '/var/host/Breaking Bad
9.8G /var/host/Breaking Bad
du -sh '/var/host/Breaking Bad/Season 02
9.8G /var/host/Breaking Bad/Season 02

Again, what kind of space allocation are we talking about? You are mistaken. The size of my drive is 42GB and the size of the entire series is 50GB. I wouldn't be able to download 13 episodes because I wouldn't have enough space. But that has nothing to do with it.

I care about 2 files. Why the file from season 1 and the file from season 2 were not automatically deleted is not clear to me. I repeat: this problem has existed since 2007!

Why can't you fix this mistake? I'm not a programmer, I'm a designer. But if I could, I would have fixed it right away, instead of waiting 14 years.

[valcomm]

This is not about allocation or transmission - this is how torrents work. Files are combined in one stream, divided into chunks and you download those chunks. The chunk with the first file of the season two didn't start right away - it started somewhere in the last file of the season 1. That's why you have that part. The same applies to the last file and part of the file of the next season.

The reason why they are kept on your disk - to allow you to share the whole chunks. If part of the previous season is deleted, you cannot share the beginning of the fist episode of the season two. Transmission doesn't want you to turn into a leecher and prevent you from sharing that first chunk that you downloaded.

This is not a bug. This is by design

[mraonea]

Technically they are called "pieces" and what I wrote is correct and is an issue with Transmission. Transmission allocates the full sized files, just to download a part of a piece, wasting disk space. That is a real bug, an old one, as can be seen from the breadcrumbs in trac, even with patches submitted to fix the issue but not accepted.

Transmission allocates the full size file to disk even though only a part of a piece of it would need to be downloaded, e.g. a 10GB file will take the whole 10GB of diskspace even though only a part of a 100kB piece from the beginning of the file would need to be downloaded.

So in effect Transmission wastes a whole 10GB of disk space just for zeroes for this particular file. Of course this adds up the more files only partially downloaded.

This could be fixed with sparse files ( https://docs.microsoft.com/en-us/windows/win32/fileio/sparse-files ) or some other solution.

[valcomm]

I'd say transmission does what its configuration says. What's your set value for:

preallocation: Number (0 = Off, 1 = Fast, 2 = Full (slower but reduces disk fragmentation), default = 1)

in JSON formatted configuration file?

[drkrol]

in JSON formatted configuration file?

"preallocation": 1,

I'm not quite sure what this parameter does? Can you please explain it again?

[valcomm]

I think we went off the topic of this issue. Which is the presence of unasked files. I hope I explained that this is by design and not a bug.

What you're asking about is the used disk space for such files. I've got '1' as well and mine .part files (the ones I didn't ask for) are small and don't take the whole size on disk. There's a difference between a size and a size on disc. In case of sparse files the former might be larger than the latter.
If someone uses not the valid options while requesting such files sizes confusion arises.
That might explain while you're getting the size on disk, while mraonea quoted a case where either the whole file was preallocated or size was asked. Or file system doesn't support sparse files.

[drkrol]

I think we went off the topic of this issue. Which is the presence of unasked files. I hope I explained that this is by design and not a bug.

About the question, it made a little more sense to me. But there is one point. It would be good - to add the ability not only to start and stop the distribution and download, but to have the ability to finish downloading the file. With this command
transmission-remote -t 1 -fin
all unnecessary files would be removed.

[valcomm]

I'd vote against such option. That prevents you from sharing all downloaded pieces.
In other words if everyone downloads Season 2 of The Breaking Bad and uses such options and original seed goes away there is no way for somebody new to download Season 2 completely.

I have such torrents for years in the queue. Cannot complete multi-gigabyte file because sources deleted .part file for not important assisting text file. So while they have the actual file, they cannot share it and let others get it.

[mraonea]

I'd say transmission does what its configuration says. What's your set value for:

preallocation: Number (0 = Off, 1 = Fast, 2 = Full (slower but reduces disk fragmentation), default = 1)

in JSON formatted configuration file?

It is the default value, which is 1.

That does not mean the correct behaviour is writing empty 10GB files to disk for a 50kB part of a single piece. That is probably one reason this issue keeps cropping up, the trac issue is complaining about the same thing and is 14 years old with patches submitted to fix it.

NTFS supports sparse files. I linked to NTFS documentation about them.