Add "Download in Sequential Order" to Torrent Settings popup window.

[digitalnomad91]

https://i.imgur.com/rfxFxzi.png

[Supralateral]

Adding a global default option (from Alt + O) would be superb as well.

[Seeker2]

Duplicate request with:
#6998

Global option will not be done, due to issues it can cause:
https://qbforums.shiki.hu/index.php/topic,4549.msg23307.html#msg23307
https://qbforums.shiki.hu/index.php/topic,4549.msg23343.html#msg23343
https://qbforums.shiki.hu/index.php/topic,4549.msg25440.html#msg25440

[Supralateral]

Global option will not be done, due to issues it can cause:

@Seeker2, the only actual issue it can cause, as listed from any of your links, is that "sequential downloading can make the next-to-last pieces (before the very end piece) rarer in the event someone stops seeding after they finish watching the video, since they'll have very little time to share the last few pieces."

The other "issues" mentioned are purely FUD. The main point seems to concern the scenario wherein the download speed cannot keep up with the bitrate of the video file. Do note that, firstly, sequential downloading can be performed on any torrent and not just video files. Secondly, that sequential downloading doesn't necessarily translate to instantaneous streaming (there are various benefits to sequential downloading, such as improved disk I/O, which qBittorrent is renowned to be poor at). Thirdly, it is not the client's fault if either the swarm or the downloader cannot transfer traffic fast enough to keep up with realtime file playback. So all of these seem irrelevant to me.

It's a shame that I'm limited in my download options considering I have thousands of torrents seeding in the client, my global client ratio is positive, and all of my torrents are overseeded in the first place.

I also wonder why per-torrent sequential downloading was implemented at all if it's considered to be problematic?

[Seeker2]

Firstly, sequential downloading (Seq. DLing for short) non-video files can help a little with disk I/O, as qBT is terrible with that.
But I'd prefer qBT's (and libtorrent's) I/O issues were improved instead of making Seq. DLing mandatory.

Secondly, a major use for Seq. DLing is video streaming, even if it's used for other file formats as well. And there a high UL rate is needed, although ironically not by the video streaming peer - which may make the torrent swarm worse off for it.

Thirdly, you claim it's not a clients fault if it ignores some core rules of BitTorrent protocol (random/rarest pieces first -- at least from seeds, also partially disabling Tit-For-Tat with other peers) potentially sabotaging a torrent swarm?!
Those rules are so important that in the absence of oversupply (meaning: LOTS of fast seeds) BitTorrent falls apart without it:
https://en.wikipedia.org/wiki/BitTorrent#cite_note-Rarest_First_and_Choke_Algorithms_Are_Enough-11

Lastly, it was added because people demanded it and it was in other BT clients, not because it wasn't "problematic":
New download more pieces in "Download first and last pieces first" feature broke "Download in sequential order" #5078
3.3.10 OSX download sequential order dont working #6442
Sequential download problem #6532
Sequential download not working #6855

[Supralateral]

But I'd prefer qBT's (and libtorrent's) I/O issues were improved instead of making Seq. DLing mandatory.

Me too, sadly I think that is a lot harder than just extending the sequential downloading functionality to the global options menu. I'd rather you didn't misrepresent my request, because I'm not suggesting we "make Seq. DLing mandatory".

although ironically not by the video streaming peer - which may make the torrent swarm worse off for it.

There isn't anything ironic about it - that's simply how P2P works. The main issue I have with the argument that sequential downloading can butcher the swarm, that's really only true if the downloader stops seeding the torrent as soon as they've consumed the stream (as in, suiting the classic definition of a leech). Anyone inclined to do so would already pause the torrent as soon as they have downloaded the data anyway, regardless of sequential downloading.

Piece prioritisation is very normal, I'd like you to point out in any BitTorrent spec or draft of spec which requires the use of the Rarest First algo. The paper you link also states "We argue in the following that tit-for-tat fairness is not appropriate in the context of peer-to-peer file replication.", so I think your opinion on sequential downloading may be a little bit unbased.
The paper also fails to mention anything about sequential seeding, or even any other piece algorithm than Rarest First. I don't know how you read the paper and took away that piece prioritisation is "sabotage" or that it makes "BitTorrent fall apart".

Let's not forget that if the sole seed disconnects from the swarm, no matter what pieces are prioritised by the downloader, half of a file with completed pieces spread out randomly is just as - if not more - useless than half of a file with sequential pieces. And the uploader can upload pieces just as fast sequentially as it can randomly.

The issues you linked are also pretty irrelevant to the discussion, as far as I can understand them. When I mentioned that sequential seeding was deemed as problematic, I was referring to the swarm health, not an incorrect (broken) interpretation by the users.

---

May I suggest, if sequential downloading is really seen as so heinous, that it only be enabled on torrents above a certain seed:leech ratio? What number would be sufficient: 10, 20, 50 seeds per leech?

[Seeker2]

"I'd rather you didn't misrepresent my request, because I'm not suggesting we "make Seq. DLing mandatory"."

Many Seq. DLing peers on the same torrent swarm can cause nearly Seq. DLing even for peers not actively using it and even if Seq. DLing peers are not the majority. (I'll explain that later.)
Possible example: #7831 (Yes, I know that's a rigged example. But I've seen it on other torrents myself when I was using uTorrent.)

"I'd like you to point out in any BitTorrent spec or draft of spec which requires the use of the Rarest First algo"

Superseeding http://www.bittorrent.org/beps/bep_0016.html
"As clients connect, it will then inform them that it received a piece -- a piece that was never sent, or if all pieces were already sent, is very rare."
A piece that has never been sent is pretty rare, especially if only the initial seed has it.
So I think that qualifies.

"Prior to this, a seed might have to upload 150% to 200% of the total size of a torrent before other clients became seeds."
That relative effectiveness needs a little history mentioned -- In 2003, BitTornado became the first BitTorrent client to implement the algorithm. https://en.wikipedia.org/wiki/Super-seeding
It was codified into a BEP around 2008 or earlier, when Seq. DLing was far rarer.
Regular BitTorrent peer+swarm behavior was already "brittle" (BitTyrant and hostile peer studies give examples) otherwise such an add-on would've been unnecessary and even harmful:
https://qbforums.shiki.hu/index.php/topic,2859.msg13616.html#msg13616
https://qbforums.shiki.hu/index.php/topic,5037.msg25196.html#msg25196

On a new torrent swarm that has some Seq. DLing peers and regular peers, creating new seeds might be much quicker if the original source seed uses Super Seeding.

Another partial example:
http://www.bittorrent.org/beps/bep_0003.html ...makes mention of "Incentives Build Robustness in BitTorrent" at http://bittorrent.org/bittorrentecon.pdf
Page 3 of 5, section 2.4.2 Rarest First
The last paragraph of that section explains why Rarest First is CRITICAL to keeping torrent swarms alive.

Admittedly, the VERY NEXT section (2.4.3 Random First Piece) seems to contradict the idea that Rarest First is always the way to go. But that's because it's important for new peers to quickly have something to share rather than always having a rare piece to share.

Even the 3.1 Pareto Efficiency section indirectly implies Rarest First. If a peer initially gets only pieces ALL other peers already have, then it has nothing to share with them and its potential UL speed would be wasted unless it uploads to new peers that have nothing.

3.5 Upload Only section says:
"Once a peer is done downloading, it no longer has useful download rates to decide which peers to upload to. The current implementation then switches to preferring peers which it has better upload rates to, which does a decent job of utilizing all available upload capacity and preferring peers which no one else happens to be uploading to at the moment."
That's not Rarest First, which can cause problems.

"The paper you link also states "We argue in the following that tit-for-tat fairness is not appropriate in the context of peer-to-peer file replication.", so I think your opinion on sequential downloading may be a little bit unbased."

Context! In 4.2.1 Fairness Issue section, it states that:
BitTorrent "does not implement a bit level tit-for-tat, but a coarse approximation based on short term download estimations.
Moreover, it is believed that a fair peer selection strategy must enforce a byte level reciprocation. We call this notion of fairness, tit-for-tat fairness."
Then says... "tit-for-tat fairness is not appropriate in the context of peer-to-peer file replication protocols like BitTorrent."
Which to me means the "coarse approximation" BitTorrent uses is much better, but it is still a form of reciprocation.

"I don't know how you read the paper and took away that piece prioritisation is "sabotage" or that it makes "BitTorrent fall apart"."

From the paper:
"We show that the rarest first algorithm guarantees close to ideal diversity of the pieces among peers. In particular, on our experiments, replacing the rarest first algorithm with source or network coding solutions cannot be justified."
I wish to repeat parts of that: "rarest first algorithm guarantees close to ideal diversity ...replacing the rarest first algorithm ...cannot be justified."
That's strangely so sternly worded, that I probably can't find anything that is so pro "Rarest First!" anywhere.

Ok, my explanation:
When only "pure" Seq. DLing peers are available... Each one sees the others as piece sources (like seeds) or leeches that only have pieces it already has. No reciprocation (Tit-For-Tat) would occur.

In mixed swarms of both regular peers and Seq. DLing peers...
"Pure" Seq. DLing peers only download the next few pieces in a row that they want, but only regular peers with nearly everything (and want nothing in return) are likely to have those Seq. pieces.

Regular peers will "happily" download from Seq. DLing peers... but the regular peers are unlikely to have the next piece the Seq. DLing peer wants in return.
As a result, regular peers will often get choked and/or snubbed by Seq. DLing peers - causing their download speeds to slow.
End result - Seq. DLing peers will usually be unable or unwilling to Tit-For-Tat with regular peers.

Because the regular peers can only download from Seq. DLing peers the pieces those have, regular peers will come to resemble the Seq. DLing peers in piece availability unless seeds are much faster and/or outnumber peers.
Seq. DLing peers will be demanding "low-number" pieces near the start of the torrent from seeds despite their relative commonness among all the peers. Other peers won't be willing to instantly upload common pieces - they hopefully have almost all of their own upload slots filled with peers that are consistently giving them something in return. (reciprocation/Tit-For-Tat)

Seeds don't reciprocate or do Tit-For-Tat with peers, they already have everything and want nothing in return. They won't even favor any remaining peers that helped them become seeds. Instead, seeds keep upload slots to "peers which it has better upload rates to".
This won't be peers that are downloading at near-max already or overloaded by trying to upload too fast (thus stuck in bufferbloat) to other peers.

"There isn't anything ironic about it - that's simply how P2P works."

All other things held equal, seeds are more likely to upload to Seq. DLing peers (due to their reduced reciprocation) than regular peers.
As a result, seeds will spend less time uploading rarer pieces, so even regular peers will "fill up" predominately with the same pieces that the Seq. DLing peers have.

"The main issue I have with the argument that sequential downloading can butcher the swarm, that's really only true if the downloader stops seeding the torrent as soon as they've consumed the stream (as in, suiting the classic definition of a leech)."

Torrents can enter "feast-or-famine" where the remaining seeds cannot keep up with demand, and all the peers are "stuck" with the same completed pieces till a seed finishes uploading a complete piece to a peer.
This can happen even when there's significant numbers of seeds. (>10% of the total peers)

qBT/libtorrent goes into a pseudo Endgame Mode almost any time a piece is downloading slowly and other peers have the same piece:
#182 (comment)
This compounds Seq. DLing issues, because with large pieces (like >1 MB size) 16KB blocks inside a single piece may be DLed from multiple seeds+peers (with increasing wastage) and saved randomly, especially if sparse files mode is used on the storage drive - losing much of the Disk I/O advantages Seq. DLing might have.

[Supralateral]

What a great response, thank you very much for the effort you put into it.

You raise a lot of good points and I agree with most of them, some I could probably contest a bit but that's neither here nor there... the very first issue you linked taught me torrents with strong availability (20+) are automatically downloaded sequentially, which is all I need - and by all accounts, a much smarter process than simply enabling a global default. Therefore I retract my initial comment, thanks again for the lengthy replies/discussion.

[thalieht]

Will be in 4.1.2.