New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] First/last parts are downloaded after everything else if prioritizing them is not selected. #7696

Closed
mzso opened this Issue Nov 11, 2017 · 17 comments

Comments

Projects
None yet
5 participants
@mzso

mzso commented Nov 11, 2017

qBittorrent version and Operating System:

3.4.0b2 Windows

What is the problem:

When prioritizing First/lasp part not selected the first/last parts are loaded after all the rest of the file. The other parts are seem to be prioritizied

What is the expected behavior:

The pieces should be downloaded randomly

Steps to reproduce:

Nothing. Don't select the feature to prioritize the first/last parts of the file.

Extra info(if any):

An option to set downloading first/last part as default would be nice.

@shirofficial

This comment has been minimized.

Show comment
Hide comment
@shirofficial

shirofficial Nov 11, 2017

Really? Why are you using the version 2.4.0b2? You should download the latest one..

shirofficial commented Nov 11, 2017

Really? Why are you using the version 2.4.0b2? You should download the latest one..

@mzso

This comment has been minimized.

Show comment
Hide comment
@mzso

mzso Nov 11, 2017

@shirofficial commented on 2017. nov. 11. 22:53 CET:

Really? Why are you using the version 2.4.0b2? You should download the latest one..

Typo... I'm using 3.4.0b2 There's no newer than this.

mzso commented Nov 11, 2017

@shirofficial commented on 2017. nov. 11. 22:53 CET:

Really? Why are you using the version 2.4.0b2? You should download the latest one..

Typo... I'm using 3.4.0b2 There's no newer than this.

@mzso

This comment has been minimized.

Show comment
Hide comment
@mzso

mzso Nov 20, 2017

Here's an (apng) animation showing the bug. Taken from 4.0.0

qbt-progress-no-first-last-500

(Click the image to view it full size)

mzso commented Nov 20, 2017

Here's an (apng) animation showing the bug. Taken from 4.0.0

qbt-progress-no-first-last-500

(Click the image to view it full size)

@Chocobo1

This comment has been minimized.

Show comment
Hide comment
@Chocobo1

Chocobo1 Nov 20, 2017

Member

AFAIK qbt doesn't do anything special here. I would guess the seeders/peers did this on purpose...?
@arvidn is that possible?

Member

Chocobo1 commented Nov 20, 2017

AFAIK qbt doesn't do anything special here. I would guess the seeders/peers did this on purpose...?
@arvidn is that possible?

@arvidn

This comment has been minimized.

Show comment
Hide comment
@arvidn

arvidn Nov 20, 2017

in libtorrent 1.1 (or maybe 1.0) the default file and piece priority changed from 1 to 4 (on a scale of 0-7).

My suspicion is that qbt sets the priority to 1, to mean the default priority, but with newer versions of libtorrent the default priority is 4, and the effect is that the pieces are instead set to lower-than-normal priority. i.e. it ends up being downloaded last

arvidn commented Nov 20, 2017

in libtorrent 1.1 (or maybe 1.0) the default file and piece priority changed from 1 to 4 (on a scale of 0-7).

My suspicion is that qbt sets the priority to 1, to mean the default priority, but with newer versions of libtorrent the default priority is 4, and the effect is that the pieces are instead set to lower-than-normal priority. i.e. it ends up being downloaded last

@mzso

This comment has been minimized.

Show comment
Hide comment
@mzso

mzso Nov 20, 2017

@Chocobo1 commented on 2017. nov. 20. 12:54 CET:

I would guess the seeders/peers did this on purpose...?

I highly doubt it. I only experienced this with QB and not µT which I used for several years.
Plus it seems to be the exact inverse of how it works when I select first/last piece. AFAIK no other clients download (or in this case don't download) this many pieces in the beginning/end of the file, µt certainly doesn't.

Plus I think it's unrealistic that all other clients started behaving in this weird way.

mzso commented Nov 20, 2017

@Chocobo1 commented on 2017. nov. 20. 12:54 CET:

I would guess the seeders/peers did this on purpose...?

I highly doubt it. I only experienced this with QB and not µT which I used for several years.
Plus it seems to be the exact inverse of how it works when I select first/last piece. AFAIK no other clients download (or in this case don't download) this many pieces in the beginning/end of the file, µt certainly doesn't.

Plus I think it's unrealistic that all other clients started behaving in this weird way.

@Chocobo1 Chocobo1 added this to the 4.0.1 milestone Nov 20, 2017

@mzso mzso changed the title from [bug] First/parts are downloaded after everything else if prioritizing them is not selected. to [bug] First/last parts are downloaded after everything else if prioritizing them is not selected. Nov 20, 2017

@Chocobo1 Chocobo1 self-assigned this Nov 21, 2017

@sledgehammer999 sledgehammer999 modified the milestones: 4.0.1, 4.0.2 Nov 22, 2017

@Chocobo1

This comment has been minimized.

Show comment
Hide comment
@Chocobo1

Chocobo1 Nov 22, 2017

Member

@mzso
Just to be sure: you added a new torrent in qbt v4.0.0 and didn't bother to toggle "download first last piece first", then you observed this issue, is that correct?
I'm trying to narrow the search in code, so far I didn't see anything wrong here.

Member

Chocobo1 commented Nov 22, 2017

@mzso
Just to be sure: you added a new torrent in qbt v4.0.0 and didn't bother to toggle "download first last piece first", then you observed this issue, is that correct?
I'm trying to narrow the search in code, so far I didn't see anything wrong here.

@arvidn

This comment has been minimized.

Show comment
Hide comment
@arvidn

arvidn Nov 22, 2017

I would have expected to find a bug in here. But it seems qbt is cleverer than I thought. it uses the file priority as the "default", when not setting the "download first/last piece" option.

arvidn commented Nov 22, 2017

I would have expected to find a bug in here. But it seems qbt is cleverer than I thought. it uses the file priority as the "default", when not setting the "download first/last piece" option.

@mzso

This comment has been minimized.

Show comment
Hide comment
@mzso

mzso Nov 22, 2017

@Chocobo1 commented on 2017. nov. 22. 16:36 CET:

@mzso

Just to be sure: you added a new torrent in qbt v4.0.0 and didn't bother to toggle "download first last piece first", then you observed this issue, is that correct?

I'm trying to narrow the search in code, so far I didn't see anything wrong here.

Well, I filed the bug for b2, so that was that, and made the animated image in 4.0.0. But yes, the setting was off.

Right now, I can't reproduce it with 4.0.1. The pieces are downloaded linearly. I'm not sure if something changed on QBT's side.
Could other options change how it works? Because I changed stuff in the meanwhile. (For example changed to TCP only from TCP+uTP, and to automatic mode from manual)

mzso commented Nov 22, 2017

@Chocobo1 commented on 2017. nov. 22. 16:36 CET:

@mzso

Just to be sure: you added a new torrent in qbt v4.0.0 and didn't bother to toggle "download first last piece first", then you observed this issue, is that correct?

I'm trying to narrow the search in code, so far I didn't see anything wrong here.

Well, I filed the bug for b2, so that was that, and made the animated image in 4.0.0. But yes, the setting was off.

Right now, I can't reproduce it with 4.0.1. The pieces are downloaded linearly. I'm not sure if something changed on QBT's side.
Could other options change how it works? Because I changed stuff in the meanwhile. (For example changed to TCP only from TCP+uTP, and to automatic mode from manual)

@mzso

This comment has been minimized.

Show comment
Hide comment
@mzso

mzso Nov 22, 2017

So it looks like I can't reproduce with b2 and 4.0.0 either. Not sure what's changed. I didn't think configuration would be relevant so I didn't make a backup.

mzso commented Nov 22, 2017

So it looks like I can't reproduce with b2 and 4.0.0 either. Not sure what's changed. I didn't think configuration would be relevant so I didn't make a backup.

@Chocobo1

This comment has been minimized.

Show comment
Hide comment
@Chocobo1

Chocobo1 Nov 23, 2017

Member

Right now, I can't reproduce it with 4.0.1. The pieces are downloaded linearly. I'm not sure if something changed on QBT's side.
Could other options change how it works? Because I changed stuff in the meanwhile. (For example changed to TCP only from TCP+uTP, and to automatic mode from manual)

I think its irrelevant.
Closing issue for now.

Member

Chocobo1 commented Nov 23, 2017

Right now, I can't reproduce it with 4.0.1. The pieces are downloaded linearly. I'm not sure if something changed on QBT's side.
Could other options change how it works? Because I changed stuff in the meanwhile. (For example changed to TCP only from TCP+uTP, and to automatic mode from manual)

I think its irrelevant.
Closing issue for now.

@Chocobo1 Chocobo1 closed this Nov 23, 2017

@mzso

This comment has been minimized.

Show comment
Hide comment
@mzso

mzso Nov 23, 2017

@Chocobo1 commented on 2017. nov. 23. 13:23 CET:

Right now, I can't reproduce it with 4.0.1. The pieces are downloaded linearly. I'm not sure if something changed on QBT's side.

Could other options change how it works? Because I changed stuff in the meanwhile. (For example changed to TCP only from TCP+uTP, and to automatic mode from manual)

I think its irrelevant.

Closing issue for now.

Alright. But if nothing was changed about this it's likely to pop up in some context or another.

mzso commented Nov 23, 2017

@Chocobo1 commented on 2017. nov. 23. 13:23 CET:

Right now, I can't reproduce it with 4.0.1. The pieces are downloaded linearly. I'm not sure if something changed on QBT's side.

Could other options change how it works? Because I changed stuff in the meanwhile. (For example changed to TCP only from TCP+uTP, and to automatic mode from manual)

I think its irrelevant.

Closing issue for now.

Alright. But if nothing was changed about this it's likely to pop up in some context or another.

@mzso

This comment has been minimized.

Show comment
Hide comment
@mzso

mzso Nov 23, 2017

And just now I observed it again...
kep

kep

mzso commented Nov 23, 2017

And just now I observed it again...
kep

kep

@mzso

This comment has been minimized.

Show comment
Hide comment
@mzso

mzso Nov 23, 2017

Also just after this I saw the torrent being downloaded from the end (except the very end...) instead of from the front, which was already strange. And some other weird pattern within this, like the ~70-80 filling first

Shouldn't the downloads be random without checking downloading the first/last part? I see everything but...

Note the inverse download, and the last part missing :
kep
kep

kep
kep

The clients are pretty varied:
kep

Something's definitely up...

mzso commented Nov 23, 2017

Also just after this I saw the torrent being downloaded from the end (except the very end...) instead of from the front, which was already strange. And some other weird pattern within this, like the ~70-80 filling first

Shouldn't the downloads be random without checking downloading the first/last part? I see everything but...

Note the inverse download, and the last part missing :
kep
kep

kep
kep

The clients are pretty varied:
kep

Something's definitely up...

@Chocobo1

This comment has been minimized.

Show comment
Hide comment
@Chocobo1

Chocobo1 Nov 23, 2017

Member

Shouldn't the downloads be random without checking downloading the first/last part?

No, it depends on many factors. For example, when availability is high, libtorrent will download it sequentially, otherwise, rarest piece could be picked first. Peers can also give hints to others to influence the choice... and there might be other heuristic algorithms that I'm not aware of.

Member

Chocobo1 commented Nov 23, 2017

Shouldn't the downloads be random without checking downloading the first/last part?

No, it depends on many factors. For example, when availability is high, libtorrent will download it sequentially, otherwise, rarest piece could be picked first. Peers can also give hints to others to influence the choice... and there might be other heuristic algorithms that I'm not aware of.

@arvidn

This comment has been minimized.

Show comment
Hide comment
@arvidn

arvidn Nov 23, 2017

one explanation of downloading from the end would be that the swarm as a whole is skewed towards early pieces (say, if a significant number of peers are streaming, or downloading sequentially). then rarest-first will tend to pick from the end towards the beginning

arvidn commented Nov 23, 2017

one explanation of downloading from the end would be that the swarm as a whole is skewed towards early pieces (say, if a significant number of peers are streaming, or downloading sequentially). then rarest-first will tend to pick from the end towards the beginning

@mzso

This comment has been minimized.

Show comment
Hide comment
@mzso

mzso Nov 23, 2017

@Chocobo1 commented on 2017. nov. 23. 16:52 CET:

No, it depends on many factors. For example, when availability is high, libtorrent will download it sequentially, otherwise, rarest piece could be picked first. Peers can also give hints to others to influence the choice... and there might be other heuristic algorithms that I'm not aware of.

@arvidn commented on 2017. nov. 23. 17:02 CET:

one explanation of downloading from the end would be that the swarm as a whole is skewed towards early pieces (say, if a significant number of peers are streaming, or downloading sequentially). then rarest-first will tend to pick from the end towards the beginning

So then, it might be that the first/last pieces are the most common (µT for one allows to set prioritizing them globally) and because of this QB leaves them last?

In this case I think QB really needs a global setting to prioritize first/last parts. Otherwise leaving them last might become a trend. (Or is a trend already) Which of course has no value to the user, one might say it's a drawback.

mzso commented Nov 23, 2017

@Chocobo1 commented on 2017. nov. 23. 16:52 CET:

No, it depends on many factors. For example, when availability is high, libtorrent will download it sequentially, otherwise, rarest piece could be picked first. Peers can also give hints to others to influence the choice... and there might be other heuristic algorithms that I'm not aware of.

@arvidn commented on 2017. nov. 23. 17:02 CET:

one explanation of downloading from the end would be that the swarm as a whole is skewed towards early pieces (say, if a significant number of peers are streaming, or downloading sequentially). then rarest-first will tend to pick from the end towards the beginning

So then, it might be that the first/last pieces are the most common (µT for one allows to set prioritizing them globally) and because of this QB leaves them last?

In this case I think QB really needs a global setting to prioritize first/last parts. Otherwise leaving them last might become a trend. (Or is a trend already) Which of course has no value to the user, one might say it's a drawback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment