fails to pin to the working tracker within a tier

[sledgehammer999]

Some users are reporting that tracker announcements with correct upload/download stats aren't happening correctly.
Libtorrent version used is 1.1.5+gitd52763805c

Relevant qbt bug: qbittorrent/qBittorrent#8108
The first post has some pictures showing the discrepancy between client and tracker stats. It is quite big.
Others report that correct stats are reported after torrent completion and during seeding, but the uploaded stats that happened during download aren't reported and are "lost".

[arvidn]

there are simulation tests that assert that announces happen at the right times. I will look into those to make sure they are correct and would cover this case, but I feel pretty confident they do.

It would be really helpful if someone experiencing this problem could post a log exposing this issue
(that means announce logs with the relevant query parameters and torrent state transitions)

[arvidn]

there are simulation tests that assert that announces happen at the right times. I will look into those to make sure they are correct and would cover this case, but I feel pretty confident they do.

It would be really helpful if someone experiencing this problem could post a log exposing this issue
(that means announce logs with the relevant query parameters and torrent state transitions)

[ignitionnight]

@arvidn I emailed you at your libtorrent address with the an attached log, the email subject is "issue #2724 log"

I downloaded a torrent from private tracker broadcasthe.net and seeded for about two and a half hours. qbittorrent reports 39.5 MiB uploaded, but the tracker only reports 11.1 MB. This experience is consistent with the issues that I originally reported in the qbittorrent issue tracker qbittorrent/qBittorrent#8108.

I downloaded a torrent almost immediately after it was loaded to the site. There were already 30ish seeds before I started leeching, but I was seeding in about a minute. After the first update on the tracker it showed only 3 MB of upload despite the qbittorrent reporting ~30 MiB uploaded.

I tested this against utorrent 2.2.1 and under similar circumstances the upload credit was 100% correct, any additional upload I contributed after an update on the tracker was reflected on the site within 30 minutes following the next update.

Thank you, and let me know if you need anything else.

[ignitionnight]

@arvidn I emailed you at your libtorrent address with the an attached log, the email subject is "issue #2724 log"

I downloaded a torrent from private tracker broadcasthe.net and seeded for about two and a half hours. qbittorrent reports 39.5 MiB uploaded, but the tracker only reports 11.1 MB. This experience is consistent with the issues that I originally reported in the qbittorrent issue tracker qbittorrent/qBittorrent#8108.

I downloaded a torrent almost immediately after it was loaded to the site. There were already 30ish seeds before I started leeching, but I was seeding in about a minute. After the first update on the tracker it showed only 3 MB of upload despite the qbittorrent reporting ~30 MiB uploaded.

I tested this against utorrent 2.2.1 and under similar circumstances the upload credit was 100% correct, any additional upload I contributed after an update on the tracker was reflected on the site within 30 minutes following the next update.

Thank you, and let me know if you need anything else.

[arvidn]

just to rule things out; are you confident that the number qBittorrent told you was correct?
There are more than one way to report uploads. Perhaps qBittorrent shows you total uploaded bytes, whereas what's reported to the tracker is only payload upload. i.e. all the protocol overhead for downloading does not count from the tracker's point of view.

[arvidn]

just to rule things out; are you confident that the number qBittorrent told you was correct?
There are more than one way to report uploads. Perhaps qBittorrent shows you total uploaded bytes, whereas what's reported to the tracker is only payload upload. i.e. all the protocol overhead for downloading does not count from the tracker's point of view.

[greenbench]

Is it possible that the difference between total uploaded bytes and only payload upload can be hundreds of megabytes - almost gigabyte? Because that's the difference i'm getting with popular torrents. Then again, before qB 4.0.x that has never happened before. All upload was 100% matched.

[greenbench]

Is it possible that the difference between total uploaded bytes and only payload upload can be hundreds of megabytes - almost gigabyte? Because that's the difference i'm getting with popular torrents. Then again, before qB 4.0.x that has never happened before. All upload was 100% matched.

[arvidn]

anything's possible :) I would guess that protocol overhead in the upload direction is about 2-3% of the downloaded payload. Assuming qBT hasn't made any recent changes in what it reports though, it does seem plausible that it's the reporting to tracker where the problem lies.

[arvidn]

anything's possible :) I would guess that protocol overhead in the upload direction is about 2-3% of the downloaded payload. Assuming qBT hasn't made any recent changes in what it reports though, it does seem plausible that it's the reporting to tracker where the problem lies.

[greenbench]

Weird. I mean, i see peers are downloading from me in real time, but then it's announcing to tracker and no upload reporting.

[greenbench]

Weird. I mean, i see peers are downloading from me in real time, but then it's announcing to tracker and no upload reporting.

[ignitionnight]

I'd agree with Greenbench. Although I don't know enough to disagree with Arvid, it really doesn't feel like that would be the case when some torrents work successfully and update correctly and others don't.

I notice this issue with upload stats mostly with highly seeded very new torrents, which happens to me often because I have RSS autodownload set up. Is the issue per chance that upload between initial download and the first announce after beginning seeding isn't being reported?

[ignitionnight]

I'd agree with Greenbench. Although I don't know enough to disagree with Arvid, it really doesn't feel like that would be the case when some torrents work successfully and update correctly and others don't.

I notice this issue with upload stats mostly with highly seeded very new torrents, which happens to me often because I have RSS autodownload set up. Is the issue per chance that upload between initial download and the first announce after beginning seeding isn't being reported?

[greenbench]

Here is how it goes for me:

1. torrent added. first announce to tracker. torrent started downloading and uploading.
2. torrent finished. second announce to tracker. download amount reported correctly but no upload reported. peers are downloading from me and uploading continues.
3. third announce to tracker. no upload reported. peers are downloading from me and uploading continues.
4. fourth announce to tracker. upload amount that was between third announce and that one reported correctly. and from that it's all correct, beside that was "lost" between first and third announces.

and my "fix" for that was:

1. torrent added. first announce to tracker. torrent started downloading and uploading.
2. torrent finished. second announce to tracker. i paused the torrent. download and upload amount reported correctly. (EDIT: it's no more than 3 seconds, but all upload between the time of the second announce and the time i paused the torrent is not reported)
3. started the torrent. third announce to tracker. upload amount reported correctly from now on.

[greenbench]

Here is how it goes for me:

1. torrent added. first announce to tracker. torrent started downloading and uploading.
2. torrent finished. second announce to tracker. download amount reported correctly but no upload reported. peers are downloading from me and uploading continues.
3. third announce to tracker. no upload reported. peers are downloading from me and uploading continues.
4. fourth announce to tracker. upload amount that was between third announce and that one reported correctly. and from that it's all correct, beside that was "lost" between first and third announces.

and my "fix" for that was:

1. torrent added. first announce to tracker. torrent started downloading and uploading.
2. torrent finished. second announce to tracker. i paused the torrent. download and upload amount reported correctly. (EDIT: it's no more than 3 seconds, but all upload between the time of the second announce and the time i paused the torrent is not reported)
3. started the torrent. third announce to tracker. upload amount reported correctly from now on.

[greenbench]

second announce to tracker. i paused the torrent. download and upload amount reported correctly.
started the torrent. third announce to tracker

quoting myself here. i honestly can't tell if there is another announce between those two.

[greenbench]

second announce to tracker. i paused the torrent. download and upload amount reported correctly.
started the torrent. third announce to tracker

quoting myself here. i honestly can't tell if there is another announce between those two.

[arvidn]

@ignitionnight I had a look at your log. I can't find anything obviously wrong with it (but there is one peculiarity).

It shows starting the download with an announce of &left= of about 436 MB and &downloaded= 0.

Once you complete the download (27 seconds later), there's an &event=completed announce with &uploaded= 3828300 (i.e. about 3.8 MB) and &left=0.

The next regular announce (about 2 hours later) has &uploaded= 41454753 (i.e. about 41MB).

Am I correct in assuming that what qBT is reporting is a lot higher than that?
You mention how the upload during the download is lost. Since you completed the download in 27 seconds, 3.1MB of upload doesn't seem outrageously small, was it more in reality?

41 MB in 2 hours is an average upload rate of ~5.7kB/s, which isn't very much. But also not obviously wrong, in case there's a lot more supply than demand.

The one odd thing I found was the download reporting. The complete announce reported a donwnload counter ~70kB greater than the subsequent announce. It sounds a bit far fetched that this would be related though, but let me know if you can think of something.

[arvidn]

@ignitionnight I had a look at your log. I can't find anything obviously wrong with it (but there is one peculiarity).

It shows starting the download with an announce of &left= of about 436 MB and &downloaded= 0.

Once you complete the download (27 seconds later), there's an &event=completed announce with &uploaded= 3828300 (i.e. about 3.8 MB) and &left=0.

The next regular announce (about 2 hours later) has &uploaded= 41454753 (i.e. about 41MB).

Am I correct in assuming that what qBT is reporting is a lot higher than that?
You mention how the upload during the download is lost. Since you completed the download in 27 seconds, 3.1MB of upload doesn't seem outrageously small, was it more in reality?

41 MB in 2 hours is an average upload rate of ~5.7kB/s, which isn't very much. But also not obviously wrong, in case there's a lot more supply than demand.

The one odd thing I found was the download reporting. The complete announce reported a donwnload counter ~70kB greater than the subsequent announce. It sounds a bit far fetched that this would be related though, but let me know if you can think of something.

[sledgehammer999]

In order to provide a bit of info on what qbt presents to the user:
There are 2 places where such info is displayed.

1. A per torrent "General" tab info at the bottom of the tranferlist.
   There the info is printed as "total size (session size)". For total size we use torrent_status::all_time_download and for session size we use torrent_status::total_payload_download
   (same thing for upload)

2. In the transferlist as separate columns for total/session which use the same values as above.

[sledgehammer999]

In order to provide a bit of info on what qbt presents to the user:
There are 2 places where such info is displayed.

1. A per torrent "General" tab info at the bottom of the tranferlist.
   There the info is printed as "total size (session size)". For total size we use torrent_status::all_time_download and for session size we use torrent_status::total_payload_download
   (same thing for upload)

2. In the transferlist as separate columns for total/session which use the same values as above.

[greenbench]

@sledgehammer999 Is this app qbittorrent/qBittorrent#8108 (comment) launching in "portable mod"? I mean, my qBittorrent.ini and qBittorrent-data.ini in AppData\Roaming will not be overwritten? And can it run simultaneously with my installed qB?

[greenbench]

@sledgehammer999 Is this app qbittorrent/qBittorrent#8108 (comment) launching in "portable mod"? I mean, my qBittorrent.ini and qBittorrent-data.ini in AppData\Roaming will not be overwritten? And can it run simultaneously with my installed qB?

[sledgehammer999]

No, it is like a regular release. It loads/saves the same files. It just logs extra things, using a libtorrent build that has extra logging enabled.
It won't run simultaneously with any other qbt process either.

[sledgehammer999]

No, it is like a regular release. It loads/saves the same files. It just logs extra things, using a libtorrent build that has extra logging enabled.
It won't run simultaneously with any other qbt process either.

[ignitionnight]

@arvidn here's some responses to your questions/feedback above.

Once you complete the download (27 seconds later), there's an &event=completed announce with &uploaded= 3828300 (i.e. about 3.8 MB) and &left=0.

The 3.8 MB is about what the tracker showed for the first hour or so, however qbit showed around 34 MB of upload pretty quickly after seeding. This is where I think the problem is, I think at this point I'm missing ~30 MB of upload that gets "lost" between the client and the tracker.

The next regular announce (about 2 hours later) has &uploaded= 41454753 (i.e. about 41MB).

about 41 MB is what qbit showed as uploaded actually and I never really got much more, however by this time the site has updated to about 11 MB. So I've gained about 7/8 MB in qbit since the last announce and gained 7/8 MB on the site since the last announce.

The log I provided was only the first 2.5 hours after downloading, as the discrepancy was already showing up (I actually got 0 upload on qbit or the site after that log was provided and the discrepancy is usually at the biggest gap early on, it tends to stay steady after a short initial period. So if I've lost 250 MB of seeding in the first 30 minutes, that usually ends up being almost the entire discrepancy when I stop seeding a day later.

I can try another larger, more popular torrent if that will give more data. I'll reset the log and find a good one tonight and let it seed for a few hours, hopefully get much more upload activity to look through.

If I'm being honest, the log seems to show the right numbers which would imply that the client is not at fault. However every other client I've used (utorrent 3.x & 2.2.1, and Deluge 1.3.15) always reports almost 100% accurate upload between the client and tracker, qbit is the only client I've used that has this problem. Can it perhaps be the timing of announces, or the frequency?

[ignitionnight]

@arvidn here's some responses to your questions/feedback above.

Once you complete the download (27 seconds later), there's an &event=completed announce with &uploaded= 3828300 (i.e. about 3.8 MB) and &left=0.

The 3.8 MB is about what the tracker showed for the first hour or so, however qbit showed around 34 MB of upload pretty quickly after seeding. This is where I think the problem is, I think at this point I'm missing ~30 MB of upload that gets "lost" between the client and the tracker.

The next regular announce (about 2 hours later) has &uploaded= 41454753 (i.e. about 41MB).

about 41 MB is what qbit showed as uploaded actually and I never really got much more, however by this time the site has updated to about 11 MB. So I've gained about 7/8 MB in qbit since the last announce and gained 7/8 MB on the site since the last announce.

The log I provided was only the first 2.5 hours after downloading, as the discrepancy was already showing up (I actually got 0 upload on qbit or the site after that log was provided and the discrepancy is usually at the biggest gap early on, it tends to stay steady after a short initial period. So if I've lost 250 MB of seeding in the first 30 minutes, that usually ends up being almost the entire discrepancy when I stop seeding a day later.

I can try another larger, more popular torrent if that will give more data. I'll reset the log and find a good one tonight and let it seed for a few hours, hopefully get much more upload activity to look through.

If I'm being honest, the log seems to show the right numbers which would imply that the client is not at fault. However every other client I've used (utorrent 3.x & 2.2.1, and Deluge 1.3.15) always reports almost 100% accurate upload between the client and tracker, qbit is the only client I've used that has this problem. Can it perhaps be the timing of announces, or the frequency?

[sledgehammer999]

So if I've lost 250 MB of seeding in the first 30 minutes

Can your available bandwidth achieve at least 250MB upload in 30 minutes? I am just trying to rule out faulty upload reporting in the client side...

[sledgehammer999]

So if I've lost 250 MB of seeding in the first 30 minutes

Can your available bandwidth achieve at least 250MB upload in 30 minutes? I am just trying to rule out faulty upload reporting in the client side...

[sledgehammer999]

Deluge 1.3.15

Do you know which version of libtorrent does that deluge version use?

[sledgehammer999]

Deluge 1.3.15

Do you know which version of libtorrent does that deluge version use?

[joshuahiggins]

I'm following this thread because I'm experiencing the same issue with Deluge (on FreeBSD) and select private sites, so it's nice to follow an open source ticket for one client tackling the issue. That said, I am surprised to read that @ignitionnight isn't experiencing this on Deluge.

Can your available bandwidth achieve at least 250MB upload in 30 minutes? I am just trying to rule out faulty upload reporting in the client side...

Anecdotally, I've found that many fiber internet users experience the issues reported here. It seems like an issue that plagues the first 2 announces. I personally didn't have this issue when I was on 200/20 internet, but now that I am on 1GB speeds I have been plagued by this problem on 2 sites while having no issue on other sites. 1 site even frequently throws errors on the first announces, while the other reports no issues. I'm lead to believe it's a site issue, but I consistently am pointed back to the client when I try to dig more into how the site handles announces.

With a fiber connection, the vast majority of uploads for popular torrents are within that first 30 minutes. 250MB can be achieved in under a minute.

Do you know which version of libtorrent does that deluge version use?

I'm curious about @ignitionnight's answer on this one too. On FreeBSD, I am running Deluge 1.3.15, but I manually upgraded libtorrent to 1.1.6.0 in hopes of resolving my issues. I believe Deluge officially supports libtorrent 1.0.x right now.

[joshuahiggins]

I'm following this thread because I'm experiencing the same issue with Deluge (on FreeBSD) and select private sites, so it's nice to follow an open source ticket for one client tackling the issue. That said, I am surprised to read that @ignitionnight isn't experiencing this on Deluge.

Can your available bandwidth achieve at least 250MB upload in 30 minutes? I am just trying to rule out faulty upload reporting in the client side...

Anecdotally, I've found that many fiber internet users experience the issues reported here. It seems like an issue that plagues the first 2 announces. I personally didn't have this issue when I was on 200/20 internet, but now that I am on 1GB speeds I have been plagued by this problem on 2 sites while having no issue on other sites. 1 site even frequently throws errors on the first announces, while the other reports no issues. I'm lead to believe it's a site issue, but I consistently am pointed back to the client when I try to dig more into how the site handles announces.

With a fiber connection, the vast majority of uploads for popular torrents are within that first 30 minutes. 250MB can be achieved in under a minute.

Do you know which version of libtorrent does that deluge version use?

I'm curious about @ignitionnight's answer on this one too. On FreeBSD, I am running Deluge 1.3.15, but I manually upgraded libtorrent to 1.1.6.0 in hopes of resolving my issues. I believe Deluge officially supports libtorrent 1.0.x right now.

[arvidn]

@ignitionnight it sounds like the numbers I see in the announces are correct, or at least appear to correct. That would suggest the problem may be related to the request itself. Perhaps it's closed prematurely, formatted in a new way or something else may cause the tracker to fail to parse out these numbers and apply them correctly. Does that sound like a reasonable assumption?

[arvidn]

@ignitionnight it sounds like the numbers I see in the announces are correct, or at least appear to correct. That would suggest the problem may be related to the request itself. Perhaps it's closed prematurely, formatted in a new way or something else may cause the tracker to fail to parse out these numbers and apply them correctly. Does that sound like a reasonable assumption?

[ignitionnight]

Can your available bandwidth achieve at least 250MB upload in 30 minutes? I am just trying to rule out faulty upload reporting in the client side...

Yes, speedtests give me about 250 Mb down 15 Mb up, depending on the number of leachers I get I can get to that in about 3-4 minutes.

Do you know which version of libtorrent does that deluge version use?

libtorrent: 1.0.11.0 is what is listed on the about popup. I should probably back off deluge being flawless, especially considering @joshuahiggins reports of it having the same problem. I switched to deluge first and didn't notice the problem, but only used it a short time on the private sites because the rss plugins didn't work the way I wanted them to.

@arvidn I think you've now gone above my head, but I'll try to keep up lol. Just go easy on me if it become obvious I don't know what I'm talking about. I think what you said could be a good assumption. The request closing prematurely sounds possible, especially if I consider the possibility of that happening in the first few requests. Perhaps the tracker drops one announced upload update because it came too soon following a previous announce?

@joshuahiggins I think your anecdotal experience aligns with mine. Although I don't have fiber I do have relatively quick cable, and this error manifests more distinctly when I'm very early to the torrent.... which thanks to my rss setup is pretty often if not always. My log was with a torrent I downloaded within 1 minute of it being upped to the site. That gave me a big headstart and the discrepancy was there immediately.

[ignitionnight]

Can your available bandwidth achieve at least 250MB upload in 30 minutes? I am just trying to rule out faulty upload reporting in the client side...

Yes, speedtests give me about 250 Mb down 15 Mb up, depending on the number of leachers I get I can get to that in about 3-4 minutes.

Do you know which version of libtorrent does that deluge version use?

libtorrent: 1.0.11.0 is what is listed on the about popup. I should probably back off deluge being flawless, especially considering @joshuahiggins reports of it having the same problem. I switched to deluge first and didn't notice the problem, but only used it a short time on the private sites because the rss plugins didn't work the way I wanted them to.

@arvidn I think you've now gone above my head, but I'll try to keep up lol. Just go easy on me if it become obvious I don't know what I'm talking about. I think what you said could be a good assumption. The request closing prematurely sounds possible, especially if I consider the possibility of that happening in the first few requests. Perhaps the tracker drops one announced upload update because it came too soon following a previous announce?

@joshuahiggins I think your anecdotal experience aligns with mine. Although I don't have fiber I do have relatively quick cable, and this error manifests more distinctly when I'm very early to the torrent.... which thanks to my rss setup is pretty often if not always. My log was with a torrent I downloaded within 1 minute of it being upped to the site. That gave me a big headstart and the discrepancy was there immediately.

[arvidn]

It appears the underlying problem is in the handling of trackers within a tier. libtorrent attempts to stick to the first working tracker within the tier, but it turns out in certain situations (it seems to be triggered by the event=completed announce) the first tracker is either considered failed or for some other reason the next announce is made to the next tracker in the list.

Some private trackers don't expect this to happen, and make it a hard failure, and ignore stats reported in those announces.

[arvidn]

It appears the underlying problem is in the handling of trackers within a tier. libtorrent attempts to stick to the first working tracker within the tier, but it turns out in certain situations (it seems to be triggered by the event=completed announce) the first tracker is either considered failed or for some other reason the next announce is made to the next tracker in the list.

Some private trackers don't expect this to happen, and make it a hard failure, and ignore stats reported in those announces.

[greenbench]

What if there is only one tracker?

[greenbench]

What if there is only one tracker?

[arvidn]

My understanding is that with a single tracker there is no problem

[arvidn]

My understanding is that with a single tracker there is no problem

[greenbench]

Hm, as far as I can tell @ignitionnight and definitely I have a problem with torrents with only one tracker.

[greenbench]

Hm, as far as I can tell @ignitionnight and definitely I have a problem with torrents with only one tracker.

[arvidn]

Please try this patch! #2837

[arvidn]

Please try this patch! #2837

[arvidn]

@sledgehammer999 would you mind making a test build of qbt?

[arvidn]

@sledgehammer999 would you mind making a test build of qbt?

[ignitionnight]

I've been quiet lately because a lot of this is beyond my knowledge, but if we could get a test build of qbt I'm happy to test it and provide logs again! Thanks

[ignitionnight]

I've been quiet lately because a lot of this is beyond my knowledge, but if we could get a test build of qbt I'm happy to test it and provide logs again! Thanks

[arvidn]

@sledgehammer999 ?

[arvidn]

@sledgehammer999 ?

[arvidn]

merged

[arvidn]

merged

[ignitionnight]

@arvidn Thank you so much for all your work on this issue. I'm assuming I just need to wait for a new build of libtorrent 1.1.5? and for that build to be incorporated into a new qbit build by @sledgehammer999 ?

[ignitionnight]

@arvidn Thank you so much for all your work on this issue. I'm assuming I just need to wait for a new build of libtorrent 1.1.5? and for that build to be incorporated into a new qbit build by @sledgehammer999 ?

[arvidn]

This will go into libtorrent 1.1.7, which I was hoping to release this weekend-but merging and testing the patches I would like in has taken a bit longer than I was hoping. So, soon.

[arvidn]

This will go into libtorrent 1.1.7, which I was hoping to release this weekend-but merging and testing the patches I would like in has taken a bit longer than I was hoping. So, soon.

[greenbench]

Thanks @arvidn! So I'm guessing incorrect upload is somewhat by-product of this bug you fixed?

[greenbench]

Thanks @arvidn! So I'm guessing incorrect upload is somewhat by-product of this bug you fixed?

[sledgehammer999]

For anyone wanting to test this, I have compiled qbittorrent 4.0.4 based on Qt 5.10.1 and libtorrent aaf9304
Link: http://builds.shiki.hu/temp/qbittorrent_4.0.4_x64_libtorrent_1.1.6+gitaaf9304a3b5.7z
PS: With Qt 5.10.1 there's a bug where the columns' size in the Content tab won't be remembered. It is fixed in master.

[sledgehammer999]

For anyone wanting to test this, I have compiled qbittorrent 4.0.4 based on Qt 5.10.1 and libtorrent aaf9304
Link: http://builds.shiki.hu/temp/qbittorrent_4.0.4_x64_libtorrent_1.1.6+gitaaf9304a3b5.7z
PS: With Qt 5.10.1 there's a bug where the columns' size in the Content tab won't be remembered. It is fixed in master.

[Thefrank]

not to necro a closed issue but... those that want to build qbittorrent on BSD with this lib, you will (still) need:
CXXFLAGS='-DBOOST_ASIO_HAS_STD_CHRONO' as a part of your configure for libttorrent otherwise you wont get a linkable qbittorrent. Also, you still cant use -std=c++11 for building libtorrent under BSD which a few non-platform specific guilds suggest adding.

[Thefrank]

not to necro a closed issue but... those that want to build qbittorrent on BSD with this lib, you will (still) need:
CXXFLAGS='-DBOOST_ASIO_HAS_STD_CHRONO' as a part of your configure for libttorrent otherwise you wont get a linkable qbittorrent. Also, you still cant use -std=c++11 for building libtorrent under BSD which a few non-platform specific guilds suggest adding.

[arvidn]

what happens if you try to build with C++11 enabled?

[arvidn]

what happens if you try to build with C++11 enabled?

[Thefrank]

edit->tl;dr: issue appears to more likely be clang+boost related than libtorrent related but does impact the ability the build libtorrent without adding extra options

So with "./configure CXXFLAGS='-D BOOST_ASIO_HAS_STD_CHRONO -std=c++11'"
it fails here

--- kademlia/dht_tracker.lo ---
In file included from kademlia/dht_tracker.cpp:38:
In file included from ../include/libtorrent/kademlia/node.hpp:41:
In file included from ../include/libtorrent/kademlia/dht_storage.hpp:38:
In file included from /usr/local/include/boost/function.hpp:70:
In file included from /usr/local/include/boost/preprocessor/iteration/detail/iter/forward1.hpp:52:
In file included from /usr/local/include/boost/function/detail/function_iterate.hpp:14:
In file included from /usr/local/include/boost/function/detail/maybe_include.hpp:22:
/usr/local/include/boost/function/function_template.hpp:159:33: error: called object type 'nullptr_t' is not a function or function pointer
          BOOST_FUNCTION_RETURN((*f)(BOOST_FUNCTION_ARGS));
                                ^~~~
/usr/local/include/boost/function/function_template.hpp:81:36: note: expanded from macro 'BOOST_FUNCTION_RETURN'
#  define BOOST_FUNCTION_RETURN(X) X
                                   ^
/usr/local/include/boost/function/function_template.hpp:925:53: note: in instantiation of member function 'boost::detail::function::void_function_obj_invoker1<nullptr_t, void, const std::__1::vector<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> >, std::__1::allocator<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> > > > &>::invoke' requested here
        { { &manager_type::manage }, &invoker_type::invoke };
                                                    ^
/usr/local/include/boost/function/function_template.hpp:716:13: note: in instantiation of function template specialization 'boost::function1<void, const std::__1::vector<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> >, std::__1::allocator<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> > > > &>::assign_to<nullptr_t>' requested here
      this->assign_to(f);
            ^
/usr/local/include/boost/function/function_template.hpp:1061:5: note: in instantiation of function template specialization 'boost::function1<void, const std::__1::vector<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> >, std::__1::allocator<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> > > > &>::function1<nullptr_t>' requested here
    base_type(f)
    ^
kademlia/dht_tracker.cpp:227:26: note: in instantiation of function template specialization 'boost::function<void (const std::__1::vector<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> >, std::__1::allocator<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> > > > &)>::function<nullptr_t>' requested here
                m_dht.get_peers(ih, f, NULL, false);
                                       ^
/usr/include/sys/_null.h:35:14: note: expanded from macro 'NULL'
#define NULL    nullptr
                ^
1 error generated.

a quick googling seems to point to a clang specific issue that I don't exactly understand
the clang+boost chrono thing can be seen here and is a clang+boost specific issue: https://svn.boost.org/trac10/ticket/10884

[Thefrank]

edit->tl;dr: issue appears to more likely be clang+boost related than libtorrent related but does impact the ability the build libtorrent without adding extra options

So with "./configure CXXFLAGS='-D BOOST_ASIO_HAS_STD_CHRONO -std=c++11'"
it fails here

--- kademlia/dht_tracker.lo ---
In file included from kademlia/dht_tracker.cpp:38:
In file included from ../include/libtorrent/kademlia/node.hpp:41:
In file included from ../include/libtorrent/kademlia/dht_storage.hpp:38:
In file included from /usr/local/include/boost/function.hpp:70:
In file included from /usr/local/include/boost/preprocessor/iteration/detail/iter/forward1.hpp:52:
In file included from /usr/local/include/boost/function/detail/function_iterate.hpp:14:
In file included from /usr/local/include/boost/function/detail/maybe_include.hpp:22:
/usr/local/include/boost/function/function_template.hpp:159:33: error: called object type 'nullptr_t' is not a function or function pointer
          BOOST_FUNCTION_RETURN((*f)(BOOST_FUNCTION_ARGS));
                                ^~~~
/usr/local/include/boost/function/function_template.hpp:81:36: note: expanded from macro 'BOOST_FUNCTION_RETURN'
#  define BOOST_FUNCTION_RETURN(X) X
                                   ^
/usr/local/include/boost/function/function_template.hpp:925:53: note: in instantiation of member function 'boost::detail::function::void_function_obj_invoker1<nullptr_t, void, const std::__1::vector<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> >, std::__1::allocator<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> > > > &>::invoke' requested here
        { { &manager_type::manage }, &invoker_type::invoke };
                                                    ^
/usr/local/include/boost/function/function_template.hpp:716:13: note: in instantiation of function template specialization 'boost::function1<void, const std::__1::vector<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> >, std::__1::allocator<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> > > > &>::assign_to<nullptr_t>' requested here
      this->assign_to(f);
            ^
/usr/local/include/boost/function/function_template.hpp:1061:5: note: in instantiation of function template specialization 'boost::function1<void, const std::__1::vector<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> >, std::__1::allocator<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> > > > &>::function1<nullptr_t>' requested here
    base_type(f)
    ^
kademlia/dht_tracker.cpp:227:26: note: in instantiation of function template specialization 'boost::function<void (const std::__1::vector<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> >, std::__1::allocator<std::__1::pair<libtorrent::dht::node_entry, std::__1::basic_string<char> > > > &)>::function<nullptr_t>' requested here
                m_dht.get_peers(ih, f, NULL, false);
                                       ^
/usr/include/sys/_null.h:35:14: note: expanded from macro 'NULL'
#define NULL    nullptr
                ^
1 error generated.

a quick googling seems to point to a clang specific issue that I don't exactly understand
the clang+boost chrono thing can be seen here and is a clang+boost specific issue: https://svn.boost.org/trac10/ticket/10884

[sledgehammer999]

@Thefrank for my PPA:

1. I use this patch which got merged and then reverted: #2321
2. As configure args I use CXXFLAGS=-std=c++11 CPPFLAGS=-std=c++11. No need for the boost DEFINE.

Could you give it a try?

[sledgehammer999]

@Thefrank for my PPA:

1. I use this patch which got merged and then reverted: #2321
2. As configure args I use CXXFLAGS=-std=c++11 CPPFLAGS=-std=c++11. No need for the boost DEFINE.

Could you give it a try?

[Thefrank]

adding that CPPFLAGS configure fails with:

configure:2842: checking whether the C compiler works
configure:2864: cc  -std=c++11  conftest.c  >&5
error: invalid argument '-std=c++11' not allowed with 'C/ObjC'
configure:2868: $? = 1
configure:2906: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "libtorrent-rasterbar"
| #define PACKAGE_TARNAME "libtorrent-rasterbar"
| #define PACKAGE_VERSION "1.1.7"
| #define PACKAGE_STRING "libtorrent-rasterbar 1.1.7"
| #define PACKAGE_BUGREPORT "arvid@libtorrent.org"
| #define PACKAGE_URL "http://www.libtorrent.org"
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }

[Thefrank]

adding that CPPFLAGS configure fails with:

configure:2842: checking whether the C compiler works
configure:2864: cc  -std=c++11  conftest.c  >&5
error: invalid argument '-std=c++11' not allowed with 'C/ObjC'
configure:2868: $? = 1
configure:2906: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "libtorrent-rasterbar"
| #define PACKAGE_TARNAME "libtorrent-rasterbar"
| #define PACKAGE_VERSION "1.1.7"
| #define PACKAGE_STRING "libtorrent-rasterbar 1.1.7"
| #define PACKAGE_BUGREPORT "arvid@libtorrent.org"
| #define PACKAGE_URL "http://www.libtorrent.org"
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }

[Thefrank]

#2724 (comment)
Tried this build (on windows obv) but...
This does not fix my duplicate torrent error from the tracker and lost upload reported data

[Thefrank]

#2724 (comment)
Tried this build (on windows obv) but...
This does not fix my duplicate torrent error from the tracker and lost upload reported data

[x9248]

Same here on IPT (iptorrents). Trackers turning to Not Working and saying dupe torrents still, upload amount is still being lost.

[x9248]

Same here on IPT (iptorrents). Trackers turning to Not Working and saying dupe torrents still, upload amount is still being lost.

[arvidn]

@Thefrank and/or @x9248 would you be willing to help me understand the cause of this error. The logs that I received from people experiencing this (or a similar) problem indicated a problem that I'm pretty sure is solved now (I managed to write a simulation to reproduce it).

Specifically, I would like a verbose log (with as many alerts enabled as possible) and also a description of the symptom (otherwise it's hard to know what to look for in the logs). My understanding is that libtorrent fails to stick to the last working torrent within a tier, and contacts the next torrent in the tier sometimes. The fact that some trackers relies on this to never happen I think is a bit of difference in interpretations of the spirit of the multi-tracker feature.

But perhaps there's some other behaviour (or misbehaviour) that also has this symptom.

[arvidn]

@Thefrank and/or @x9248 would you be willing to help me understand the cause of this error. The logs that I received from people experiencing this (or a similar) problem indicated a problem that I'm pretty sure is solved now (I managed to write a simulation to reproduce it).

Specifically, I would like a verbose log (with as many alerts enabled as possible) and also a description of the symptom (otherwise it's hard to know what to look for in the logs). My understanding is that libtorrent fails to stick to the last working torrent within a tier, and contacts the next torrent in the tier sometimes. The fact that some trackers relies on this to never happen I think is a bit of difference in interpretations of the spirit of the multi-tracker feature.

But perhaps there's some other behaviour (or misbehaviour) that also has this symptom.

[Thefrank]

I didn't have verbose logging on but I did log communication between the client and tracker(s). Would that be helpful? If yes, do you have an email where I can send a .cap file?

[Thefrank]

I didn't have verbose logging on but I did log communication between the client and tracker(s). Would that be helpful? If yes, do you have an email where I can send a .cap file?

[arvidn]

arvid@libtorrent.org

[arvidn]

arvid@libtorrent.org

[ignitionnight]

Not that it's worth much without a full log, but I tried testing this again with sledgehammer's most recent build from 5 days ago.

Two torrents were auto downloaded an hour after appearing on the rss feed. They had no immediate upload action, but added 30-40 MB of upload after a few hours all upload reported correctly for these two torrents.

A third torrent was auto downloaded immediately after showing on the rss feed, within 5 minutes, so it generated a lot of upload. My torrent client reported ~140 uploaded MB, but the site only ever got to 80 MB.

It seems very consistent that any initial upload gathered at the very beginning of a torrent being downloaded is eventually lost, I haven't been able to notice if this is the upload that is gathered during the initial leaching, or if it is a longer period than that. I would have emailed the log, but the log that was produced by the most recent build was not verbose.

[ignitionnight]

Not that it's worth much without a full log, but I tried testing this again with sledgehammer's most recent build from 5 days ago.

Two torrents were auto downloaded an hour after appearing on the rss feed. They had no immediate upload action, but added 30-40 MB of upload after a few hours all upload reported correctly for these two torrents.

A third torrent was auto downloaded immediately after showing on the rss feed, within 5 minutes, so it generated a lot of upload. My torrent client reported ~140 uploaded MB, but the site only ever got to 80 MB.

It seems very consistent that any initial upload gathered at the very beginning of a torrent being downloaded is eventually lost, I haven't been able to notice if this is the upload that is gathered during the initial leaching, or if it is a longer period than that. I would have emailed the log, but the log that was produced by the most recent build was not verbose.

[Thefrank]

under FreeBSD I can't get qBittorrent to build (fails) with libtorrent 1.1.7 with debugging (builds fine). I can keep gathering packet captures with the windows build but I don't have any way of gathering verbose logs currently on any platform

[Thefrank]

under FreeBSD I can't get qBittorrent to build (fails) with libtorrent 1.1.7 with debugging (builds fine). I can keep gathering packet captures with the windows build but I don't have any way of gathering verbose logs currently on any platform

[sledgehammer999]

under FreeBSD I can't get qBittorrent to build (fails) with libtorrent 1.1.7 with debugging (builds fine).

Fails at linking stage? Can you post the errors? (in a pastebin preferably, to not clutter the bug report)

[sledgehammer999]

under FreeBSD I can't get qBittorrent to build (fails) with libtorrent 1.1.7 with debugging (builds fine).

Fails at linking stage? Can you post the errors? (in a pastebin preferably, to not clutter the bug report)

[Thefrank]

@sledgehammer999
libtorrent was built from master (4b368e1) with the same configure used below. I did have to manually add the libtorrent path in PKG_CONFIG_PATH because make install puts it into a non-standard place

result of./configure CXXFLAGS='-D BOOST_ASIO_HAS_STD_CHRONO -std=c++11' --enable-debug
https://pastebin.com/FsrJKdqD

result of make
https://pastebin.com/DQKxNHvy

[Thefrank]

@sledgehammer999
libtorrent was built from master (4b368e1) with the same configure used below. I did have to manually add the libtorrent path in PKG_CONFIG_PATH because make install puts it into a non-standard place

result of./configure CXXFLAGS='-D BOOST_ASIO_HAS_STD_CHRONO -std=c++11' --enable-debug
https://pastebin.com/FsrJKdqD

result of make
https://pastebin.com/DQKxNHvy

[sledgehammer999]

libtorrent was built from master

@Thefrank that's your problem. We don't yet support libtorrent master. You should use the RC_1_1 branch.

[sledgehammer999]

libtorrent was built from master

@Thefrank that's your problem. We don't yet support libtorrent master. You should use the RC_1_1 branch.

[Thefrank]

ok I feel stupid. everything builds fine now. ill start using that to get more detailed logs

[Thefrank]

ok I feel stupid. everything builds fine now. ill start using that to get more detailed logs