P2P transfer stalled while webseeding

[aritra24]

Hi, wanted to thank you for making this project. I had a question about if perhaps the way I'm using the project needs reconfiguration. My current setup is a swarm where none of the nodes have access to the file being transferred. Once of the nodes has the file added as a web seed. My expectation was that as this node downloads pieces it would start transferring them. But that doesn't seem to happen. I see that the other nodes stay at 0 pieces until the initial node gets all the pieces and only then does the transfer start. Is there perhaps some kind of configuration that needs changing or perhaps this is expected behaviour and I should try something else to reduce the time. Thanks for any help. Below is an image of the status page showing one of the nodes where no transfer is happening while it can be seen a peer of it's (The node with the webseed) has pieces available.

.

[aritra24]

I did see a similar question at #881 but there wasn't a solution found there so I opened a new question. I would like to note that this is running on a docker network all running locally currently for testing purposes.

[anacrolix]

Thanks for identifying an existing similar issue. I do think they are the same. The steps I'd check are the same here as they are there. With Seed=true, the peers should upload regardless of their total download state.

[aritra24]

Hi Matt, thanks for your response. I do have seed set to true. These are some of the configs I'm using right now

config.NoDHT = false
	config.DisablePEX = false
	config.DataDir = conf.DataDirectory
	config.Debug = conf.Debug
	config.ListenPort = conf.Port
	config.DisableAggressiveUpload = false
	config.AlwaysWantConns = true
	config.AcceptPeerConnections = true
	config.EstablishedConnsPerTorrent = 20
	config.PieceHashersPerTorrent = 10
	config.Extensions = peer_protocol.NewPeerExtensionBytes(peer_protocol.ExtensionBitDht | peer_protocol.ExtensionBitFast | peer_protocol.ExtensionBitV2)
	config.Seed = true
        config.DisableTrackers = true

If there's anything you think I could check which would help figure out what's happening do let me know and I can try it out. I do want to add I tried disabling the webseed server halfway through the transfer and that causes the transfer to completely stop.

[anacrolix]

I looked again and I can't spot the issue. All I can think of is setting some log messages in PeerConn.uploadAllowed. There's a check to avoid seeding until all wanted parts of the torrent are downloaded before seeding, unless Seed is set to true in the config, which you have. But the behaviour makes it sound like that check is failing for some reason.

Try logging in

if c.t.seeding() {
    return true
}

Maybe log the return result of c.t.seeding() then let it do its thing.

If you look in Torrent.seeding you'll see:

if cl.config.DisableAggressiveUpload && t.needData() {
	return false
}

That's the bit that sounds like it isn't working correctly.

[aritra24]

I tried setting a break point there and uploadAllowed returns true. Though I think what I see is c.PeerRequests is nil here If it helps I tried taking a tcpdump and see that the node only sends an interested request after about 250 seconds.

[anacrolix]

I wrote a test for this, and can't seem to reproduce it. I thought I had for a while but I had MaxAllocPeerRequestDataPerConn set to the value used in TestingConfig which is too low. Please check out https://github.com/anacrolix/torrent/tree/issue-916.

cd tests/webseed-partial-seed
GOPPROF=http go test -race -v

You can also check the endpoints to see the state of the leecher, go to (most likely) http://localhost:6061/TestWebseedPartialSeed/leecher and see if it's started downloading before the seeder has finished.

[aritra24]

Your test does seem to be working for me as well, I'll try debugging some more and see if I can find what is it that I'm doing differently. One thing I noticed while debugging was there's a few of these logs, though I'm not sure if it's relevant
[2024-03-23 12:46:23 +0000 DBG] *torrent.PeerConn 0xc000229500: waiting for alloc limit reservation: reservation cancelled [github.com/anacrolix/torrent peerconn.go:598]
I'll try debugging some more perhaps I can find something more.

[aritra24]

Hmm so I am getting some kind of replication which seems to tell me it's perhaps the torrent file I'm using. Since when I switched the test to using my torrent file I see the issue. I'm not sure if it's caused by the torrent being served by a local server? Or if something is missing in the torrent file itself. Though I'm not sure why that would be since only the seeder is requesting the pieces and where/who is providing it the pieces should probably not hinder it's ability to serve pieces to the leecher? 🤔

[aritra24]

No, I tried it on seperate servers as well, and still see the same thing where after about 5 minutes (Almost exactly 5 minutes which was odd) the other nodes started downloading again. Is there a timeout for when it reattempts to read? I tried on a 3 node system and both only restarted after about 5 minutes.

[aritra24]

@anacrolix could you point me to how the request to send an interested message is sent? I found the point where we receive it at. Seeing the tcpdump it looks like the interested in coming in after a significantly long time from the leecher so I wanted to add some debug logging around there to see what's happening.

[anacrolix]

Could you send me your torrent privately? Are you using a custom storage implementation at all?

[aritra24]

Sure, let's connect over email or any other chat platform you use? I am not using any custom storage. I just have the data directory configured, but use the default storage other than that.

[aritra24]

I was able to find the issue, I believe it's a bug, here while ordering pieces one of the params used to sort is if i_availability < j_availability then i is prioritised higher. But this breaks when i_availability is zero for a large number of pieces. Since they might consume the entirety of MaxUnverifiedBytes for the torrent. Thus freezing up until a large enough number of pieces become available such that atleast one of the pieces fits inside the limit of MaxUnverifiedBytes. I believe either the sorting algo needs to account for a zero availability piece or the ordering should skip past all pieces that have no availability. I've raised a sample PR for it in case it's acceptable. As a workaround we can set the MaxUnverifiedBytes to 0.

[anacrolix]

Thanks. I noticed I could only reproduce the issue when I ran the webseed through python rangehttpserver too (the alternatives are so fast you don't see the issue in time). That implementation is unbelievably slow for some reason.

[anacrolix]

Thanks for finding this. There's a test for it in the tests directory. I've pushed a fix, 528c5f6. Let me know if it doesn't fix the issue. It might be possible to modify some sorting down the road to improve the case where availability is zero, I'm still thinking about what effect that would have.

[aritra24]

It did resolve the issue, thanks for the fix!