How to seed a lot of files?

[kevinsimper]

I have a lot of gallary images, so how would you seed a lot of files the smartest, like 100.000+ gallary images?

I am right now just seeding them with client.seed but it does not really scale.

Preferably I would only start seeding something when a peer connects.

I did search the issues and could not find anyone asking, so I thought this could be useful for others also to know.

[feross]

The way that torrents work requires a high number of messages just to remain "findable" by peers. There are messages to the DHT (which take ~3 seconds per torrent) and to the tracker (much faster). These need to be resent every 15 minutes.

So, 300 torrents * 3 seconds = 15 minutes. So, adding around 300 torrents will mean that there's a constant low-level amount of UDP traffic to the DHT. Going too many factors above this number will probably overwhelm your router.

(This is just for the traditional BitTorrent DHT. Talking to WebTorrent trackers should be a lot more efficient.)

I recommend taking a look at "web seeding", which lets you host the files via http/https with something like Apache or nginx (which is simpler and more efficient) and include the http link in your .torrent file. Most torrent clients will only use the web seed when there are few or no peers available.

npm install create-torrent -g

create-torrent --urlList http://example.com/files/my-torrent1 /path/to/my-torrent1

You can read a bit more about web seeding here: http://getright.com/seedtorrent.html

[kevinsimper]

Thanks for that really detailed answer, I can see the problem.

The reason however that I am using torrent are because the internet connection really often drop here in South America, so downloading by http is often a really bad experience. You would be amazed how bad it is! 😅

I think the solution is, whenever the client hits the page, the server callbacks and starts the torrent seeding. This way I am only seeding the pictures which I know are in demand.

[feross]

@kevinsimper I think you'll find that torrent clients like WebTorrent do a better job of downloading over http than a traditional web browser because they're fetching smaller pieces each time, so if one piece request fails the whole download won't fail.

But I think your approach of on-demand seeding sounds workable too! Kudos to you for making things better for yourself and others in South America -- good stuff!

(Did you know that rsync was invented by a New Zealander who wanted something that made fewer roundtrips than scp because New Zealand is so far away from everything?)

[lock]

This thread has been automatically locked because it has not had recent activity. To discuss futher, please open a new issue.