Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How webseed really works? Do we have to seed the file that has webseed url? #1393

Closed
hz0324 opened this issue May 17, 2018 · 13 comments
Closed
Labels

Comments

@hz0324
Copy link

@hz0324 hz0324 commented May 17, 2018

For example, we have a magnet like this:

magnet:?xt=urn:btih:000ee0007db5c059931c59471b3dbda97240a607&tr=udp%3A%2F%2Fexplodie.org%3A6969&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Ftracker.empire-js.us%3A1337&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337&tr=wss%3A%2F%2Ftracker.btorrent.xyz&tr=wss%3A%2F%2Ftracker.fastcast.nz&tr=wss%3A%2F%2Ftracker.openwebtorrent.com&ws=xxx.xx.xxx.xx/000ee0007db5c059931c59471b3dbda97240a607/Secret.png

The last part is the weebseed url. You can just visit the url and get the file. In such case, do we have to seed the file to make this magnet link work? What will happen if there is no seeder but the webseed url is working?

I have tried to search for this question ,but nothing related found.
I also have tried to download this magnet link while webseed url is working, but no seeders for this file. It din't start the downloading. I am confused.

Please help, thanks.
Thanks.

@hz0324 hz0324 changed the title How webseed really works? Do we have to seed the files that has webseed url? How webseed really works? Do we have to seed the file that has webseed url? May 17, 2018
@KayleePop

This comment has been minimized.

Copy link
Contributor

@KayleePop KayleePop commented May 17, 2018

Make sure to include http:// or https:// at the beginning 😉

&ws=http://xxx.xx.xxx.xx/000ee0007db5c059931c59471b3dbda97240a607/Secret.png

@hz0324

This comment has been minimized.

Copy link
Author

@hz0324 hz0324 commented May 17, 2018

@KayleePop Thank you, I didn't know that.

I have tried again. It is still not downloading if there is no seeders for this file. Does seeder necessary?
Thanks.

@KayleePop

This comment has been minimized.

Copy link
Contributor

@KayleePop KayleePop commented May 17, 2018

No, a seeder isn't required.

Try pointing to the directory instead of the file:

&ws=http://xxx.xx.xxx.xx/000ee0007db5c059931c59471b3dbda97240a607

@feross

This comment has been minimized.

Copy link
Member

@feross feross commented May 17, 2018

Actually you still need at least one peer in order to fetch the .torrent file metadata, which is not part of the magnet link.

If you want the magnet link to work even when there are no peers, you need to also include a link to the .torrent file in the magnet link using the xs URL parameter.

See this magnet link as an example: magnet:?xt=urn:btih:dd8255ecdc7ca55fb0bbf81323d87062db1f6d1c&dn=Big+Buck+Bunny&tr=udp%3A%2F%2Fexplodie.org%3A6969&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Ftracker.empire-js.us%3A1337&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337&tr=wss%3A%2F%2Ftracker.btorrent.xyz&tr=wss%3A%2F%2Ftracker.fastcast.nz&tr=wss%3A%2F%2Ftracker.openwebtorrent.com&ws=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2F&xs=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2Fbig-buck-bunny.torrent

@feross feross closed this May 17, 2018
@feross feross added the question label May 17, 2018
@Truelion

This comment has been minimized.

Copy link

@Truelion Truelion commented May 17, 2018

just want to add that i too have been trying, there are limited docs, examples and support. I feel WebTorrent is an amazing api, just no direction, it's like a gold nugget burried in mud, you have to get to it.

@Truelion

This comment has been minimized.

Copy link

@Truelion Truelion commented May 17, 2018

What really annoys me is how quickly -- and without satisfaction -- these threads are closed out, as if other threads are so much more important, or that it's clogging someones inbox in annoyance. Why not leave the discussion open for feedback, learning, sharing, at least for more than a day...Some of these are closed in minutes LIKE A FAST FOOD joint.

Not another stack overflow where the mods get annoyed fast.

@hz0324

This comment has been minimized.

Copy link
Author

@hz0324 hz0324 commented May 17, 2018

@Truelion Yeah, you are right. We really like the cool feeling of WebTorrent, but somewhat frustrated by the doc now.

You can build a simple app really quickly according to the guide and API doc. But when you want to build something more serious, the documentation looks so simple and crude. You have to bother others to find the answer like I do.

Honestly, they are being really helpful and I am really grateful for that. But we still want a more thorough documentation, at lease we can do things more confident.

@SilentBot1

This comment has been minimized.

Copy link
Member

@SilentBot1 SilentBot1 commented May 17, 2018

@hz0324

As mentioned by Feross above, you do not need a WRTC seed, but what you need is a way of getting the metadata of the torrent. With just an infoHashes a typical BitTorrent client would query the Mainline DHT for peers then use ut_metadata to get the metadata from a peer, assuming a peer exists with the metadata. With a magnetURI you can specify trackers using &tr, in combination with using the DHT these trackers would be used to find peers and again use ut_metadata to get the metadata from the peers.

In both situations a peer must exist with the metadata, so there must be a seeding peer.

This can be avoided by using the &xs parameter in the magnetURI or by specifying torrent.xs and then calling torrent._getMetadataFromServer() after being added to the client, which allows you to specify a HTTP(s) path to the .torrent file which contains the necessary metadata needed to start.

The torrent metadata must be held before a peer can start downloading any pieces of the torrent, if there are no peers to share the metadata and no &xs or torrent.xs set, even with a WebSeed set, the torrent will not begin downloading.

Hope this helps.

@Truelion

This comment has been minimized.

Copy link

@Truelion Truelion commented May 17, 2018

Honestly, i feel like Feross deserves an engineering award for innovation, seems like he is the main contributor to offering help and yes, the help we get is much appreciated. I've built the crude apps as the docs shows, unfortunately, anything more advanced and everyone points to the CLI tools.

Hey! If i wanted the CLI tools i wouldn't even be here. 20 years in engineering and it's difficult to dig through the code to build anything more sophisticated

@SilentBot1

This comment has been minimized.

Copy link
Member

@SilentBot1 SilentBot1 commented May 17, 2018

@Truelion this is an issue tracker for the project, the title wasn't a issue but a question so it was closed though the thread still remains unlocked so discussion can still occur, don't take this as not caring, it just clutters up the project when there are actual issues which need addressing.

The titles question technically isn't about the WebTorrent library either but about part of the BitTorrent protocol, specifically BEP 19, which defines what a WebSeed is and how it should function.

I agree that there is little documentation around WebTorrent, specifically WebSeeds, I'm in the middle of writing documentation which explains WebSeeds, how they work, an example web server configuration and how to add and remove WebSeeds from a torrent. I should have it finished within the next two days.

@hz0324 @Truelion Are there any other areas which you feel could use more documentation?

Edit: initial draft of documentation available over here though no PR yet as I don't feel comfortable with the quality of my writing, I need to refine it more before I'm happy.

@KayleePop

This comment has been minimized.

Copy link
Contributor

@KayleePop KayleePop commented May 17, 2018

Ah, I thought I was forgetting something.

To expound on that, here's how you generate a .torrent file

let client = new WebTorrent()
client.seed('/path/to/file', function (torrent) {
  torrent.torrentFile // a buffer of the .torrent file
  // host it at the url you put after &xs=
})
@hz0324

This comment has been minimized.

Copy link
Author

@hz0324 hz0324 commented May 18, 2018

@KayleePop Thank you very much, you are being very helpful.

@SilentBot1 I am Ok now, I have FINALLY make it working. I think you guys can put more attention to the basic concepts in P2P area. Although you guys are experts in this area, we are not. I think the most users of this project are not either.

Sometimes it is really hard to understand how this project works under the hood. And, since we are not familiar about the topic, I don't even know where to find information. For example, I find the link about WebSeed in the doc, but it is really hard to understand why and when to use it, and how to use it exactly.

Besides that, I think some of the APIs are not clear enough, lacking detail information. For example, addPeers(), what will happen? why and when to use it? What's the pros and cons?

I really like this project and appreciate what you guys did, hope it gets better and better.

@Truelion

This comment has been minimized.

Copy link

@Truelion Truelion commented May 18, 2018

@SilentBot1 - thank you for trying to help. @hz0324 has stated what i think is a general picture of how most feel about WebTorrent in general. Poor documentation, no samples beyond a simple client.seed() usage. We may be new to bit torrent terms in general, things like DHT is a foreign topic. How and when and why do i use it, peer discovery? WebSeeds.

Here's how you can improve the docs:

  • lots of small code examples covering simple to advanced topics

  • more smaller code examples of doing just one thing, like:
    1. just show download speed simply, don't burry it in 1000 lines of a demo
    2. how many peers connected?
    3. how many seeding

  • gather small code samples into a bigger "mini-app" for context later (see instant.io replica below)

  • cover ideas which are native to the designers but foreign to us (DHT, Peer Discovery, etc)

  • give us design ideas and larger architectural pictures for achieving things like

  • WebSeeding > this needs pages and pages of info devoted to this one area !!

  • magnet uris - ideas for use, storage, what are some possibilities?

  • how to do more in client browser js, we're not always going for a Node app

  • And finish off with a "how to build an instant.io" replica 👍

In general, lots of info is missing from the docs, above is a starting point.

@lock lock bot locked as resolved and limited conversation to collaborators Aug 16, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.