[CLOSED] Support rarest piece first strategy

[feross]

Issue by fisch0920
Thursday May 22, 2014 at 14:15 GMT
Originally opened as https://github.com/feross/bittorrent-client/pull/20

---

Fixes #18.

Torrents now support an additional 'strategy' option, supporting 'rarest' and 'sequential'. Note that we'll probably want to add a third 'hybrid' option in the future that tries to intelligently blend the two, taking into account any open file_streams associated with the torrent.

---

fisch0920 included the following code: https://github.com/feross/bittorrent-client/pull/20/commits

[feross]

Comment by fisch0920
Thursday May 22, 2014 at 14:16 GMT

---

Note that I left the default strategy as sequential if left unspecified.

[feross]

Comment by fisch0920
Thursday May 22, 2014 at 14:18 GMT

---

rarity_map implementation is based on @astro's implementation.

[feross]

Comment by astro
Friday May 23, 2014 at 06:41 GMT

---

In my bitorama experiment, I run rarest/random mode by default but add sequential pieces to the interesting queue whenever there's a file_stream waiting for pieces to complete.

It's a good idea to go rarest/random by default so that swarms can maintain an even pieces distribution.

[feross]

Comment by fisch0920
Friday May 23, 2014 at 17:24 GMT

---

Latest commit in this PR adds tests for simultaneous downloads, which I previously thought wasn't working. Possibly with all of the recent changes / refactoring, simultaneously downloading multiple torrents works correctly now both with DHT enabled and disabled, and theses tests will ensure we won't regress on this.

Definitely still need to fix the DHT implementation, but it's a great sign to see these tests passing.

Referenced in this issue #24

[feross]

Comment by fisch0920
Friday May 30, 2014 at 15:31 GMT

---

Latest commit in this PR ports over the recent, more legit choking implementation from torrent-stream. This should now address https://github.com/feross/bittorrent-client/issues/15

[feross]

Comment by fisch0920
Saturday May 31, 2014 at 04:42 GMT

---

Latest commits in this PR add support for resuming torrent downloads by adding initial piece verification. Note that this required shuffling around some of the ordering of things and now torrents emit both 'metadata' and 'ready' events. 'metadata' when the torrent's received its metadata, and 'ready' when it's fully ready to start transport (meaning the storage, swarm, and trackers have all been initialized including initial piece verification of the storage).

[feross]

Comment by feross
Tuesday Jun 03, 2014 at 17:22 GMT

---

This is awesome, merged.

A few thoughts: eventually, I think we'll want to get to an API where the user doesn't need to specify a strategy 99% of the time. We'll just do the right thing.

• Default to "rarest" strategy.
• If they ask for a file stream, then fetch that file using "sequential" strategy at highest priority. When finished, continue with "rarest" strategy for rest of torrent.
• If they select() a file, then fetch that file using "rarest" strategy (rarest pieces within that file). When finished, continue with "rarest" strategy for rest torrent.

[feross]

Comment by feross
Tuesday Jun 03, 2014 at 17:56 GMT

---

@fisch0920 I made some new commits based on my feedback. Please take a look and make sure it all looks good to you.

[feross]

Comment by fisch0920
Thursday Jun 05, 2014 at 00:56 GMT

---

@feross I've been pretty busy lately, so I haven't had time to go over things thoroughly, but all your changes look solid and definitely clean things up for the better.

There was one syntax error in webtorrent that I've fixed in this PR #71

[feross]

Comment by feross
Thursday Jun 05, 2014 at 01:04 GMT

---

Thanks for the catch! Also, I just made you a committer on webtorrent and bittorrent-client. I'm less busy than I was for the last two weeks, so I'm going to start working again on evenings. :) Hah, just as you become busy...

[feross]

Comment by fisch0920
Thursday Jun 05, 2014 at 01:16 GMT

---

Tag team match -- you're in! :)