Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(iroh): downloader #1420

Merged
merged 97 commits into from
Sep 13, 2023
Merged

feat(iroh): downloader #1420

merged 97 commits into from
Sep 13, 2023

Commits on Aug 22, 2023

  1. starting from scratch

    @divagant-martian
    divagant-martian committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    20da0ef View commit details
    Browse the repository at this point in the history

  2. add download handle

    @divagant-martian
    divagant-martian committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    03869bd View commit details
    Browse the repository at this point in the history

  3. add sources and download info

    @divagant-martian
    divagant-martian committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    ceed4fd View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2023

  1. add response sender

    @divagant-martian
    divagant-martian committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    aab7849 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into download-manager

    @divagant-martian
    divagant-martian committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    3480f79 View commit details
    Browse the repository at this point in the history

  3. add fields for scheduling, code cancellation

    @divagant-martian
    divagant-martian committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    313cba4 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2023

  1. handle receiving a start download request

    @divagant-martian
    divagant-martian committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    031b59d View commit details
    Browse the repository at this point in the history

  2. improve docs

    @divagant-martian
    divagant-martian committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    47fcf3a View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2023

  1. add collection parser and store

    @divagant-martian
    divagant-martian committed Aug 26, 2023
    Configuration menu
    Copy the full SHA
    81c77ce View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into download-manager

    @divagant-martian
    divagant-martian committed Aug 26, 2023
    Configuration menu
    Copy the full SHA
    7a97b2c View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2023

  1. add docs

    @divagant-martian
    divagant-martian committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    9227ea3 View commit details
    Browse the repository at this point in the history

  2. add pending count fn to Dialer

    @divagant-martian
    divagant-martian committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    b8c916c View commit details
    Browse the repository at this point in the history

  3. add dialer, availability registry and concurrency limits

    @divagant-martian
    divagant-martian committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    e8c1f99 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2023

  1. add parts

    @divagant-martian
    divagant-martian committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    b08733d View commit details
    Browse the repository at this point in the history

  2. remove unused source enum. delegated now to the availability_registry

    @divagant-martian
    divagant-martian committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    325d1c9 View commit details
    Browse the repository at this point in the history

  3. add get_best_candidate

    @divagant-martian
    divagant-martian committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    370b436 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2023

  1. dial selected best peer

    @divagant-martian
    divagant-martian committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    40664a8 View commit details
    Browse the repository at this point in the history

  2. add comments

    @divagant-martian
    divagant-martian committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    aa2b3ef View commit details
    Browse the repository at this point in the history

  3. filling in random gaps

    @divagant-martian
    divagant-martian committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    82ab766 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into download-manager

    @divagant-martian
    divagant-martian committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    ba29846 View commit details
    Browse the repository at this point in the history

  5. start adding errors based on fsm error management

    @divagant-martian
    divagant-martian committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    0e362d5 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. remove progress stuff

    @divagant-martian
    divagant-martian committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    386edae View commit details
    Browse the repository at this point in the history

  2. do not add swear words in commit messages

    @divagant-martian
    divagant-martian committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    3284fa9 View commit details
    Browse the repository at this point in the history

  3. reworking active vs scheduled requests

    @divagant-martian
    divagant-martian committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    e75402c View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2023

  1. rework pending and connected info plus retry count

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    e40ee09 View commit details
    Browse the repository at this point in the history

  2. simplify cancelling a request

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    0fb64b5 View commit details
    Browse the repository at this point in the history

  3. rework get_peer_connection_for_download

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    29a8068 View commit details
    Browse the repository at this point in the history

  4. excluding peers is not necessary

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    05045a2 View commit details
    Browse the repository at this point in the history

  5. remove unused import

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    e8f79bf View commit details
    Browse the repository at this point in the history

  6. add retry constants

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    6d7f992 View commit details
    Browse the repository at this point in the history

  7. add retry backoff calculation

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    91ca199 View commit details
    Browse the repository at this point in the history

  8. remove obsolte code

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    0bab11c View commit details
    Browse the repository at this point in the history

  9. handle requests ready for processing

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    1353179 View commit details
    Browse the repository at this point in the history

  10. typos and docs

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    bac6c0d View commit details
    Browse the repository at this point in the history

  11. connect download

    @divagant-martian
    divagant-martian committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    892245d View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2023

  1. remove unused method

    @divagant-martian
    divagant-martian committed Sep 2, 2023
    Configuration menu
    Copy the full SHA
    558d1b9 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2023

  1. typo

    @divagant-martian
    divagant-martian committed Sep 3, 2023
    Configuration menu
    Copy the full SHA
    d28d1ea View commit details
    Browse the repository at this point in the history

  2. add on_download_completed

    @divagant-martian
    divagant-martian committed Sep 3, 2023
    Configuration menu
    Copy the full SHA
    191a726 View commit details
    Browse the repository at this point in the history

  3. lifetime issues

    @divagant-martian
    divagant-martian committed Sep 3, 2023
    Configuration menu
    Copy the full SHA
    79d9fd2 View commit details
    Browse the repository at this point in the history

  4. banning via removing the connection

    @divagant-martian
    divagant-martian committed Sep 3, 2023
    Configuration menu
    Copy the full SHA
    7427f00 View commit details
    Browse the repository at this point in the history

  5. prevent banned peers from taking all the connected peer slots

    @divagant-martian
    divagant-martian committed Sep 3, 2023
    Configuration menu
    Copy the full SHA
    8673f79 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2023

  1. add peer expiry

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    b046f37 View commit details
    Browse the repository at this point in the history

  2. extend to add and remove peers from the registry

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    cda280e View commit details
    Browse the repository at this point in the history

  3. add dummy registry

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    ca380d1 View commit details
    Browse the repository at this point in the history

  4. fix imports

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    4def2ab View commit details
    Browse the repository at this point in the history

  5. spawn service

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    d00a340 View commit details
    Browse the repository at this point in the history

  6. not so hot hot-swaping

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    6eacf27 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into download-manager

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    11388cd View commit details
    Browse the repository at this point in the history

  8. hot-swaping in examples

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    5ba80fe View commit details
    Browse the repository at this point in the history

  9. fix example

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    c518f15 View commit details
    Browse the repository at this point in the history

  10. fix some docs and imports

    @divagant-martian
    divagant-martian committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    c9cbc39 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. fix docs

    @divagant-martian
    divagant-martian committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    0f75cec View commit details
    Browse the repository at this point in the history

  2. our lord clippy

    @divagant-martian
    divagant-martian committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    2514acd View commit details
    Browse the repository at this point in the history

  3. more clippy

    @divagant-martian
    divagant-martian committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    c1b0b6e View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into download-manager

    @divagant-martian
    divagant-martian committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    5b4b120 View commit details
    Browse the repository at this point in the history

  5. allow to add initial peers at the same time with the download

    @divagant-martian
    divagant-martian committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    c6dcd3b View commit details
    Browse the repository at this point in the history

  6. debug failing test

    @divagant-martian
    divagant-martian committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    d3535c7 View commit details
    Browse the repository at this point in the history

  7. more debugging

    @divagant-martian
    divagant-martian committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    35416d0 View commit details
    Browse the repository at this point in the history

  8. actually print the event

    @divagant-martian
    divagant-martian committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    39aa67f View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. validate entries are strictly newer

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    dc8b9a8 View commit details
    Browse the repository at this point in the history

  2. receive dialer by param in preparation to abstract it away

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    01094e5 View commit details
    Browse the repository at this point in the history

  3. Revert "receive dialer by param in preparation to abstract it away" 

    This reverts commit 01094e5.
    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    d22373e View commit details
    Browse the repository at this point in the history

  4. abstract away the dialer to get rid of the magicEndpoint

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    d8d1512 View commit details
    Browse the repository at this point in the history

  5. propagate changes

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    354ab27 View commit details
    Browse the repository at this point in the history

  6. make trait public to keep the docs as they are

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    c035640 View commit details
    Browse the repository at this point in the history

  7. abstract away all the io

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    fe09521 View commit details
    Browse the repository at this point in the history

  8. write some invariants and plug them in the main downloader loop

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    b208ee8 View commit details
    Browse the repository at this point in the history

  9. remove unused piece of code

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    5ee842a View commit details
    Browse the repository at this point in the history

  10. more invariants

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    3ccbb3f View commit details
    Browse the repository at this point in the history

  11. move cancellation vack to the downloader, start smoke test

    @divagant-martian
    divagant-martian committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    5787c80 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2023

  1. NO SMOKE HERE!

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    f86af83 View commit details
    Browse the repository at this point in the history

  2. cleeepeeee

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    20d0626 View commit details
    Browse the repository at this point in the history

  3. restructure tests

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    e359e73 View commit details
    Browse the repository at this point in the history

  4. fmt

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    c78c658 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into download-manager

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    1039497 View commit details
    Browse the repository at this point in the history

  6. after typo

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    b04b681 View commit details
    Browse the repository at this point in the history

  7. add deduplication test

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    ececd4a View commit details
    Browse the repository at this point in the history

  8. add cancellation test

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    cba64e2 View commit details
    Browse the repository at this point in the history

  9. fix invariant check for concurrency limits

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    5e80d82 View commit details
    Browse the repository at this point in the history

  10. add max concurrent requests test

    @divagant-martian
    divagant-martian committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    8aa0356 View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2023

  1. refactor AvailabilityRegistry to ProviderMap

    @divagant-martian
    divagant-martian committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    5fe0c4f View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2023

  1. remove ProviderMap as a trait

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    0e3907b View commit details
    Browse the repository at this point in the history

  2. reduce get code duplication

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    442b3ac View commit details
    Browse the repository at this point in the history

  3. group io get code

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    aaef3c1 View commit details
    Browse the repository at this point in the history

  4. better renaming

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    bd7ceb0 View commit details
    Browse the repository at this point in the history

  5. add some more docs

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    1e2c194 View commit details
    Browse the repository at this point in the history

  6. cleaner file names

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    f159a72 View commit details
    Browse the repository at this point in the history

  7. code cleanup

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    aec7256 View commit details
    Browse the repository at this point in the history

  8. add metrics

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    4f92881 View commit details
    Browse the repository at this point in the history

  9. cleanup tests

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    89f6f29 View commit details
    Browse the repository at this point in the history

  10. make item public to use it in the docs

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    ce214d8 View commit details
    Browse the repository at this point in the history

  11. make item public to use it in the docs

    @divagant-martian
    divagant-martian committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    a33d8bb View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. review suggestions

    @divagant-martian
    divagant-martian committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    12c63dd View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2023

  1. restore sync tests

    @divagant-martian
    divagant-martian committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    8ff922d View commit details
    Browse the repository at this point in the history

  2. check invariants in debug as well

    @divagant-martian
    divagant-martian committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    7e704d7 View commit details
    Browse the repository at this point in the history

  3. misc updates from pr feedback

    @divagant-martian
    divagant-martian committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    aba53f2 View commit details
    Browse the repository at this point in the history

  4. adjust error management

    @divagant-martian
    divagant-martian committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    4a7e4b2 View commit details
    Browse the repository at this point in the history