Awesome Endeavour: Async Iterators

[daviddias]

@libp2p/javascript-team see thread at Awesome Endeavour: Async Iterators

Migration Notice: As libp2p modules migrate to async/await there will be some level of incompatibility when trying to include the latest release of modules. You can see the latest support minor versions of each module below.

    "libp2p-bootstrap": "^0.9.x",
    "libp2p-circuit": "^0.3.x",
    "libp2p-delegated-content-routing": "^0.2.x",
    "libp2p-delegated-peer-routing": "^0.2.x",
    "libp2p-floodsub": "^0.17.x",
    "libp2p-gossipsub": "^0.0.x",
    "libp2p-kad-dht": "^0.15.x",
    "libp2p-mdns": "^0.12.x",
    "libp2p-mplex": "^0.8.x",
    "libp2p-secio": "^0.11.x",
    "libp2p-spdy": "^0.13.x",
    "libp2p-tcp": "^0.13.x",
    "libp2p-webrtc-star": "^0.16.x",
    "libp2p-websockets": "^0.12.x",
    "libp2p-websocket-star": "^0.10.x",
    "libp2p-websocket-star-rendezvous": "^0.4.x"

JS Libp2p Refactor

As more of the dependencies for libp2p are finishing their migrations, we are starting on the internals of js-libp2p. Due to the fact that there are a LOT of things that need to be updated internally, we will be cutting the work into smaller, isolated chunks. This will allow us to iterate, avoiding a much larger PR.

Working Branch

All async PRs to js-libp2p internals that break compatibility should be opened against the refactor/async-await branch.

Submitting a PR

If there is a subsystem that you wish to take on, please add a comment to the internals refactor Trello card, or open a DRAFT PR, so that others know it's being worked on.

• The existing Libp2p test suite will be removed from the refactor branch. This will allow each subsystem refactor to add new tests and avoid duplication. When refactoring a subsystem, you may pull pertinent tests from master and refactor those to be async, if you are not sure what should be tested.
• A PR should have a full test suite for that system. Whenever possible, we should test the internals of a system, without relying on all of libp2p. This ensures we're able to test in better isolation, and will help with being able to iterate on the refactor.

Async Iterators Guidance

JS Libp2p is migrating to iterable streams from pull-streams. If you are not familiar with iterable streams you can see an overview of their various forms here. They are used in much the same way that pull-streams are, except they can be iterated on.

For a more in depth, evolving guide on how to use iterable streams, check out the iterables readme. If you have a use case that is not yet in the guide that other would benefit from, please submit a PR!

[daviddias]

Trello Board

[jacobheun]

I've updated the original issue post to include our approach for iterating on the internal refactor of js-libp2p.

[daviddias]

@jacobheun @vasco-santos one thing I couldn't find notes of but I believe to be the case. The js-libp2p top level API should be updated to remove callbacks and pull streams and only expose the async iterators interface. Is there a card for this?

[vasco-santos]

@jacobheun @vasco-santos one thing I couldn't find notes of but I believe to be the case. The js-libp2p top level API should be updated to remove callbacks and pull streams and only expose the async iterators interface. Is there a card for this?

Hey! Yes, that is part of the work for refactor/async-await branch. The trello card should be the Refactor js-libp2p internals