Browser Documentation Helia/Libp2p #103

TheGreatAlgo · 2023-04-25T06:22:49Z

Following our discussions at IPFS Thing on implementing Helia (which was incredibly helpful and saved so much time), I wanted to write some top level documentation on implementing it and how to get an in browser node working that supports IPNS and IPFS queries as quickly as possible. It is still missing information and could definitely expand more on it, but I figure this is a good starting point? My goal here was to discuss each component, why its needed and quick example on its implementation.

whizzzkid

please fix!

whizzzkid · 2023-04-26T23:34:46Z

IMPLEMENTATION.MD

+
+## DataStore and BlockStore
+
+The datastore is used to hold important information relating to peers and DHT records. If you do not define how you want to store it, it will use one in memory.


optionally, we can link the datastore implementation that it uses in memory? i.e. https://github.com/ipfs/helia/blob/main/packages/helia/src/storage.ts#L28 I don't think there are plans to move that out and make a standalone package, but good to have a reference.

whizzzkid · 2023-04-26T23:35:05Z

IMPLEMENTATION.MD

+
+Blockstore is used to hold the blocks on ipfs you want to store locally
+
+```    


Suggested change

```

```js

ditto for syntax highlighting.

whizzzkid · 2023-04-26T23:37:07Z

IMPLEMENTATION.MD

+## Transports
+
+Transports define the method in which we connect to different nodes on the network. When implementing Helia its important to know a couple things. 
+1. Where Helia will be run (Browser (Firefox, Chrome, Safari), Node)


the language can be simplified, Browsers (Firefox, Chrome, Safari), JS runtimes (e.g. node, deno, bun)

whizzzkid · 2023-04-26T23:37:41Z

IMPLEMENTATION.MD

+1. Where Helia will be run (Browser (Firefox, Chrome, Safari), Node)
+2. Which nodes likely have data you need
+
+The first point is important since tcp is not available in browser for example. It can only be used when running in Nodejs. If you are running in browser, WebTransports are not available in Safari but it is coming to Firefox. https://caniuse.com/webtransport


so we need to remove from the examples.

not sure what you mean?

I thought I saw an example, maybe I was wrong.

whizzzkid · 2023-04-27T02:38:05Z

IMPLEMENTATION.MD

+
+This allows our node to relay traffic to other nodes that you may wish not to be connected directly to the public. Not really helpful in the browser but can be implemented via:
+
+**TO BE FILLED IN**


I am not too familiar with this portion and how it works, was hoping someone more knowledgeable could fill in the gap

maybe just remove it, or add a TODO.

whizzzkid · 2023-04-27T02:40:49Z

IMPLEMENTATION.MD

+
+
+


inconsistent spacing, this does not matter in md but looks better.

whizzzkid · 2023-04-27T02:41:09Z

IMPLEMENTATION.MD

+const libp2p = await createLibp2p({
+    ... // placeholder for remaining config
+    peerDiscovery: [
+            bootstrap({


inconsistent indentation

whizzzkid · 2023-04-27T02:42:28Z

IMPLEMENTATION.MD

+In the below example, we will want to load IPNS and IPFS since we will be resolving some IPNS names and fetching content as well. 
+
+You will also want to wait a couple seconds to generate all the peers and establish connection to network before trying to access content
+```


can this be added to https://github.com/ipfs-examples/helia-examples/tree/main/examples?

sure, do i do a pull request just on this portion?

yes, please do that, thanks!

Faolain · 2023-05-10T23:57:26Z

bumping @whizzzkid whenever you get a moment ^

whizzzkid

Thanks!

whizzzkid · 2023-05-11T05:42:39Z

IMPLEMENTATION.MD

+1. Where Helia will be run (Browser (Firefox, Chrome, Safari), Node)
+2. Which nodes likely have data you need
+
+The first point is important since tcp is not available in browser for example. It can only be used when running in Nodejs. If you are running in browser, WebTransports are not available in Safari but it is coming to Firefox. https://caniuse.com/webtransport


I thought I saw an example, maybe I was wrong.

whizzzkid · 2023-05-11T05:43:06Z

IMPLEMENTATION.MD

+
+This allows our node to relay traffic to other nodes that you may wish not to be connected directly to the public. Not really helpful in the browser but can be implemented via:
+
+**TO BE FILLED IN**


maybe just remove it, or add a TODO.

whizzzkid · 2023-05-11T05:43:46Z

IMPLEMENTATION.MD

+In the below example, we will want to load IPNS and IPFS since we will be resolving some IPNS names and fetching content as well. 
+
+You will also want to wait a couple seconds to generate all the peers and establish connection to network before trying to access content
+```


yes, please do that, thanks!

TheGreatAlgo · 2023-05-15T09:41:51Z

made requested changes and added example to helia examples

SgtPooki

This is a great writeup. I know @achingbrain had some questions about this and maybe wanted this to be a blog post, or potentially post in libp2p directly and then link to it from helia, but I'll leave that decision to him.

SgtPooki · 2023-05-25T19:56:11Z

IMPLEMENTATION.MD

+
+## Circuit Relay
+
+This allows our node to relay traffic to other nodes that you may wish not to be connected directly to the public. Not really helpful in the browser but can be implemented via:


@achingbrain I saw that we have circuit-relay enabled in the new default libp2p config for the browser. Is this statement true?

SgtPooki · 2023-05-25T19:56:31Z

IMPLEMENTATION.MD

+
+This allows our node to relay traffic to other nodes that you may wish not to be connected directly to the public. Not really helpful in the browser but can be implemented via:
+
+**TO DO**


The text here, and content below, needs to be updated.

I am not familiar enough to write this portion. I am still learning helia

SgtPooki · 2023-05-25T19:57:52Z

IMPLEMENTATION.MD

+
+## Stream Muxing
+
+It allows multiple conversations on the same line by giving it a unique identifier.


Suggested change

It allows multiple conversations on the same line by giving it a unique identifier.

It allows multiple conversations(Streams) on the same line(Connection) by giving it a unique identifier.

It could be useful to use/translate common libp2p terminology with a legend or something.

SgtPooki · 2023-05-25T19:58:51Z

IMPLEMENTATION.MD

+        validators: {
+            ipns: ipnsValidator,
+        },
+        selectors: {
+            ipns: ipnsSelector,
+        },


This isn't necessary for DHT config to work. It could be beneficial to introduce the ipns config as a subitem of ## DHT

also not too familiar with everything. Still learning and documenting places that caused difficulties for me

SgtPooki · 2023-05-25T19:59:58Z

IMPLEMENTATION.MD

+            list: [
+                "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
+                "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
+                "/dnsaddr/bootstrap.libp2p.io/p2p/QmZa1sAxajnQjVM8WjWXoMbmPd7NsWhfKsPkErzpm9wGkp",
+                "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
+                "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",


this should be updated to use the same as https://github.com/ipfs/helia/blob/main/packages/helia/src/utils/bootstrappers.ts, which is based off of https://github.com/ipfs/kubo/blob/da28fbc65a2e0f1ce59f9923823326ae2bc4f713/config/bootstrap_peers.go#L17

It might be good to mention the place to look(links above) to keep your bootstrap list up to date.

SgtPooki · 2023-05-25T20:03:28Z

IMPLEMENTATION.MD

+
+In the below example, we will want to load IPNS and IPFS since we will be resolving some IPNS names and fetching content as well. 
+
+You will also want to wait a couple seconds to generate all the peers and establish connection to network before trying to access content


it might be useful to show the users code of how they can do this:

const isHeliaReadyToFetch = async (helia) => { // return when helia has peers connected. Can simply wait for peer connection event, or handle manually with interval/other. }

added a quick example

Faolain · 2023-07-07T22:03:16Z

bumping again @whizzzkid , imo some additional documentation is better than none

achingbrain · 2023-09-14T14:27:50Z

Thank you so much for opening this PR and for taking the time to respond to feedback.

Everything suggested in it is very sensible, so much so that it's basically become the default browser config you get when you invoke createHelia in a browser.

This is different to early Helia releases which were a lot more bare bones, which felt right at the time but lead to a lot of duplicated configuration for users, some of it incorrect. Instead now default config makes it a lot easier to get started.

This leads to the odd situation where although we're super grateful this PR was opened, the recommendations in the added markdown file are somewhat redundant so I'm going to close this, but thanks again for taking the time to document how to set things up so diligently.

For future documentation efforts we have enabled the wiki on this repo so we can iterate on the docs a bit faster.

TheGreatAlgo added 3 commits April 25, 2023 08:08

add implementation overview

39ae5b2

Delete Implementation.md

1bc0522

Create IMPLEMENTATION.MD

0a749db

TheGreatAlgo requested a review from a team as a code owner April 25, 2023 06:22

whizzzkid reviewed Apr 27, 2023

View reviewed changes

fix: minor edits

f143b30

whizzzkid approved these changes May 11, 2023

View reviewed changes

whizzzkid and others added 3 commits May 10, 2023 23:45

Merge branch 'main' into documentation-how-to

3ede3d2

fix: update to do

c87d2c5

fix: missing semicolon in example

c952cc1

TheGreatAlgo mentioned this pull request May 15, 2023

Fix: Browser Example ipfs-examples/helia-examples#62

Open

SgtPooki self-requested a review May 24, 2023 16:22

SgtPooki reviewed May 25, 2023

View reviewed changes

fix: update example and wording

6a52654

achingbrain force-pushed the main branch 6 times, most recently from fbeed88 to 9f9bc60 Compare August 16, 2023 18:21

achingbrain closed this Sep 14, 2023


		## DataStore and BlockStore

		The datastore is used to hold important information relating to peers and DHT records. If you do not define how you want to store it, it will use one in memory.


		Blockstore is used to hold the blocks on ipfs you want to store locally

		```


		This allows our node to relay traffic to other nodes that you may wish not to be connected directly to the public. Not really helpful in the browser but can be implemented via:

		TO BE FILLED IN


		## Circuit Relay

		This allows our node to relay traffic to other nodes that you may wish not to be connected directly to the public. Not really helpful in the browser but can be implemented via:


		This allows our node to relay traffic to other nodes that you may wish not to be connected directly to the public. Not really helpful in the browser but can be implemented via:

		TO DO


		## Stream Muxing

		It allows multiple conversations on the same line by giving it a unique identifier.

	It allows multiple conversations on the same line by giving it a unique identifier.
	It allows multiple conversations(Streams) on the same line(Connection) by giving it a unique identifier.


		In the below example, we will want to load IPNS and IPFS since we will be resolving some IPNS names and fetching content as well.

		You will also want to wait a couple seconds to generate all the peers and establish connection to network before trying to access content

Browser Documentation Helia/Libp2p #103

Browser Documentation Helia/Libp2p #103

Conversation

TheGreatAlgo commented Apr 25, 2023

whizzzkid left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Faolain commented May 10, 2023

whizzzkid left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheGreatAlgo commented May 15, 2023

SgtPooki left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SgtPooki May 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Faolain commented Jul 7, 2023 • edited Loading

achingbrain commented Sep 14, 2023 • edited Loading

SgtPooki May 25, 2023 •

edited

Loading

Faolain commented Jul 7, 2023 •

edited

Loading

achingbrain commented Sep 14, 2023 •

edited

Loading