Experimental: private-networks doesn't allow webui to load

[NorseGaud]

Version information:

Description:

Followed https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#private-networks, except for adding a bootstrap node, and I can't seem to load the webui. It just spins forever until it times out.

I run:

❯ ipfs init --profile server

❯ go get github.com/Kubuxu/go-ipfs-swarm-key-gen/ipfs-swarm-key-gen
❯ ~/go/bin/ipfs-swarm-key-gen > ~/swarm.key
❯ ln -s ~/swarm.key ~/.ipfs/swarm.key

❯ ipfs bootstrap rm --all

❯ ipfs daemon
Initializing daemon...
go-ipfs version: 0.7.0-ea77213
Repo version: 10
System version: amd64/darwin
Golang version: go1.14.12
Swarm listening on /ip4/10.8.1.9/tcp/4001
Swarm listening on /ip4/10.8.1.9/udp/4001/quic
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.0.107/tcp/4001
Swarm listening on /ip4/192.168.0.107/udp/4001/quic
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
API server listening on /ip4/127.0.0.1/tcp/5001
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready

Is a bootstrap node required?

[welcome]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[Stebalien]

Yes. The webui is downloaded on-demand and at least one node on your private network needs a copy.

You can make this work by:

1. Connecting one (fresh) node to the main network.
2. Pin "/ipfs/bafybeif4zkmu7qdhkpf3pnhwxipylqleof7rl6ojbe7mq3fzogz6m4xk3i" (the current webui).
3. Connect that node to the private network.

[NorseGaud]

Thanks @Stebalien ! I've got my ipfs and cluster daemon running, but the pin add is just frozen and doesn't ever complete:

❯ tail -f /tmp/ipfs-*                                                           

==> /tmp/ipfs-cluster-daemon.log <==
        /ip4/127.0.0.1/tcp/9096/p2p/12D3KooWB68YmSswqqP1Pr7MMejGhe3whqaQw5q4mtEo9EkxjNPw


2021-01-19T13:04:53.034-0500    INFO    restapi REST API (HTTP): /ip4/127.0.0.1/tcp/9094
2021-01-19T13:04:53.034-0500    INFO    ipfsproxy       IPFS Proxy: /ip4/127.0.0.1/tcp/9095 -> /ip4/127.0.0.1/tcp/5001
2021-01-19T13:04:53.035-0500    INFO    crdt    crdt Datastore created. Number of heads: 0. Current max-height: 0
2021-01-19T13:04:53.035-0500    INFO    crdt    'trust all' mode enabled. Any peer in the cluster can modify the pinset.
2021-01-19T13:04:53.046-0500    INFO    cluster Cluster Peers (without including ourselves):
2021-01-19T13:04:53.046-0500    INFO    cluster     - No other peers
2021-01-19T13:04:53.046-0500    INFO    cluster ** IPFS Cluster is READY **

==> /tmp/ipfs-daemon.log <==
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic
API server listening on /ip4/127.0.0.1/tcp/5001
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready

> ipfs pin add /ipfs/bafybeif4zkmu7qdhkpf3pnhwxipylqleof7rl6ojbe7mq3fzogz6m4xk3i
(nothing happens and I have to ctrl+c)

Here is how I start my ipfs and cluster:

ipfs-cluster-service init --consensus crdt
ipfs init --profile server
LIBP2P_FORCE_PNET=1
go get github.com/Kubuxu/go-ipfs-swarm-key-gen/ipfs-swarm-key-gen
~/go/bin/ipfs-swarm-key-gen > ~/swarm.key
ln -s ~/swarm.key ~/.ipfs/swarm.key
ipfs bootstrap rm --all
# Run daemons
ipfs daemon &>/tmp/ipfs-daemon.log &
sleep 10
ipfs-cluster-service daemon &>/tmp/ipfs-cluster-daemon.log &

[Stebalien]

You need to pin from a node on the public network first, then move that node to your private network. E.g., 1. Create a temporary node on the public network. 2. Pin it. 3. Add the swarm key to the temporary node. 4. Restart the temporary node and connect to your cluster. 5. Pin from the cluster.

[Stebalien]

Visiting for triage:

• We'd like to avoid bundling the webui because it's large.
• We should provide a better error message and a loading screen.
• When applicable, we should display a message describing how to import the webui into a private swarm.

[NorseGaud]

Is there a way to get the webui without syncing to the public network? Like, download what's necessary and place it manually in the right spot?

[aschmahmann]

@NorseGaud at a high level what you're asking is how do I get information from the public network into my private network. Fundamentally to do this you need some level of "bridge", but assuming your main constraint is keeping your private network isolated you can do this by spinning up a public node in addition to your private one and utilizing ipfs dag export/import.

For example:
On Public Node:

ipfs dag export bafybeif4zkmu7qdhkpf3pnhwxipylqleof7rl6ojbe7mq3fzogz6m4xk3i > webui.car

On Private Node:

ipfs dag import webui.car

The versions of the webui previously used by go-ipfs can be found https://github.com/ipfs/go-ipfs/blob/675ef92a2ab06ae70b1f982e37908db4085e724f/core/corehttp/webui.go.

I don't recall offhand if private networks can utilize DNSLink, but if so then you can also follow along the latest published webui via webui.ipfs.io and periodically having a node run ipfs name resolve /ipns/webui.ipfs.io and if it's any different then last time doing an export+import again.

Note: new releases of go-ipfs may bundle updated webui's so you'll need to track that as well unless you're using the DNSLink tracking mechanism above.

[NorseGaud]

Thanks for the info!! Unfortunately I'm not able to connect at all to the public network. 😔

[aschmahmann]

@NorseGaud what do you mean? Can you move data via USB? If so then you can just go to a totally different machine on a different network to export the .car file and then put it on a USB and move it to the private network machine and import the .car file