Revise the way private network mycelium instances are connected

[LeeSmet]

It seems that currently, when a private network is created which runs mycelium, a node is set up which connects to the public nodes, and this happens for each mycelium enabled private network. Because private network namespaces are isolated from eachother, the nodes won't be able to connect to one another over a link local interface. As a result, traffic which is local to the node (from one private network to another on the same node) will have to travel to a public node and back. On top of this, creating multiple connections to a public node from the same zos node is rather wasteful for bandwidth and cpu time for both the farmer and the public nodes.

It would be a lot better if the private network mycelium instances simply connect to the mycelium instance running on the zos host. This way, traffic between 2 private networks on the same node can be routed by the host mycelium instance, and only 1 connections is made to public nodes which avoids duplicated protocol messages and thus wasted bandwdith/cpu time

[sabrinasadik]

Reminder: The public nodes are not the problem

If multiple VMs and therefore network namespaces are deployed on a farm which is in a residential network, the consumer-grade ISP provided router with it's buffer bloat can generally not handle the amount of packets per second being transmitted. That will also probably trigger limitations from the ISP itself.

[ashraffouda]

@LeeSmet what u mean is we make mycelium on user network part of the myc network on the host? can u elaborate more plz? :)

[sabrinasadik]

Right now, every private network tries to talk to the outside world independently by connecting to public Mycelium nodes. Even if two private networks are on the same computer, they don’t talk directly—they go all the way out to the public network and then back in.

The better idea:
Let the private networks on the same Zos node talk to the single Mycelium "hub" on that host. Then:

• Traffic between networks on the same zos node stays local.
• Only one connection is needed to talk to the public nodes.
• It saves bandwidth and CPU

[Omarabdul3ziz]

as suggested to change peer list for the private mycelium instances to be the address that myc listening on on the host. the host address --peers tcp://100.127.0.1:9651

tracing path from a vm to another vm deployed on the same node i got

root@jrvm31:~# tracepath 539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653
 1?: [LOCALHOST]                        0.042ms pmtu 1500
 1:  5ba:2fab:f733:1a0e::1                                 0.214ms
 1:  5ba:2fab:f733:1a0e::1                                 0.240ms
 2:  5ba:2fab:f733:1a0e::1                                 0.074ms pmtu 1400
 2:  no reply
 3:  no reply
 4:  539:e7d7:8c5f:50ae:fee:f26a:c9a2:c36e                 1.221ms
 5:  539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653                1.548ms reached
     Resume: pmtu 1400 hops 5 back 5

it looks it doesn't go outside the node, but also this is the same path it takes when the instance runs with the public peers.

is there a better way to trace the path to the other vm to make sure it didn't go outside. or does that means mycelium was already internally routing traffic between vms on the same node?

[Omarabdul3ziz]

using traceroute to trace connection form vm1 to vm0 deployed on the same node.

before the changes

root@jrvm31:~# traceroute 539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653
traceroute to 539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653 (539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653), 30 hops max, 80 byte packets
 1  5ba:2fab:f733:1a0e::1 (5ba:2fab:f733:1a0e::1)  0.196 ms  0.032 ms  0.025 ms
 2  * * *
 3  539:e7d7:8c5f:50ae:: (539:e7d7:8c5f:50ae::)  0.259 ms  0.259 ms  0.206 ms
 4  539:e7d7:8c5f:50ae:fee:f26a:c9a2:c36e (539:e7d7:8c5f:50ae:fee:f26a:c9a2:c36e)  0.444 ms  0.370 ms  0.297 ms
 5  539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653 (539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653)  1.222 ms  41.277 ms  41.082 ms

after the changes

root@jrvm31:~# traceroute 539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653
traceroute to 539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653 (539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653), 30 hops max, 80 byte packets
 1  5ba:2fab:f733:1a0e::1 (5ba:2fab:f733:1a0e::1)  0.203 ms  0.982 ms  0.937 ms
 2  * * *
 3  539:e7d7:8c5f:50ae:: (539:e7d7:8c5f:50ae::)  0.904 ms  0.898 ms  1.742 ms
 4  539:e7d7:8c5f:50ae:fee:f26a:c9a2:c36e (539:e7d7:8c5f:50ae:fee:f26a:c9a2:c36e)  2.627 ms  2.618 ms  2.612 ms
 5  539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653 (539:e7d7:8c5f:50ae:ff0f:c935:b5b6:c653)  2.509 ms  42.533 ms  42.296 ms

it is the same