Skip to content

Latest commit

 

History

History
101 lines (66 loc) · 4.09 KB

Network-Configuration.adoc

File metadata and controls

101 lines (66 loc) · 4.09 KB
Raw

Network Configuration

Basic network information

Network configuration details for each IC-OS:

  • SetupOS

    • Basic network connectivity is checked via pinging nns.ic0.app and the default gateway. Virtually no network traffic goes through SetupOS.

  • HostOS

    • The br6 bridge network interface is set up and passed to the GuestOS VM through qemu (refer to hostos/rootfs/opt/ic/share/guestos.xml.template).

  • GuestOS

    • An internet connection is received via the br6 bridge interface from qemu.

Deterministic MAC Address

Each IC-OS node must have a unique but deterministic MAC address. To solve this, a schema has been devised.

Schema

  • The first 8-bits:

    • IPv4 interfaces: 4a

    • IPv6 interfaces: 6a

  • The second 8-bits:

    • We reserve the following hexadecimal numbers for each IC-OS:

      • SetupOS: 0f

      • HostOS: 00

      • GuestOS: 01

      • Boundary-GuestOS: 02

    • Note: any additional virtual machine on the same physical machine gets the next higher hexadecimal number.

  • The remaining 32-bits:

    • Deterministically generated

Example MAC addresses

  • SetupOS: {4a,6a}:0f:<deterministically-generated-part>

  • HostOS: {4a,6a}:00:<deterministically-generated-part>

  • GuestOS: {4a,6a}:01:<deterministically-generated-part>

  • BoundaryOS: {4a,6a}:02:<deterministically-generated-part>

  • Next Virtual Machine: {4a,6a}:03:<deterministically-generated-part>

Note that the MAC address is expected to be lower-case and to contain colons between the octets.

Deterministically Generated Part

The deterministically generated part is generated using the following inputs:

  1. IPMI MAC address (the MAC address of the BMC)

    1. Obtained via $ ipmitool lan print | grep 'MAC Address'`

  2. Deployment name

    1. Ex: mainnet

The concatenation of the IPMI MAC address and deployment name is hashed:

$ sha256sum "<IPMI MAC ADDRESS><DEPLOYMENT NAME>"
  # Example:
  $ sha256sum "3c:ec:ef:6b:37:99mainnet"

The first 32-bits of the sha256 checksum are then used as the deterministically generated part of the MAC address.

# Checksum
f409d72aa8c98ea40a82ea5a0a437798a67d36e587b2cc49f9dabf2de1cedeeb
# Deterministically Generated Part
f409d72a

Deployment name

The deployment name is added to the MAC address generation to further increase its uniqueness. The deployment name mainnet is reserved for production. Testnets must use other names to avoid any chance of a MAC address collision in the same data center.

The deployment name is retrieved from the deployment.json configuration file, generated as part of the SetupOS:

{
  "deployment": {
    "name": "mainnet"
  }
}

IPv6 Address

The IP address can be derived from the MAC address and vice versa: As every virtual machine ends in the same MAC address, the IPv6 address of each node on the same physical machine can be derived, including the hypervisor itself. In other words, the prefix of the EUI-64 formatted IPv6 SLAAC address is swapped to get to the IPv6 address of the next node.

When the corresponding IPv6 address is assigned, the IEEE’s 64-bit Extended Unique Identifier (EUI-64) format is followed. In this convention, the interface’s unique 48-bit MAC address is reformatted to match the EUI-64 specifications.

The network part (i.e. ipv6_prefix) of the IPv6 address is retrieved from the config.json configuration file. The host part is the EUI-64 formatted address.

Active backup

Note
 This feature is currently under development. See ticket NODE-869.

In order to simplify the physical cabling of the machine, Linux’s active-backup bonding technique is utilized. This operating mode also improves redundancy when more than one 10-gigabit ethernet network interface is connected to the switch. A node operator can decide to either just use one or all of the 10GbE network interfaces in the bond. The handling of the uplink and connectivity is taken care of by the Linux operating system.

Details can be found in:

ic/ic-os/setupos/rootfs/opt/ic/bin/generate-network-config.sh

Note that this mode does not increase the bandwidth/throughput. Only one link will be active at the same time.