feat(devnet): Local Gossamer Devnet

[timwu20]

Changes

• docker-compose.yml and associated docker images to setup a local devnet in a containerised environment.
• prometheus metrics are automatically piped to Datadog
• README.md for setup and running instructions

Tests

go test devnet/

Issues

• Continuous Integration Gossamer Devnet #1823

Primary Reviewer

• @noot

[codecov]

Codecov Report

Merging #2008 (0cbd42f) into development (6babe76) will increase coverage by 0.07%.
The diff coverage is 43.90%.

@@               Coverage Diff               @@
##           development    #2008      +/-   ##
===============================================
+ Coverage        60.22%   60.30%   +0.07%     
===============================================
  Files              194      201       +7     
  Lines            26585    26909     +324     
===============================================
+ Hits             16012    16228     +216     
- Misses            8692     8785      +93     
- Partials          1881     1896      +15     

Flag	Coverage Δ
unit-tests	60.30% <43.90%> (+0.07%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cmd/gossamer/flags.go	66.66% <ø> (ø)
dot/config.go	25.89% <ø> (-0.74%)	⬇️
dot/network/config.go	64.47% <ø> (ø)
devnet/cmd/update-dd-agent-confd/main.go	33.33% <33.33%> (ø)
dot/network/host.go	63.22% <43.75%> (-0.85%)	⬇️
cmd/gossamer/config.go	75.07% <100.00%> (-2.05%)	⬇️
dot/services.go	68.70% <100.00%> (+0.40%)	⬆️
dot/network/service.go	59.45% <0.00%> (-9.37%)	⬇️
dot/state/block_notify.go	81.90% <0.00%> (-8.58%)	⬇️
dot/network/discovery.go	59.23% <0.00%> (-6.03%)	⬇️
... and 28 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 84ec792...0cbd42f. Read the comment docs.

[noot]

@timwu20 looks epic, will test this out tomorrow!! do we want to add this to CI? (or wait until it's merged?)

[timwu20]

@timwu20 looks epic, will test this out tomorrow!! do we want to add this to CI? (or wait until it's merged?)

I'm still working on the actual ECS integration using the docker-compose ECS plugin. I'll integrate the CI then. I thought there would be value for the team to be able to run it locally first. The docker-compose.yml file will have more aws specific annotations and changes for it to deploy to ECS.

[qdm12]

Should we have this file in the docs directory somewhere instead? 🤔

[noot]

@timwu20 readme is a bit confusing, for this step:

To symlink to the repository root

# cd into your Gossamer root
cd ~/dev/gossamer

do you mean the --basepath? so I need to copy the docker files to the basepath?

[timwu20]

@timwu20 readme is a bit confusing, for this step:

To symlink to the repository root

# cd into your Gossamer root
cd ~/dev/gossamer

do you mean the --basepath? so I need to copy the docker files to the basepath?

I'm getting rid of this whole step. I got it working by just running docker-compose up in the devnet folder.

[qdm12]

nit usually build arguments and env variables are in uppercase out there, although lowercase works too.

[qdm12]

It would be nice to be able to fiddle with the bootnodes at runtime with i.e.:

Suggested change

	ENTRYPOINT service datadog-agent start && gossamer --key=${key} --bootnodes=/ip4/10.5.0.2/tcp/7001/p2p/12D3KooWMER5iow67nScpWeVqEiRRx59PJ3xMMAYPTACYPRQbbWU --publish-metrics --rpc --pubip=${pubip}
	ENV BOOTNODES=/ip4/10.5.0.2/tcp/7001/p2p/12D3KooWMER5iow67nScpWeVqEiRRx59PJ3xMMAYPTACYPRQbbWU
	ENTRYPOINT service datadog-agent start && gossamer --key=${key} --bootnodes=${BOOTNODES} --publish-metrics --rpc --pubip=${pubip}

You could also split that string in multiple ENV if you want (i.e. BOOTNODE_PORT=7001) using sensible defaults.

[timwu20]

I don't see a case where we would want to adjust the bootnodes for the gssmr devnet.

[qdm12]

At least the IP address 10.5.0.2 could be changed if we change the docker network right?

[timwu20]

In the next PR I do actually have to change the bootnode ports, so port will be configurable on bob.Dockerfile. I'm trying to hardcode the configuration for alice.Dockerfile since that is the lead node. I think being explicit in this case shows that everything is dependant on alice. The dockerfiles that I'm creating here aren't meant to be used standalone, but actually only orchestrated using the docker-compose.yml

[qdm12]

Did you pin tt := tt as well?

[qdm12]

For LAN ip addresses, you can go get github.com/vishvananda/netlink and use

func Test_findLANIP(t *testing.T) {
	ip, err := findLANIP()
	require.NoError(t, err)
	t.Log(ip)
}

var (
	ErrNoLinkFound            = errors.New("no link was found")
	ErrDefaultGatewayNotFound = errors.New("default gateway not found")
)

func findLANIP() (ip net.IP, err error) {
	links, err := netlink.LinkList()
	if err != nil {
		return nil, fmt.Errorf("cannot list links: %w", err)
	} else if len(links) == 0 {
		return nil, ErrNoLinkFound
	}

	for _, link := range links {
		routes, err := netlink.RouteList(link, netlink.FAMILY_ALL)
		if err != nil {
			return nil, fmt.Errorf(
				"cannot list routes for link %d: %w",
				link.Attrs().Index, err)
		}

		for _, route := range routes {
			if route.Dst == nil {
				return route.Gw, nil
			}
		}
	}

	return nil, ErrDefaultGatewayNotFound
}

Although that's limited to unix systems and introduces an indirect dependency on syscall (so for windows we would need different build tags etc.)

[qdm12]

I think you can also use secrets: in docker-compose.yml, if that may be of use. LINK

[qdm12]

Can't you use the stdlib's flag package for this?

[timwu20]

I prefer go-flags and just using a struct. It has served me well for previous projects.

[timwu20]

I don't know why I can't reply to your feedback about the lan IP stuff. Since this will only ever be used by the devnet, I'd rather not bring in extra code when the user can just force the node to broadcast the intended address.

[kishansagathiya]

should we return error here?

If we are here, that means we are failing to set externalAddr. externalAddr is getting used in the code after this, but not getting set. Looks like we should return the error here.

[timwu20]

This is how the code was. In the event we can't get a public IP via pubip I would assume nothing gets broadcasted. What do you think @noot?

[noot]

		libp2p.AddrsFactory(func(as []ma.Multiaddr) []ma.Multiaddr {
			addrs := []ma.Multiaddr{}
			for _, addr := range as {
				if !privateIPs.AddrBlocked(addr) {
					addrs = append(addrs, addr)
				}
			}
			if externalAddr == nil {
				return addrs
			}

			return append(addrs, externalAddr)
		}),

other addresses can still be advertised even if pubip.Get fails, so I think that's why I didn't make it return an error

[qdm12]

I don't know why I can't reply to your feedback about the lan IP stuff.

It's because it's replies to conversations above (probably hidden since they are many conversations) 😉

[github-actions]

🎉 This PR is included in version 0.6.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀