Run functional tests with a local waku fleet

[igor-sirotin]

Requires:

With

• feat_: API to use a custom Waku fleet #6414
• test_: enhance support for running functional tests with local status-backend #6463

Description

1. When running functional tests, create a local waku fleet

• 1 boot node and 1 store node should be enough

• here's an example docker compose for it:

  version: '3.8'
  
  networks:
    waku_net:
      driver: bridge
  
  services:
    boot-1:
      build: .
      command: >
        --ports-shift=1
        --dns4-domain-name=boot-1
        --relay=true --filter=true --lightpush=true
      ports:
        - "60001:60001"
        - "9001:9001"
        - "8646:8646"
      networks:
        waku_net:
          aliases:
            - boot-1
  
    store:
      build: .
      command: >
        --ports-shift=2
        --dns4-domain-name=store
        --relay=true --store=true
        --staticnode="/dns4/boot-1/tcp/60001/p2p/<boot-1-peer-id>"
      ports:
        - "60002:60002"
        - "9002:9002"
        - "8647:8647"
      networks:
        waku_net:
          aliases:
            - store

• which I was using with a common Dockerfile:

  FROM wakuorg/nwaku:v0.34.0
  
  ENTRYPOINT [ \
    "/usr/bin/wakunode", \
    "--nat=extip:127.0.0.1", \
    "--peer-exchange=true", \
    "--rest-address=0.0.0.0", "--rest-admin", \
    "--keep-alive=true", "--max-connections=18000", \
    "--discv5-discovery=true", "--discv5-enr-auto-update=True", \
    "--cluster-id=16", "--shard=32", "--shard=64", \
    "--tcp-port=60000", "--discv5-udp-port=9000", \
    "--log-level=DEBUG", \
    # Docker DNS server
    "--dns-addrs-name-server=127.0.0.11" \ 
  ]

2. Build fleet config into a JSON file.

There are 2 ways:

1. Static keys (simple)
   2. Generate 2 keys using openssl rand -hex 32
   3. Run waku nodes with --nodekey, so that they always have the same peer IDs.
   4. Write a static JSON committed to the repo
2. Collect addresses (medium difficulty, not sure there's any reason to bother)
   1. Run a script that connects to both nodes by RPC
   2. Request {{HOST}}/debug/v1/info from each node, get ENRs from the responses
   3. Write it all to a JSON file (could be in temp dir on CI)

JSON config file format is as FleetMap. E.g.:

{
  "status-go.test": {
    "clusterId": 16,
    "wakuNodes": [
      "enr:<ENR of the boot node>"
    ],
    "discV5BootstrapNodes": [],
    "storeNodes": [
      {
        "id": "store-1",
        "enr": "enr:<ENR of the store node>",
        "addr": "",
        "fleet": "status-go.test"
      }
    ]
  }
}

3. Pass the path to this JSON file to the functional tests.

Just run the tests with --waku-fleets-config=<path-to-config> --waku-fleet=status-go.test 🎉