Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic Waku simulation #1

Open
oskarth opened this issue Dec 9, 2019 · 11 comments
Open

Basic Waku simulation #1

oskarth opened this issue Dec 9, 2019 · 11 comments
Assignees

Comments

@oskarth
Copy link
Member

@oskarth oskarth commented Dec 9, 2019

Problem

We don't know if Waku mode will actually significantly decrease bandwidth vacp2p/specs#49

We have a theoretical model that claims it well, but we need more confirmation. Core integration is one way (and the ultimate goal), but it'd be good to know earlier if we are actually solving the problem.

Acceptance criteria

This is a minimal simulation for Waku. To start with, it should

  • launch N waku nodes
  • that send M messages over some basic (possibly hardcoded configuration)
  • prints statistics that can be correlated to bandwidth
  • do this with both bloom filter and topic interest (i.e. similar to the model)

I.e. you can run a script and it shows you some basic BW usage for some time period. Essentially testing the model and ensure it reflects reality.

As well as:

  • more precise follow-up issues created for specific conditions (mailserver, discovery, geth, bad peers, message confirmations, etc).

Solution

Start with shh client, then figure out how get them talk. Static nodes / bootstrap nodes etc. See nim-eth for additional changes (also related: vacp2p/pm#4)

Notes

See https://github.com/status-im/nim-beacon-chain for how they do it as inspiration. Start with basics though, something that can be done in a few days.

status-im/nimbus#426

Further work

After basics are done:

  • Extend with mailserver
  • Extend with better monitoring/logging (Prometheus/Grafana)
  • Extend with multiple clients (geth)
  • Extend with more adverse conditions (network drops, spikes, etc)
@oskarth oskarth added this to Triage in Waku project Dec 9, 2019
@oskarth oskarth moved this from Triage to To do in Waku project Dec 9, 2019
@oskarth oskarth self-assigned this Dec 9, 2019
@oskarth

This comment has been minimized.

Copy link
Member Author

@oskarth oskarth commented Dec 11, 2019

Basic very early WIP here in PR. Having some issues with nim-eth connectivity (though to be fair I only played with it very briefly so far). @kdeme any ideas? Based it on shh client status-im/nim-eth#145

We want this to be separate unix procs, so we can have a more realistic simulation with various components, etc. So can't take current in-proc tests as is, hence basing it on basic client

Expected result: See messages (handler log) appearing in watch process.

Actual result: Connected to Whisper nodes, messages appear to be sent and filters installed, but no messages coming through.

Note: Noticed some WRN 2019-12-11 13:46:56+08:00 Message PoW too low topics="whisper" tid=8053 minPow=0.2 pow=0.0003289473684210526, unclear if related or mainnet artifact.

@kdeme

This comment has been minimized.

Copy link

@kdeme kdeme commented Dec 11, 2019

Yes, definitely separate procs would be the way to go. Both for being able to gather metrics/bandwidth usage/memory usage/ etc. per node and to keep it understandable and scalable.

That warning is likely to be a mainnet artifact, as same clients (with same PoW requirement I assume) are used to post/subscribe. Also, I see that the postMessage call is not providing a maximum PoW calculation time, so it should normally reach the PoW target.

@oskarth

This comment has been minimized.

Copy link
Member Author

@oskarth oskarth commented Dec 12, 2019

Trying to run with https://github.com/status-im/status-go/tree/develop/_assets/systemd/bootnode and using that enode with local config I get:

INF 2019-12-12 12:50:50+08:00 Failed to bond with bootstrap nodes, retrying topics="discovery" tid=6707
TRC 2019-12-12 12:50:50+08:00 Bonding to peer topics="discovery" tid=6707 n=Node[1.171.46.175:30303]
TRC 2019-12-12 12:50:50+08:00 >>> ping topics="discovery" tid=6707 n=Node[1.171.46.175:30303]
TRC 2019-12-12 12:50:51+08:00 Bonding failed, didn't receive pong from topics="discovery" tid=6707 n=Node[1.171.46.175:30303]
INF 2019-12-12 12:50:51+08:00 Failed to bond with bootstrap nodes, retrying topics="discovery" tid=6707

The bootstrap service is running (lsof -i :30303) and nim-eth is trying to connect to that node, similar to Dockerbootnode placeholder

@kdeme

This comment has been minimized.

Copy link

@kdeme kdeme commented Dec 12, 2019

You could now use the wakunode at status-im/nimbus#437 .

It is far from finished but should be usable to start a bootnode, and several other nodes and post messages through the RPC interface (via web3, plain curl, and hopefully soon nim-web3).

I will try to add the actual Waku Mode settings soon.

@oskarth

This comment has been minimized.

Copy link
Member Author

@oskarth oskarth commented Dec 13, 2019

You can run nodes e.g. like this:
./build/wakunode --log-level:DEBUG --bootnode-only --nodekey:5dc5381cae54ba3174dc0d46040fe11614d0cc94d41185922585198b4fcef9d3
./build/wakunode --log-level:DEBUG --bootnodes:enode://e5fd642a0f630bbb1e4cd7df629d7b8b019457a9a74f983c0484a045cebb176def86a54185b50bbba6bbf97779173695e92835d63109c23471e6da382f922fdb@0.0.0.0:30303 --rpc --tcp-port:30304 --udp-port:30304
./build/wakunode --log-level:DEBUG --bootnodes:enode://e5fd642a0f630bbb1e4cd7df629d7b8b019457a9a74f983c0484a045cebb176def86a54185b50bbba6bbf97779173695e92835d63109c23471e6da382f922fdb@0.0.0.0:30303 --rpc --tcp-port:30305 --udp-port:30305 --rpc-binds:localhost:8546
And then through the RPC interfaces subscribe filters and/or post messages, etc.
You can test this just with curl or use web3 + node.js for now.
some test code I still had laying around: https://gist.github.com/kdeme/17006627154cbb796df845a0b7a14fb2
Gist
whisper web3 tests
whisper web3 tests. GitHub Gist: instantly share code, notes, and snippets.
The curl way, quite less user friendly :):
curl --header "Content-Type: application/json" -X POST --data '{"jsonrpc":"2.0","method":"shh_version","params":[],"id":1}' localhost:8545
Anyway, the idea would be that I update nim-web3 to include these JSON RPC calls. And then simulations could be written in nim I think.
https://github.com/status-im/nim-web3/
GitHub
status-im/nim-web3
Contribute to status-im/nim-web3 development by creating an account on GitHub.
Some scripting to startup the node setup (could be also via some docker images), and then some nim app to send/subscribe messages for several nodes. And then ideally some metrics enabled for monitoring also.
But other suggestions are welcome

@oskarth oskarth moved this from To do to In progress in Waku project Dec 16, 2019
@oskarth

This comment has been minimized.

Copy link
Member Author

@oskarth oskarth commented Dec 17, 2019

Brief update: Using a local network with bootstrap node etc it is running over lo, and with RPC tools like nethog (process oriented) become less useful. Worthwhile to:

  • TODO: Try running through Docker for cleaner stats
  • TODO: Get envelope counter up and running to get basic multiplier stats

See https://github.com/status-im/nimbus/pull/437/files#diff-a3fe75c397a0d5a4da33c4d6a48f75b7 for current runner code

FYI @adambabik too

@oskarth

This comment has been minimized.

Copy link
Member Author

@oskarth oskarth commented Dec 19, 2019

Brief update:

  1. Messed around a bit with Docker, but need to setup a better base image with all Waku deps etc (had some issues with Nimbus base image), then we could a similar style to https://github.com/status-im/status-protocol-go-bandwidth-test for out of band verification. Also see status-im/status-protocol-go-bandwidth-test#1 for current issue with go setup (didn't try to debug/fix)

  2. We have metrics now, thanks to @kdeme!
    image See status-im/nimbus#437 README for latest instructions.

  3. Next step is probably to add some background noise to show diff with waku-server and waku-client, as well as to script multiple nodes, etc.

  4. I'll probably not have time to look at this more before holidays. Didn't get as far as I would've liked with this but oh well. If anyone wants to pick it up please go ahead.

Addendum: Check beacon chain Dockerfile, possibly more up to date

@oskarth oskarth mentioned this issue Dec 19, 2019
4 of 4 tasks complete
@oskarth oskarth moved this from In progress to To do in Waku project Dec 19, 2019
@oskarth oskarth moved this from To do to Backlog/Scratch in Waku project Dec 19, 2019
@kdeme kdeme moved this from Backlog/Scratch to In progress in Waku project Jan 13, 2020
@kdeme

This comment has been minimized.

Copy link

@kdeme kdeme commented Jan 14, 2020

Updated status-im/nimbus#437, needs further work on network set up + traffic generation and some other items but a hardcoded (star) network can be launched and traffic can be generated by running quicksim, see readme.

@kdeme

This comment has been minimized.

Copy link

@kdeme kdeme commented Jan 30, 2020

So I think the basic version fulfils the requirements:
https://github.com/status-im/nimbus/tree/master/waku#testing-waku-protocol

Network can be started in Star, FullMesh or via discovery (v4). Configurable amount of peers can be started. Additionally a light node with using bloom filter and a light node using set topic list is started. Metrics are collected of all nodes and some are printed also on the console.

Follow-up issue(s) would contain still:

  • Extend with mailserver -> mail client not finished in nim-eth Waku, mail server not started yet
  • Extend with multiple clients (geth) + option to user Docker
  • Extend with more adverse conditions (network drops, spikes, etc)
  • Add/measure message confirmations? -> no confirmations yet in nim-eth Waku

The Waku version tested does not use latest spec yet (alist).

@oskarth

This comment has been minimized.

Copy link
Member Author

@oskarth oskarth commented Jan 30, 2020

Awesome work @kdeme, do you want to post a mini write up in Vac forum or Discuss with screenshots of the configurations and roughly what it means? Think a lot of people would be interested

@oskarth

This comment has been minimized.

Copy link
Member Author

@oskarth oskarth commented Feb 3, 2020

I'll probably include simulation numbers in upcoming write-up, so if you prefer I can do above (or we can do both)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Waku project
  
In progress
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.