Generate MobyMask watcher using codegen

[nikugogoi]

Part of #127

• Generated storage mode watcher
• Update codegen to support indexed reference type event arguments #136

[avivash]

@nikugogoi thanks for generating this watcher!

I've got a question though...(@ashwinphatak may also be able to answer this) when running yarn server in either this watcher or the eden-watcher I get this error:

I assume it's in reference to either the getBlocks or getFullBlocks calls in the utils package? Is this a known issue or am I missing a step in my setup? (I'm following the instructions in here). Thanks 🙏🏼

[nikugogoi]

• Which release version of ipld-eth-server are you using?
• Could you also paste the watcher config YAML.

[avivash]

• Which release version of ipld-eth-server are you using?

Oh I didn't realize I was supposed to have ipld-eth-server running. I'll work on getting that running(is there a particular version I should be using?)

• Could you also paste the watcher config YAML.

I was simply using the moby-mask-watcher generated on this branch, so I didn't have my own config YAML

[avivash]

Okay, i am running ipld-eth-server 2.0.0, but still seeing the same error. I've tried using both the moby-mask-watcher/environments/local.toml and sample toml file here as the config file for ipld-eth-server, is that the correct approach? Thanks

[avivash]

@nikugogoi Ian just showed me the stack-orchestrator. Should I just run that before running yarn server in the watcher directory or is there a more specific orchestrator for running watchers that I should be using?

[nikugogoi]

@nikugogoi Ian just showed me the stack-orchestrator. Should I just run that before running yarn server in the watcher directory or is there a more specific orchestrator for running watchers that I should be using?

@avivash changes have been pushed to stack-orchestrator and you can use the develop branch to run the services for watcher.
There is also a demo for erc721-watcher with the stack-orchestrator which you can follow.

[avivash]

@nikugogoi Ian just showed me the stack-orchestrator. Should I just run that before running yarn server in the watcher directory or is there a more specific orchestrator for running watchers that I should be using?

@avivash changes have been pushed to stack-orchestrator and you can use the develop branch to run the services for watcher. There is also a demo for erc721-watcher with the stack-orchestrator which you can follow.

@nikugogoi perfect, I'll take a look. Thanks!

[avivash]

So when I follow the demo instructions while using the develop branch of stack-orchestrator this health check fails

after running this:

./wrapper.sh -f true \
  -m true \
  -s v4 \
  -l latest \
  -v remove \
  -p ../config.sh

I'm not sure if you've come across that before? I do have a call with Ian this Friday, so he's going to take a look at my stack too 👍🏼

[nikugogoi]

@abdulrabbani00 could you take a look at the failing health check?

[abdulrabbani00]

@nikugogoi and @avivash - Can you please paste your config file?

The health check will fail if docker is unable to successfully perform nc -v localhost 5432. @avivash, can you please try to increase the number of retries, intervals, and/or timeouts. It seems like it might take the DB more or less time to start depending on your local dev machine.

[avivash]

@nikugogoi and @avivash - Can you please paste your config file?

The health check will fail if docker is unable to successfully perform nc -v localhost 5432. @avivash, can you please try to increase the number of retries, intervals, and/or timeouts. It seems like it might take the DB more or less time to start depending on your local dev machine.

Thanks @abdulrabbani00, I will try that out 👍🏼 .

here is my config.sh:

#!/bin/bash
# This script is simply a configuration file that will be loaded by other files.
# It allows users to specify where related directories are instead of having to hardcode them.

vulcanize_repo_base_dir=/Users/andrewvivash/vulcanize
vulcanize_ops=/Users/andrewvivash/vulcanize/ops
vulcanize_stack_orchestrator=/Users/andrewvivash/vulcanize/stack-orchestrator
vulcanize_ipld_eth_db=/Users/andrewvivash/vulcanize/ipld-eth-db
vulcanize_ipld_eth_beacon_indexer=/Users/andrewvivash/vulcanize/ipld-eth-beacon-indexer
vulcanize_go_ethereum=/Users/andrewvivash/vulcanize/go-ethereum
vulcanize_ipld_eth_server=/Users/andrewvivash/vulcanize/ipld-eth-server
vulcanize_ipld_eth_beacon_db=/Users/andrewvivash/vulcanize/ipld-eth-beacon-db
vulcanize_eth_statediff_fill_service=/Users/andrewvivash/vulcanize/eth-statediff-fill-service
vulcanize_test_contract=/Users/andrewvivash/vulcanize/ipld-eth-db-validator/test/contract

# USE SINGLE QUOTES ONLY!!!!!!
genesis_file_path='start-up-files/go-ethereum/genesis.json'
extra_args='--metrics --metrics.expensive --metrics.addr 0.0.0.0 --metrics.port 6060'
db_write=true
eth_forward_eth_calls=false
eth_proxy_on_error=false
eth_http_path=''

watched_address_gap_filler_enabled=false
watched_address_gap_filler_interval=5

eth_beacon_capture_mode=boot
# provide only the file name under the ipld-eth-beacon-indexer/ repo.
eth_beacon_config_file=/Users/andrewvivash/vulcanize/ipld-eth-beacon-indexer/config/cicd/boot.ipld-eth-beacon-indexer.json

[abdulrabbani00]

@avivash - Thanks, please let me know if the interval changes help, if not then it might be interesting to remove the health check entirely and see how long it takes for the DB to actually be reachable.

[avivash]

@abdulrabbani00 Thanks, I just increased the retries and got past the previous failing health check, but it seems another one is failing in go-ethereum:

[abdulrabbani00]

@avivash - I believe I know the culprit, I will update the README to reflect the changes if my hunch is correct.

Can you please do the following:

cd /Users/andrewvivash/vulcanize/go-ethereum
git checkout -b v1.10.19-statediff-v4

cd /Users/andrewvivash/vulcanize/eth-statediff-fill-service
git checkout -b sharding

cd /Users/andrewvivash/vulcanize/ipld-eth-server
git checkout -b sharding

These local repositories need the latest changes for the v4 database. Let me know if this works, sorry for the inconvenience.

[avivash]

Thanks for the quick reply @abdulrabbani00. I've just tried switching to those branches but i seem to be getting the same error:

[abdulrabbani00]

@avivash - Could you please send the entire log output for latest-go-ethreum-1's docker container.... I would be curious to see if this error has to do with any docker specific settings or errors.

[avivash]

@abdulrabbani00 thanks. i've just included the entire output of

 ./wrapper.sh -f true \
  -m true \
  -s v4 \
  -l latest \
  -v remove \
  -p ../config.sh

output.txt

[abdulrabbani00]

@avivash - Can you update the following file: docker/local/docker-compose-go-ethereum.yml

USE_GENESIS: "false"

Although I should mention, with this option blocks will not be periodically mined. I can take a look at fixing the bug that caused this, and more than likely provide the desired outcome. If not, @prathamesh0, could you check to see why the genesis file is not being used as expected.

[avivash]

@abdulrabbani00 it seems i still get the same error, unfortunately 🤔

[abdulrabbani00]

If this is urgent @avivash, I can spend some time trying to debug it, but I have a hunch that @prathamesh0 will have a quick fix. If this is something that can wait until tomorrow then I'll defer it to him, if not let me know and I will try to debug.

[avivash]

If this is urgent @avivash, I can spend some time trying to debug it, but I have a hunch that @prathamesh0 will have a quick fix. If this is something that can wait until tomorrow then I'll defer it to him, if not let me know and I will try to debug.

I think tomorrow is okay 👍🏼 thanks for taking a look today. The main goal is to get this running so I'm able to integrate it into https://github.com/danfinlay/MobyMask by the end of next week

[abdulrabbani00]

@avivash - Thank you, and I appreciate your patience.

@prathamesh0 - Can you take a look, after pulling the latest changes from develop I noticed I was unable to build the go-ethereum container. I was however, able to build it when I didnt use the genesis file. Please let me know if you need any assistance.

[abdulrabbani00]

@avivash - I stand corrected, after double-checking all of the dependent repositories and ensuring they are on the right branch, everything seems. to work as expected. I would ask that you do the following:

1. Ensure that you have checked out on all the correct branches:

cd /Users/andrewvivash/vulcanize/go-ethereum
git checkout -b v1.10.19-statediff-v4

cd /Users/andrewvivash/vulcanize/eth-statediff-fill-service
git checkout -b sharding

cd /Users/andrewvivash/vulcanize/ipld-eth-server
git checkout -b sharding

2. Ensure that there are no orphaned containers running and that ports 8545 and 8546 are not occupied by another process on your machine.
3. Check to make sure the file path for the genesis file exists: start-up-files/go-ethereum/genesis.json.

If these steps still fail it might be easier to get on a quick call and try to debug. Let me know.

[prathamesh0]

@avivash @abdulrabbani00

• I tried out the setup locally. Initially I got ERROR: for contract Container "db782bcbddee" is unhealthy. where db782bcbddee was the container id for go-ethereum service. Got past this by increasing the number of healthcheck.retries (5 -> 10) in docker/local/docker-compose-go-ethereum.yml as @avivash did, after which all the services seemed to have started.

• I did not encounter container for service "go-ethereum" is unhealthy.
  From console logs (output.txt) by @avivash I can see that forge test --fork-url http://localhost:8545 is failing. This tells me that go-ethereum service is having some trouble starting; which is also why it's health check seems to be eventually failing. What would help here is logs of the go-ethereum's container after the error is encountered.
  Use docker logs to get those:

  # Get the container id for latest_go-ethereum from output of:
  docker ps
  # Check logs for latest_go-ethereum:
  docker logs -f <CONTAINER ID>

• Some misc. points:

  • Checkout to correct tags/branches where local repos are used:

    # go-ethereum
    git fetch origin && git checkout v1.10.19-statediff-4.0.3-alpha
    
    # eth-statediff-fill-service
    git fetch origin && git checkout v4.0.2-alpha
    
    # ipld-eth-db-validator
    git fetch origin && git checkout v4.0.2-alpha

  • Good practice to stop all containers before retrying as they are aren't stopped automatically on error:

    docker-compose -f ../docker/latest/docker-compose-db-sharding.yml down -v --remove-orphans

[avivash]

Thanks @abdulrabbani00 and @prathamesh0, I've tried those tips(clearing those ports, pruning docker, switching branches), but still get the same error. Here is the output of my latest_go-ethereum container:
output-go-ethereum.txt

[abdulrabbani00]

Hey, @avivash, wanna jump on a quick call and see if we can't get to the bottom of this? Ill send out an invite

[avivash]

Hey @abdulrabbani00, I'm in a meeting right now, but I have a call with Ian in 30 minutes(10am PDT) where we're going to try to figure this out if you'd like to join that?

[abdulrabbani00]

@avivash - I can stay on for like 10 minutes before I have to leave.

[avivash]

@abdulrabbani00 okay that sounds good thanks, hopefully it's just a small change that needs to be made

[abdulrabbani00]

@ashwinphatak , @i-norden , @prathamesh0 , @avivash , @nikugogoi - Andrew and I figured out the issue. It has nothing to do with any of the scripts/start up processes. The issue arises because Andrew is running ARM (m1 chip) and geth was not compiling in the correct format. As of right now our entire architecture is based on AMD64.

To work around this, Andrew installed geth from the github release, skipped geth-build, and everything worked properly. I just wanted to make everyone aware of this as I am sure it will come up. I will also create a ticket for adapting the architecture to support both AMD and ARM.

[avivash]

Really appreciate all the help today! I'll add those notes we discussed to the stack-orchestrator readme 🙏🏼