Complex ways to manage validator client database path

[nalepae]

Describe the bug

This ticket discuss the way how validator client database path is managed.
I'm on MacOS.

Here are the interesting corresponding lines when running: ./prysm.sh validator --help:

cmd OPTIONS:
...
--datadir value.      Data directory for the databases (default: "/Users/manu/Library/Eth2")
...

validator OPTIONS:
...
  --wallet-dir value      Path to a wallet directory on-disk for Prysm validator accounts (default: "/Users/manu/Library/Eth2Validators/prysm-wallet-v2")
...

Without specifying --datadir

1. Running this command: ./prysm.sh validator --goerli
VC DB is stored at /Users/manu/Library/Eth2Validators/prysm-wallet-v2/direct/validator.db.
==> validator.dbis stored in the default value of --wallet-dir/direct. (direct subdirectory is added.)

2. Running this command: ./prysm.sh validator --goerli --validators-external-signer-url http://localhost:9000
VC DB is stored at /Users/manu/Library/Eth2Validators/prysm-wallet-v2/validator.db.
==> validator.db is stored in the default value of --wallet-dir. (Compared to 1., there is no direct subdirectory added.)

3. Running this command: ./prysm.sh validator --goerli --wallet-dir /tmp/wd
VC DB is stored at /tmp/wd/direct/validator.db.
==> validator.db is stored in the specified value of --wallet-dir/direct (direct subdirectory is added.)

4. Running this command: ./prysm.sh validator --goerli --validators-external-signer-url http://localhost:9000 --wallet-dir /tmp/wd
VC DB is stored at /tmp/wd/validator.db.
==> validator.db is stored in the specified value of --wallet-dir. (Compared to 3., there is no direct subdirectory added.)

With specifying --datadir

1. Running this command: ./prysm.sh validator --goerli --datadir /tmp/dd
VC DB is stored at /tmp/dd/validator.db.
==> validator.dbis stored in the specified value of --datadir. (there is no direct subdirectory added.)

2. Running this command: ./prysm.sh validator --goerli --validators-external-signer-url http://localhost:9000 --datadir /tmp/dd
VC DB is stored at /tmp/dd/validator.db.
==> validator.dbis stored in the specified value of --datadir. (there is no direct subdirectory added.)

3. Running this command: ./prysm.sh validator --goerli --wallet-dir /tmp/wd --datadir /tmp/dd
VC DB is stored at /tmp/dd/validator.db.
==> validator.dbis stored in the specified value of --datadir. (there is no direct subdirectory added.) --datadir supersedes --wallet-dir.

4. Running this command: ./prysm.sh validator --goerli --validators-external-signer-url http://localhost:9000 --wallet-dir /tmp/wd
VC DB is stored at /tmp/dd/validator.db.
==> validator.dbis stored in the specified value of --datadir. (there is no direct subdirectory added.) --datadir supersedes --wallet-dir.

Summary

• Default --datadir value is never used. If no --datadir is specified, VC DB is stored in the (default or not) value of --wallet-dir. If no --validators-external-signer-url is specified, a /direct subdirectory is added.
• If --datadir is specified, the value of --wallet-dir is discarded. VC DB is always stored under the value specified under --datadir, whatever the use of --validators-external-signer-url (No direct subdirectory added).

Current issues

• In my opinion, there is no really reason to ever store the DB under --wallet-dir (default or specified value). The DB should always be stored under --datadir (default or specified value), as mentioned in the --datadir help message.
• If a user do not specify --datadir and switch the --validators-external-signer-url flag, then the previous validator.db file won't be used any more. (Because of the /direct subdirectory)
• Tools like Migrate Up, Migrate Down, EIP-3976 export or EIP-3076 import rely on --dadatir, and not on --wallet-dir

Proposed solution

• Always store the DB under the value of --datadir, and never add /direct subdirectory.

[nalepae]

Also, if --interop-num-validators is used, the behaviour differs.

[james-prysm]

reopening this issue for web3signer pathing