Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisions for setting up a full node #232

Closed
wants to merge 33 commits into from

Conversation

zachfc
Copy link
Contributor

@zachfc zachfc commented Jul 2, 2024

Site: https://docs.dydx.exchange/infrastructure_providers-validators/how_to_set_up_full_node
GitHub: https://github.com/dydxprotocol/v4-documentation/blob/main/pages/infrastructure_providers-validators/how_to_set_up_full_node.md

Revising this page for clarity -- since I've been here there's been some confusion from customers about this and the following topic, Running a Full Node. This rewrite:

  • adds some context to each procedure
  • reorganizes content (especially procedure steps) for flow
  • attempts to make examples more digestible
  • rewords anything that could be ambiguous

I called out a couple places to check for accuracy in particular, but really the whole thing needs to be vetted by an engineer. If something is even a little confusing/ seems off to you, please call it out

Copy link

vercel bot commented Jul 2, 2024

Someone is attempting to deploy a commit to the dYdX Trading Team on Vercel.

A member of the Team first needs to authorize it.


## Prerequisites
- Linux (Ubuntu Server 22.04.3 or later recommended)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Linux the only OS this works on? Asking because there's a .zip and .tar.gz source code download

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope you can run this on any ARM or AMD64 servers, which should cover newer Macs and AMD based windows too

## Prerequisites
- Linux (Ubuntu Server 22.04.3 or later recommended)
- 8-core CPU (ARM or x86_64)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "8-core" accurate? I assumed this is what was meant by 8-cpu.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep thats right

# Set Up a Full Node

> Examples on this page are for mainnet deployments by dYdX token holders. For information on alternative deployment types, including testnet deployments in the United States, see the [Network Constants page](../infrastructure_providers-network/network_constants.mdx).
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check for accuracy

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works for now but unfortunately the deployment information is kind of spread out between documents. Like for example, the resources page has some info too


## Download the `dydxprotocold` binary and initialize the data directory
The binary contains the software you need to operate a full node. Initializing a data directory dedicates a folder on your system to recordkeeping, such as logging.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check for accuracy

@zachfc zachfc changed the title Docs: Revisions for "Set Up a Full Node" Revisions for setting up a full node Jul 2, 2024

## Fetch and install the latest `genesis.json` file
The `genesis.json` file defines an initial state for the dYdX chain.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check for accuracy

```

When you start your full node it will automatically use the snapshot you saved to its data directory.
Copy link
Contributor Author

@zachfc zachfc Jul 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check for accuracy
Let me know if I can be more specific than "uses"

Copy link

vercel bot commented Jul 2, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
v4-teacher ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 3, 2024 3:26pm


## Prerequisites
- Linux (Ubuntu Server 22.04.3 or later recommended)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope you can run this on any ARM or AMD64 servers, which should cover newer Macs and AMD based windows too

# Set Up a Full Node

> Examples on this page are for mainnet deployments by dYdX token holders. For information on alternative deployment types, including testnet deployments in the United States, see the [Network Constants page](../infrastructure_providers-network/network_constants.mdx).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works for now but unfortunately the deployment information is kind of spread out between documents. Like for example, the resources page has some info too

## Prerequisites
- Linux (Ubuntu Server 22.04.3 or later recommended)
- 8-core CPU (ARM or x86_64)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep thats right

1. Find the latest `Release protocol` from the [v4 Chain Releases](https://github.com/dydxprotocol/v4-chain/releases/) page. Download the compressed source code file appropriate for your system.

For example, to use `Release protocol/v5.0.5` on Ubuntu, download `Source code.tar.gz`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm this might be off. We want them to download the dydxprotocold-xyz.tar.gz files (example) rather than the source code

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

The binary contains the software you need to operate a full node. Initializing a data directory dedicates a folder on your system to recordkeeping, such as logging.

1. Find the latest `Release protocol` from the [v4 Chain Releases](https://github.com/dydxprotocol/v4-chain/releases/) page. Download the compressed source code file appropriate for your system.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it would be good to add some command line instructions on how to do this too.

https://gist.github.com/steinwaywhw/a4cd19cda655b8249d908261a62687f8?permalink_comment_id=3924479

1. Find the latest `Release protocol` from the [v4 Chain Releases](https://github.com/dydxprotocol/v4-chain/releases/) page. Download the compressed source code file appropriate for your system.

For example, to use `Release protocol/v5.0.5` on Ubuntu, download `Source code.tar.gz`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

For example, to use `Release protocol/v5.0.5` on Ubuntu, download `Source code.tar.gz`.

2. Extract and rename the binary from `Source code` to `dydxprotocold`. Move the file to a directory in your `$PATH` so that you can initialize it with your command line.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, i think we should try to paste in snippets of code that basically does the step whenever possible.

  1. The renaming part might be necessary, but the "move file" is prob not necessary.
  2. Alternatively, you can create a symbolic link instead of renaming:
ln -s /path/to/target my_link
  1. Also, do we define $PATH somewhere? First time seeing it on this page

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that defining $PATH should be out of scope for this. I think it falls into general developer knowledge/ not dYdX domain/ something users should Google. In my mind the same goes for things like curl, "command line", RAM, NVME storage

Happy to discuss, and defer to you if you think the audience needs guidance on adding things to their PATH

```

When you have confirmed that your full node is up to date with the rest of the dYdX
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should prob add some section or page around when the node needs to be upgraded to the new binary.

This should prob be another PR tho


```bash
nohup dydxprotocold start
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need nohup?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, I kept it from the existing content. I'll remove it for now because more straightforward examples are better and it sounds unnecessary

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

--home=$DYDX_HOME
--non-validating-full-node=true
> /tmp/fullnode.log 2>&1 &
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to pipe the logs here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't -- I'll remove it so the example is simpler


# Example values
FULL_NODE_IP_ADDRESS=192.168.0.150
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where do you find this info?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One way on Mac is runningipconfig getifaddr en0 (wifi) or ipconfig getifaddr en1 (ethernet)

I'm tempting to say this should be out of scope, too. There are a lot of ways to get your IP and a few factors for whether a command works or not, but at the end of the day most developers will have a way to get the one they want to use (probably by googling it), whether it's for a VM or their physical machine

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha, i think we should skip

zachfc and others added 9 commits July 8, 2024 11:25
…ode.md

Co-authored-by: ttl33 <19664986+ttl33@users.noreply.github.com>
…ode.md

Co-authored-by: ttl33 <19664986+ttl33@users.noreply.github.com>
…ode.md

Co-authored-by: ttl33 <19664986+ttl33@users.noreply.github.com>
…ode.md

Co-authored-by: ttl33 <19664986+ttl33@users.noreply.github.com>
…ode.md

Co-authored-by: ttl33 <19664986+ttl33@users.noreply.github.com>
Zachary added 5 commits July 8, 2024 14:01
@zachfc
Copy link
Contributor Author

zachfc commented Jul 8, 2024

Hey @ttl33, could you take another look at this whenever you have time? Implemented most of your suggestions -- the things I have not addressed:

  • I left a comment on 2 things that I think we should not add (PATH and IP info)
  • I'll need to find out about that REST endpoint/ other way to get the live chain version. The callouts are there, will just need to add more detail in the step

Also, I'll use this PR to merge: #236
Signed my commits on that one, this one is unverified

> For example, for protocol version 5.0.5 on an AMD system, download `dydxprotocold-v5.0.5-linux-amd64.tar.gz`.

Alternatively, you can install the latest release protocol by modifying [this curl command](https://gist.github.com/steinwaywhw/a4cd19cda655b8249d908261a62687f8) for your case.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of pointing to another github gist, I recommend pasting in the actual curl command that the user can copy/paste to download dydxprotocold-v5.0.5-linux-amd64.tar.gz

IMO, we should minimize the number of redirects to other sites

The `dydxprotocold` binary contains the software you need to operate a full node. You must use the same version of the software as the network to which you want to connect.

1. Find the current version of the dYdX network.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we explain how to figure this out? IMO, there's a pretty good chance users on this site don't know how to do this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely -- I don't know how to do it and need to find out. Do you know what the best way or best options are?


**Option 1:** Rename the file. Edit the filename from `dydxprotocold-<version>-<architecture>` to simply `dydxprotocold`. You might need to add your current directory to your $PATH or move the file to a directory in your $PATH.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add the example command like the following?

mv dydxprotocold-v5.0.6-linux-arm64 dydxprotocold

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'll add this and start doing that with each step that calls for it

I do think using example values is clearer, like mv dydxprotocold-<version>-<architecture> dydxprotocold. They call attention to custom attributes and stop the user from trying to run a command as-is

**Option 2:** Create a symbolic link to the file using the name `dydxprotocold`. Run the following command:
```bash
ln -s /path/to/your/binary dydxprotocold
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we continue using the same example like this?

Suggested change
ln -s /path/to/your/binary dydxprotocold
ln -s mv dydxprotocold-v5.0.6-linux-arm64 dydxprotocold

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reformatting these to both use "/path/to/dydxprotocold--" -- it's my opinion that variables/example names are clearer than "real" values, because it stands out from cases where we want them to run a command as-is and also helps users understand how things connect

Definitely debatable and we should apply the same formatting to all examples when we decide, so happy to discuss

Using the `--home` command line flag, provide the path to your data directory.

Once you have entered your parameters, your command should resemble the following:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this example doesn't seem necessary

Run your command to start the full node.

2. Monitor your full node's progress by tailing the log (optional).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should skip instructions like > /tmp/fullnode.log 2>&1 & and tailing your logs.

i think users should be able to choose and we should be less opinionated about this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed


# Example values
FULL_NODE_IP_ADDRESS=192.168.0.150
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha, i think we should skip

3. Confirm that your full node has finished syncing by comparing its current block to the dYdX chain. The full node is caught up with the dYdX chain head when it reaches the dYdX chain's current block.

To determine your full node's current block, use a block explorer like this example on [mintscan.io](https://www.mintscan.io/dydx).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tells you the network's latest height. Not your full node's block height.

To determine your full node's current block, use a block explorer like this example on [mintscan.io](https://www.mintscan.io/dydx).

To determine the dYdX chain's current block, use the program [v4block_subscribe.py](https://github.com/chiwalfrm/dydxexamples/blob/1d46b7a75499205d9c1c1986ae4ae8f21b6c1385/v4block_subscribe.py).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how you find your full node's height. I recommend not using this script, but rather something more simple.

does this work?

curl localhost:26657/status

zachfc and others added 8 commits July 10, 2024 14:06
…ode.md

Co-authored-by: ttl33 <19664986+ttl33@users.noreply.github.com>
…ode.md

Co-authored-by: ttl33 <19664986+ttl33@users.noreply.github.com>
…ode.md

Co-authored-by: ttl33 <19664986+ttl33@users.noreply.github.com>
…ode.md

Co-authored-by: ttl33 <19664986+ttl33@users.noreply.github.com>
@zachfc
Copy link
Contributor Author

zachfc commented Jul 11, 2024

Hey @ttl33, I've made most of these updates as well. However, Mohammed let me know that he pushed significant updates to this page today -- it includes a ton of good info that I don't have, so I'm now incorporating these additions in my docs. I'll push up a pretty new/reworked page in this PR: #236

@zachfc
Copy link
Contributor Author

zachfc commented Jul 19, 2024

Closing this PR, comments will be addressed in a rewrite of this page, PR #241

@zachfc zachfc closed this Jul 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants