New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: deploy faucet as a service #4
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Uses Packer to define a VM image which has a Rust environment pre-installed on it. Since we already had a build setup defined in Ansible, we use Ansible as the provisioner for the node. The way that Packer uses Ansible is quite strange. It runs it on the VM it spins up, rather than running it on the machine where Packer is executing. So Ansible has to be installed first. This was done using a script that uses a retry loop, since I was running into the usual problem with the Apt package manager and file locks.
The faucet is now introduced into the deployment process. Since we don't release it as a binary, we need to build it. This means the build VM is no longer an optional component. The build VM has been changed to use a droplet snapshot which already has a Rust installation on it, which significantly reduces the overall deploy time. The `build_node` role was changed to `build_safe_network_binary`, which can build any binary in the workspace and upload it to S3. We use this to build `faucet`, then use it to build `safenode` if the user has used the `--branch` and `--org` arguments. For an unknown reason there seems to be some kind of issue with systemd and the journaling service on the Ubuntu 22.10 base image. The `systemd-journald` service needs to be restarted before you see any output in the journal, and the faucet service seems to need to be restarted outside of the Ansible module before it will actually begin executing properly.
This should hopefully provide an alternative peer list on an automated basis. It sets up some infrastructure to execute arbitrary scripts against remote hosts. The easiest way to do it without faffing around with shell piping stuff was to have two steps: scp the script to the remote host, then execute execute it via ssh. This should hopefully be establish the mechanism for running any scripts against remote hosts.
jacderida
force-pushed
the
faucet-deployment
branch
from
August 19, 2023 00:46
99b4107
to
4fcd5a0
Compare
RolandSherwin
approved these changes
Aug 21, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was able to deploy a testnet with faucet and was able to get tokens from it!
PR looks good to me! 🚀
There seems to be some intermittent problems at the moment with connectivity between DO machines and the apt repositories. Here we introduce a node image that has the general `apt-get update` done in advance, as well as having several packages installed. It also installs logstash and the AWS plugin, which should significantly reduce deployment time. Also increase the SSH timeout values from 5 to 30 seconds and set Ansible forks to 20. This was fine to be set to 50 on Linux, but didn't seem to work very well on macOS if it was 30 or above. Trying 20. Might need to vary this value on the command line.
jacderida
force-pushed
the
faucet-deployment
branch
from
August 24, 2023 14:06
242d899
to
a77f336
Compare
joshuef
approved these changes
Aug 24, 2023
Make a couple of trivial changes based on PR feedback. Also attempting to force `reqwest` to the latest version to see if it resolves a CI failure with `cargo-deny`.
Rather than downloading the RPC client via HTTPs, it is retrieved directly from S3 using the AWS SDK. This was done because a problem showed up in CI with the `reqwests` library being rejected by `cargo-deny`, due to a security vulnerability in the dependency chain. It turned out that I had actually already used the AWS SDK on another branch anyway, for retrieving the logs from S3. So I was able to do this refactor and remove the dependency on `reqwest`.
jacderida
force-pushed
the
faucet-deployment
branch
from
August 25, 2023 01:14
f8f2a6c
to
051433c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
890bdf2 chore: provide image definition for a build machine
Uses Packer to define a VM image which has a Rust environment pre-installed on it.
Since we already had a build setup defined in Ansible, we use Ansible as the provisioner for the
node. The way that Packer uses Ansible is quite strange. It runs it on the VM it spins up, rather
than running it on the machine where Packer is executing. So Ansible has to be installed first. This
was done using a script that uses a retry loop, since I was running into the usual problem with the
Apt package manager and file locks.
46494ad feat: deploy faucet as a service
The faucet is now introduced into the deployment process.
Since we don't release it as a binary, we need to build it. This means the build VM is no longer an
optional component. The build VM has been changed to use a droplet snapshot which already has a Rust
installation on it, which significantly reduces the overall deploy time.
The
build_node
role was changed tobuild_safe_network_binary
, which can build any binary in theworkspace and upload it to S3. We use this to build
faucet
, then use it to buildsafenode
if theuser has used the
--branch
and--org
arguments.For an unknown reason there seems to be some kind of issue with systemd and the journaling service
on the Ubuntu 22.10 base image. The
systemd-journald
service needs to be restarted before you seeany output in the journal, and the faucet service seems to need to be restarted outside of the
Ansible module before it will actually begin executing properly.
4fcd5a0 feat: extend inventory report for sample peers
This should hopefully provide an alternative peer list on an automated basis.
It sets up some infrastructure to execute arbitrary scripts against remote hosts. The easiest way to
do it without faffing around with shell piping stuff was to have two steps: scp the script to the remote
host, then execute execute it via ssh.
This should hopefully be establish the mechanism for running any scripts against remote hosts.
a77f336 feat: use base image for nodes
There seems to be some intermittent problems at the moment with connectivity between DO machines and
the apt repositories. Here we introduce a node image that has the general
apt-get update
done inadvance, as well as having several packages installed.
It also installs logstash and the AWS plugin, which should significantly reduce deployment time.
Also increase the SSH timeout values from 5 to 30 seconds and set Ansible forks to 20. This was fine
to be set to 50 on Linux, but didn't seem to work very well on macOS if it was 30 or above. Trying
20. Might need to vary this value on the command line.
00a3129 chore: respond to feedback
Make a couple of trivial changes based on PR feedback.
Also attempting to force
reqwest
to the latest version to see if it resolves a CI failure withcargo-deny
.051433c refactor: download rpc client direct from S3
Rather than downloading the RPC client via HTTPs, it is retrieved directly from S3 using the AWS
SDK.
This was done because a problem showed up in CI with the
reqwests
library being rejected bycargo-deny
, due to a security vulnerability in the dependency chain. It turned out that I hadactually already used the AWS SDK on another branch anyway, for retrieving the logs from S3. So I
was able to do this refactor and remove the dependency on
reqwest
.