Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: deploy faucet as a service #4

Merged
merged 6 commits into from Aug 25, 2023

Conversation

jacderida
Copy link
Contributor

@jacderida jacderida commented Aug 19, 2023

  • 890bdf2 chore: provide image definition for a build machine

    Uses Packer to define a VM image which has a Rust environment pre-installed on it.

    Since we already had a build setup defined in Ansible, we use Ansible as the provisioner for the
    node. The way that Packer uses Ansible is quite strange. It runs it on the VM it spins up, rather
    than running it on the machine where Packer is executing. So Ansible has to be installed first. This
    was done using a script that uses a retry loop, since I was running into the usual problem with the
    Apt package manager and file locks.

  • 46494ad feat: deploy faucet as a service

    The faucet is now introduced into the deployment process.

    Since we don't release it as a binary, we need to build it. This means the build VM is no longer an
    optional component. The build VM has been changed to use a droplet snapshot which already has a Rust
    installation on it, which significantly reduces the overall deploy time.

    The build_node role was changed to build_safe_network_binary, which can build any binary in the
    workspace and upload it to S3. We use this to build faucet, then use it to build safenode if the
    user has used the --branch and --org arguments.

    For an unknown reason there seems to be some kind of issue with systemd and the journaling service
    on the Ubuntu 22.10 base image. The systemd-journald service needs to be restarted before you see
    any output in the journal, and the faucet service seems to need to be restarted outside of the
    Ansible module before it will actually begin executing properly.

  • 4fcd5a0 feat: extend inventory report for sample peers

    This should hopefully provide an alternative peer list on an automated basis.

    It sets up some infrastructure to execute arbitrary scripts against remote hosts. The easiest way to
    do it without faffing around with shell piping stuff was to have two steps: scp the script to the remote
    host, then execute execute it via ssh.

    This should hopefully be establish the mechanism for running any scripts against remote hosts.

  • a77f336 feat: use base image for nodes

    There seems to be some intermittent problems at the moment with connectivity between DO machines and
    the apt repositories. Here we introduce a node image that has the general apt-get update done in
    advance, as well as having several packages installed.

    It also installs logstash and the AWS plugin, which should significantly reduce deployment time.

    Also increase the SSH timeout values from 5 to 30 seconds and set Ansible forks to 20. This was fine
    to be set to 50 on Linux, but didn't seem to work very well on macOS if it was 30 or above. Trying
    20. Might need to vary this value on the command line.

  • 00a3129 chore: respond to feedback

    Make a couple of trivial changes based on PR feedback.

    Also attempting to force reqwest to the latest version to see if it resolves a CI failure with
    cargo-deny.

  • 051433c refactor: download rpc client direct from S3

    Rather than downloading the RPC client via HTTPs, it is retrieved directly from S3 using the AWS
    SDK.

    This was done because a problem showed up in CI with the reqwests library being rejected by
    cargo-deny, due to a security vulnerability in the dependency chain. It turned out that I had
    actually already used the AWS SDK on another branch anyway, for retrieving the logs from S3. So I
    was able to do this refactor and remove the dependency on reqwest.

Uses Packer to define a VM image which has a Rust environment pre-installed on it.

Since we already had a build setup defined in Ansible, we use Ansible as the provisioner for the
node. The way that Packer uses Ansible is quite strange. It runs it on the VM it spins up, rather
than running it on the machine where Packer is executing. So Ansible has to be installed first. This
was done using a script that uses a retry loop, since I was running into the usual problem with the
Apt package manager and file locks.
The faucet is now introduced into the deployment process.

Since we don't release it as a binary, we need to build it. This means the build VM is no longer an
optional component. The build VM has been changed to use a droplet snapshot which already has a Rust
installation on it, which significantly reduces the overall deploy time.

The `build_node` role was changed to `build_safe_network_binary`, which can build any binary in the
workspace and upload it to S3. We use this to build `faucet`, then use it to build `safenode` if the
user has used the `--branch` and `--org` arguments.

For an unknown reason there seems to be some kind of issue with systemd and the journaling service
on the Ubuntu 22.10 base image. The `systemd-journald` service needs to be restarted before you see
any output in the journal, and the faucet service seems to need to be restarted outside of the
Ansible module before it will actually begin executing properly.
This should hopefully provide an alternative peer list on an automated basis.

It sets up some infrastructure to execute arbitrary scripts against remote hosts. The easiest way to
do it without faffing around with shell piping stuff was to have two steps: scp the script to the remote
host, then execute execute it via ssh.

This should hopefully be establish the mechanism for running any scripts against remote hosts.
Copy link
Member

@RolandSherwin RolandSherwin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to deploy a testnet with faucet and was able to get tokens from it!
PR looks good to me! 🚀

README.md Outdated Show resolved Hide resolved
There seems to be some intermittent problems at the moment with connectivity between DO machines and
the apt repositories. Here we introduce a node image that has the general `apt-get update` done in
advance, as well as having several packages installed.

It also installs logstash and the AWS plugin, which should significantly reduce deployment time.

Also increase the SSH timeout values from 5 to 30 seconds and set Ansible forks to 20. This was fine
to be set to 50 on Linux, but didn't seem to work very well on macOS if it was 30 or above. Trying
20. Might need to vary this value on the command line.
Make a couple of trivial changes based on PR feedback.

Also attempting to force `reqwest` to the latest version to see if it resolves a CI failure with
`cargo-deny`.
Rather than downloading the RPC client via HTTPs, it is retrieved directly from S3 using the AWS
SDK.

This was done because a problem showed up in CI with the `reqwests` library being rejected by
`cargo-deny`, due to a security vulnerability in the dependency chain. It turned out that I had
actually already used the AWS SDK on another branch anyway, for retrieving the logs from S3. So I
was able to do this refactor and remove the dependency on `reqwest`.
@jacderida jacderida merged commit 5ef7168 into maidsafe:main Aug 25, 2023
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants