Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

P 419 deploy automation #2418

Merged
merged 39 commits into from
Feb 6, 2024
Merged

P 419 deploy automation #2418

merged 39 commits into from
Feb 6, 2024

Conversation

BillyWooo
Copy link
Collaborator

@BillyWooo BillyWooo commented Jan 23, 2024

The major task is to provide a deployment script for production working mode.

  • The whole deployment doesn't depend on source repo anymore.
  • The script will
    • download all required release packages,
    • prepare the working env and
    • start the worker service.

It requires some configs/keys files to be prepared and kept locally on the working server.
Try to explore the help message for more details.

Old description is deprecated.
Auto deploy script moved into private repo.
Here is only some minor change/fix.

@BillyWooo BillyWooo requested a review from a team January 23, 2024 23:23
Copy link

linear bot commented Jan 23, 2024

SGX_SDK=/opt/intel/sgxsdk
SGX_ENCLAVE_SIGNER=$SGX_SDK/bin/x64/sgx_sign

VERSION_PATTERN="^p[0-9]+\.[0-9]+\.[0-9]+-[0-9]+-w[0-9]+\.[0-9]+\.[0-9]+-[0-9]+$"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems that this pattern supports only versions with patch version like p0.9.17-9172-w0.0.1-102 , should we also support something like p0.9.18-w0.0.2 in case we have major release?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say we will have p0.9.18-9180-w0.0.2-100.

Copy link
Contributor

@zhouhuitian zhouhuitian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a shellcheck command, but I don't know if it has been executed before.

Comment on lines 138 to 143
if sudo -l -U $USER 2>/dev/null | grep -q 'may run the following'; then
source "$SGX_SDK/environment"
else
echo "$USER doesn't have sudo permission"
exit 1
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also can use this command to determine whether the current user is sudo

if [ $(id -u) -eq 0 ]; then
    echo "sudo"
else
    echo "Doesn't have sudo permission"
fi

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That won't work though

root has uid 0, but non-zero uid doesn't mean it can't sudo. In fact, the code will always land in else branch unless executed with root.

Comment on lines 146 to 151
sudo mkdir -p "$BASEDIR"
sudo chown -R $USER:$GROUPS "$BASEDIR"
for d in "$LOG_BACKUP_BASEDIR" "$WORKER_BACKUP_BASEDIR" "$RELAYCHAIN_ALICE_BASEDIR" "$RELAYCHAIN_BOB_BASEDIR" \
"$PARACHAIN_ALICE_BASEDIR" "$WORKER_BASEDIR"; do
mkdir -p "$d"
done
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another version:

folders=("folder1" "folder2" "folder3" "folder4")
for folder in "${folders[@]}"
do
    if [ ! -d "$folder" ]; then
        mkdir "$folder"
    else
        echo "exists"
    fi
done

fi
}

function restart_services {
sudo systemctl daemon-reload
restart_parachain_services
restart_worker_services
echo "Done"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

echo "The service has been successfully restarted."

for ((i = 0; i < $WORKER_COUNT; i++)); do
worker_dir="$WORKER_BASEDIR/w$i"
mkdir -p "$worker_dir"
for f in 'key.txt' 'spid.txt' 'enclave.signed.so' 'litentry-worker'; do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, if there are more than two, put them into an array for iteration.

Copy link
Collaborator

@Kailai-Wang Kailai-Wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine! Some small comments

@@ -30,7 +30,7 @@ DISCORD_AUTH_TOKEN=
ACHAINABLE_AUTH_KEY=
ONEBLOCK_NOTION_KEY=
NODEREAL_API_KEY=
GENIIDATA_API_KEY=142cf1b0-1ca7-11ee-bb5e-9d74c2e854ac
GENIIDATA_API_KEY=
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it affect any of our tests?
I remember @kziemianek mentioned this is the public key

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it won't affect any tests. The key is always overwritten.

@@ -156,10 +157,16 @@ def offset_port(offset):


def setup_environment(offset, config, parachain_dir):
if not os.path.isfile("./local-setup/.env"):
shutil.copy("./local-setup/.env.dev", "./local-setup/.env")

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed?
I see the following line is

load_dotenv("./local-setup/.env.dev")

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks. fixed.

@BillyWooo BillyWooo merged commit 94bd3a7 into dev Feb 6, 2024
10 checks passed
@BillyWooo BillyWooo deleted the p-419-deploy-automation branch February 6, 2024 08:22
BillyWooo added a commit that referenced this pull request Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants