-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Redesign Build Deployment Process (External) #125
base: main
Are you sure you want to change the base?
Changes from all commits
6aed57a
8b0317f
7d3cd86
355db20
3c17d18
e402a61
ffcdbc8
d6a6809
102899a
50a7b63
db85f75
2dab7a1
43f3d5a
89ed59a
37c7599
e962d13
8d2464d
d49a275
efaf216
baac678
629b3a1
fee9437
9622279
6e3f2be
1745634
21ca992
58093d3
368900b
bf3e9f7
6eebf87
e760598
9444e60
40beb80
29ebf49
1d0a937
308eca0
35e09cb
d919a34
11cdafb
1dda106
66bd0a6
426ae50
3b298fc
4335f39
b330f96
9b1dca5
ab0e157
7c78c18
27c2f42
c8b6dca
40bf355
803e9c2
7ee8e61
af3c2d3
91d5251
d3bf89f
c58b5dd
14775b5
f595b0d
2f17e5d
25dcba1
82cf4e9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
|
||
import json | ||
import logging | ||
import requests | ||
import sys | ||
|
||
if __name__ == '__main__': | ||
logging.basicConfig(level=logging.DEBUG) | ||
|
||
''' | ||
Workflow "docker image" uses image_build_push.yml | ||
From above commented out code, and checked via terminal as well, | ||
workflow id for the "docker image" can be fetched using: | ||
https://api.github.com/repos/MukuFlash03/e-mission-server/actions/workflows | ||
https://api.github.com/repos/e-mission/e-mission-server/actions/workflows | ||
|
||
For MukuFlash03: id = 75506902 | ||
For e-mission-server: id = 35580278 | ||
''' | ||
|
||
download_url = "https://api.github.com/repos/e-mission/e-mission-server/actions/workflows/35580278/runs" | ||
logging.debug("About to fetch workflow runs present in docker image workflow present in e-mission-server from %s" % download_url) | ||
r = requests.get(download_url) | ||
if r.status_code != 200: | ||
logging.debug(f"Unable to fetch workflow runs, status code: {r.status_code}") | ||
sys.exit(1) | ||
else: | ||
workflow_runs_json = json.loads(r.text) | ||
logging.debug(f"Successfully fetched workflow runs") | ||
|
||
workflow_runs = workflow_runs_json["workflow_runs"] | ||
if workflow_runs: | ||
successful_runs = [run for run in workflow_runs \ | ||
if run["status"] == "completed" and \ | ||
run["conclusion"] == "success" and \ | ||
run["head_branch"] == "master" | ||
] | ||
if successful_runs: | ||
sorted_runs = successful_runs.sort(reverse=True, key=lambda x: x["updated_at"]) | ||
sorted_runs = sorted(successful_runs, reverse=True, key=lambda x: x["updated_at"]) | ||
latest_run_id = sorted_runs[0]["id"] | ||
print(f"::set-output name=run_id::{latest_run_id}") |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,158 @@ | ||
name: docker-image-push-public-dash | ||
|
||
on: | ||
push: | ||
branches: [ main ] | ||
|
||
workflow_dispatch: | ||
inputs: | ||
docker_image_tag: | ||
description: "Latest Docker image tags passed from e-mission-server repository on image build and push" | ||
required: true | ||
|
||
env: | ||
DOCKER_USER: ${{secrets.DOCKER_USER}} | ||
DOCKER_PASSWORD: ${{secrets.DOCKER_PASSWORD}} | ||
|
||
jobs: | ||
fetch_run_id: | ||
runs-on: ubuntu-latest | ||
|
||
outputs: | ||
run_id: ${{ steps.get_run_id.outputs.run_id }} | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: Set up Python | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: '3.x' | ||
|
||
- name: Install Python dependencies | ||
run: | | ||
pip install requests | ||
|
||
- name: Run Python script | ||
id: run_script | ||
run: | | ||
echo "Fetching latest successful run ID from e-misison-server docker image workflow" | ||
python .github/fetch_runID.py | ||
|
||
- name: Get Run ID | ||
id: get_run_id | ||
run: echo "run_id=${{ steps.run_script.outputs.run_id }}" >> "$GITHUB_OUTPUT" | ||
|
||
fetch_tag: | ||
needs: fetch_run_id | ||
runs-on: ubuntu-latest | ||
|
||
env: | ||
RUN_ID: ${{ needs.fetch_run_id.outputs.run_id }} | ||
|
||
outputs: | ||
docker_image_tag: ${{ steps.get_docker_tag.outputs.docker_image_tag }} | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: Use Run ID from previous fetch_run_id job | ||
run: echo Run ID from previous job ${{ env.RUN_ID }} | ||
|
||
- name: Download artifact | ||
uses: actions/download-artifact@v4 | ||
with: | ||
# TODO: Create a token with basic repo permissions | ||
name: docker-image-tag | ||
github-token: ${{ secrets.GH_PAT_TAG }} | ||
repository: e-mission/e-mission-server | ||
run-id: ${{ env.RUN_ID }} | ||
|
||
- name: Print artifact tag | ||
id: get_docker_tag | ||
run: | | ||
cat tag_file.txt | ||
docker_image_tag=$(cat tag_file.txt) | ||
echo $docker_image_tag | ||
echo "docker_image_tag=$(echo $docker_image_tag)" >> $GITHUB_OUTPUT | ||
build: | ||
needs: fetch_tag | ||
|
||
runs-on: ubuntu-latest | ||
|
||
env: | ||
DOCKER_IMAGE_TAG_1: ${{ needs.fetch_tag.outputs.docker_image_tag }} | ||
DOCKER_IMAGE_TAG_2: ${{ github.event.inputs.docker_image_tag }} | ||
Comment on lines
+84
to
+85
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit, future fix: better names There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Clarification: Better names for the environment variables? |
||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: Print input docker image tag | ||
run: | | ||
echo "Event name: ${{ github.event_name }}" | ||
echo "Latest docker image tag (push): ${{ env.DOCKER_IMAGE_TAG_1 }}" | ||
echo "Latest docker image tag (workflow_dispatch): ${{ env.DOCKER_IMAGE_TAG_2 }}" | ||
|
||
- name: Update .env file | ||
run: | | ||
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then | ||
echo "Workflow_dispatch: New server image built and pushed, Updating image tag in .env" | ||
echo "SERVER_IMAGE_TAG=$DOCKER_IMAGE_TAG_2" > .env | ||
else | ||
echo "Push event: Restoring latest server image tag in .env" | ||
echo "SERVER_IMAGE_TAG=$DOCKER_IMAGE_TAG_1" > .env | ||
fi | ||
|
||
- name: Add, Commit, Push changes to .env file | ||
run: | | ||
git config --local user.email "action@github.com" | ||
git config --local user.name "GitHub Action" | ||
Comment on lines
+108
to
+109
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit, future fix: configure this to be more meaningful; "GitHub Action" doing what? |
||
if git diff --quiet; then | ||
echo "Latest timestamp already present in .env file, no changes to commit" | ||
else | ||
git add .env | ||
git commit -m "Updated docker image tag in .env file to the latest timestamp" | ||
git push origin | ||
fi | ||
|
||
- name: docker login | ||
run: | # log into docker hub account | ||
docker login -u $DOCKER_USER -p $DOCKER_PASSWORD | ||
|
||
- name: Get current date # get the date of the build | ||
id: date | ||
run: echo "::set-output name=date::$(date +'%Y-%m-%d--%M-%S')" | ||
|
||
- name: Run a one-line script | ||
run: echo running in repo ${GITHUB_REPOSITORY#*/} branch ${GITHUB_REF##*/} on ${{ steps.date.outputs.date }} | ||
|
||
- name: build docker image | ||
run: | | ||
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then | ||
SERVER_IMAGE_TAG=$DOCKER_IMAGE_TAG_2 docker compose -f docker-compose.prod.yml build | ||
else | ||
SERVER_IMAGE_TAG=$DOCKER_IMAGE_TAG_1 docker compose -f docker-compose.prod.yml build | ||
fi | ||
docker images | ||
|
||
- name: rename docker images | ||
run: | | ||
docker image tag em-pub-dash-prod/frontend:latest $DOCKER_USER/${GITHUB_REPOSITORY#*/}:${GITHUB_REF##*/}_${{ steps.date.outputs.date }} | ||
docker image tag em-pub-dash-prod/viz-scripts:latest $DOCKER_USER/${GITHUB_REPOSITORY#*/}_notebook:${GITHUB_REF##*/}_${{ steps.date.outputs.date }} | ||
|
||
- name: push docker images | ||
run: | | ||
docker push $DOCKER_USER/${GITHUB_REPOSITORY#*/}:${GITHUB_REF##*/}_${{ steps.date.outputs.date }} | ||
docker push $DOCKER_USER/${GITHUB_REPOSITORY#*/}_notebook:${GITHUB_REF##*/}_${{ steps.date.outputs.date }} | ||
|
||
- name: Create a text file | ||
run: | | ||
echo ${{ steps.date.outputs.date }} > public_dash_tag_file.txt | ||
echo "Created tag text file" | ||
|
||
- name: Upload Artifact | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: public-dash-image-tag | ||
path: public_dash_tag_file.txt | ||
overwrite: true |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -108,7 +108,6 @@ celerybeat.pid | |
*.sage.py | ||
|
||
# Environments | ||
.env | ||
.venv | ||
env/ | ||
venv/ | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit, future fix: why are we checking this out twice?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on this, each job will need its own checkout/environment