Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
eac8251
WIP- #1516 Add CI test to latest tag
umesh-timalsina Apr 3, 2020
395a13f
WIP- checkout tag in CI step
umesh-timalsina Apr 3, 2020
3734100
Add release tag
umesh-timalsina Apr 3, 2020
1fe3490
WIP- remove wrong - in action file
umesh-timalsina Apr 3, 2020
12bd528
Add docker release action (demo)
umesh-timalsina Apr 3, 2020
0515a1a
Fix indentation in workflow file
umesh-timalsina Apr 3, 2020
3f8633d
WIP-remove npm dry run
umesh-timalsina Apr 3, 2020
14f5a96
WIP- Add a demo release action
umesh-timalsina Apr 3, 2020
ffcc356
WIP- Change exposed port in custom-docker-compose-overrides
umesh-timalsina Apr 3, 2020
f37ea3d
WIP- Switch to branch before docker build
umesh-timalsina Apr 3, 2020
02272cc
WIP- Switch to branch before docker build
umesh-timalsina Apr 4, 2020
e1d2bd8
WIP- Fix branch switching in kitchen-sink image
umesh-timalsina Apr 4, 2020
0e00442
WIP- Add Tag arguments
umesh-timalsina Apr 4, 2020
6b832ab
WIP- Add environment tag arguments as docker build
umesh-timalsina Apr 4, 2020
8485922
Merge branch master into 1516-docker-release-action
umesh-timalsina Apr 6, 2020
6732621
WIP- Move compose customizations to a single file
umesh-timalsina Apr 6, 2020
f2e7108
WIP- fix branch checkout
umesh-timalsina Apr 6, 2020
d0006b0
WIP- Fix docker-compose command with a uniform project name across di…
umesh-timalsina Apr 7, 2020
15d27eb
WIP- Fix typo in custom overrides file
umesh-timalsina Apr 7, 2020
96f26ac
WIP- add correct mount path for the deployment script
umesh-timalsina Apr 7, 2020
1a97941
WIP- Consolidate action to a single workflow file
umesh-timalsina Apr 7, 2020
a74cee3
WIP- Rename TAG environment variables in action files
umesh-timalsina Apr 7, 2020
6e7dda9
WIP- remove branch ref and change images in the overrides file
umesh-timalsina Apr 7, 2020
0f821a4
WIP- Fix kitchensink file
umesh-timalsina Apr 7, 2020
516fdd8
merge branch master into 1516-docker-release-action
umesh-timalsina Apr 7, 2020
81c559b
merge branch master into 1516-deploy-deepforge
umesh-timalsina Apr 7, 2020
528fa76
WIP- Add project name for multidirectory consistency
umesh-timalsina Apr 7, 2020
78ed420
Manually set semver tags for release
umesh-timalsina Apr 7, 2020
7f3e8c7
WIP- remove tag_semver from github action
umesh-timalsina Apr 7, 2020
32d7506
WIP- remove spaces in tag names
umesh-timalsina Apr 7, 2020
727aa49
WIP- Fix semver regex in release docker build action
umesh-timalsina Apr 8, 2020
65492d7
Fixed issue with quotes and refactored sed cmd
brollb Apr 8, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions .deployment/deploy-deepforge
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,19 @@ export DEEPFORGE_DEPLOYMENT_DIR

. ~/.nvm/nvm.sh

SERVER_NAME=server
SERVER_NAME="server"

if [[ $1 = "stable" ]]; then
SERVER_NAME="server_stable"
fi;

# Merging the custom override yml file
yaml-merge docker/docker-compose.yml "$DEEPFORGE_DEPLOYMENT_DIR"/docker-compose-overrides.yml > custom-docker-compose.yml

# Pulling the latest docker image, stopping the server, removing and restarting it
docker-compose --file custom-docker-compose.yml pull $SERVER_NAME
docker-compose --file custom-docker-compose.yml stop $SERVER_NAME
docker-compose --file custom-docker-compose.yml rm -f $SERVER_NAME
docker-compose --file custom-docker-compose.yml up -d $SERVER_NAME
docker-compose --file custom-docker-compose.yml -p deepforge pull $SERVER_NAME
docker-compose --file custom-docker-compose.yml -p deepforge stop $SERVER_NAME
docker-compose --file custom-docker-compose.yml -p deepforge rm -f $SERVER_NAME
docker-compose --file custom-docker-compose.yml -p deepforge up -d $SERVER_NAME

docker image prune -f
1 change: 1 addition & 0 deletions .deployment/dev-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ pip uninstall tensorflow -y
conda install tensorflow==1.14 -y

deepforge start --server

16 changes: 16 additions & 0 deletions .deployment/docker-compose-overrides.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,19 @@ services:
- "${TOKEN_KEYS_DIR}:/token_keys"
- "${DEEPFORGE_DEPLOYMENT_DIR}:/.deployment"

server_stable:
entrypoint: /.deployment/dev-entrypoint.sh
environment:
- "MONGO_URI=mongodb://mongo:27017/deepforge"
- "DEEPFORGE_HOST=https://editor.deepforge.org"
- "DEEPFORGE_PUBLIC_KEY=/token_keys/public_key"
- "DEEPFORGE_PRIVATE_KEY=/token_keys/private_key"
image: deepforge/kitchen-sink:stable
ports:
- "8889:8888"
volumes:
- "$HOME/.deepforge/blob:/data/blob"
- "${TOKEN_KEYS_DIR}:/token_keys"
- "${DEEPFORGE_DEPLOYMENT_DIR}:/.deployment"
depends_on:
- mongo
43 changes: 39 additions & 4 deletions .github/workflows/docker-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,50 @@ on:
push:
branches:
- master
release:
types:
- published

jobs:
publish:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- name: Get Release Tags
id: get_release_tags
run: |
echo "::set-env name=RELEASE_TAG_NAME::$(echo ${GITHUB_REF} | sed -E 's/.*v?([0-9]+)\.([0-9]+)\.([0-9]+)?/\1.\2.\3,\1.\2,\1/')"
echo "::set-env name=TAG::stable"
if: github.event_name == 'release'

- name: Get Push Tags
id: get_push_tags
run: |
echo "::set-env name=TAG::latest"
echo "::set-env name=RELEASE_TAG::$(echo ${GITHUB_REF:11})"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm, this will get the number only (remove the leading v from the release tag), right?

As an example the release tag is v2.1.1 whereas the docker image tag is 2.1.1 for the 2.1.1 release.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it will not do that. But the action has an option to set TAG_SEMVER, which will automatically tag all semantic versions accordingly if 1.0.0 then 1, 1.0, 1.0.0 are the three tags set. The Push tag should be refs/branches/master and latest.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upon closer inspection of the action code, You can either use custom tags or semver tags, but not both.
https://github.com/elgohr/Publish-Docker-Github-Action/blob/master/entrypoint.sh#L70-L86

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it will not do that. But the action...

Wait, so this will not follow the tagging convention when building the release docker images? Could this be updated to follow the conventions?

if: github.event_name == 'push'

- name: Publish to docker hub (server)
uses: elgohr/Publish-Docker-Github-Action@master
with:
name: deepforge/server
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
tags: "latest"
dockerfile: docker/Dockerfile
tags: "${{ env.TAG }},${{ env.RELEASE_TAG }}"
tag_semver: "${{ env.TAG_SEMVER }}"

- name: Publish to docker hub (kitchen-sink)
uses: elgohr/Publish-Docker-Github-Action@master

with:
name: deepforge/kitchen-sink
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
tags: "latest"
dockerfile: docker/Dockerfile.kitchensink
tags: "${{ env.TAG }},${{ env.RELEASE_TAG }}"
dockerfile: Dockerfile.kitchensink
buildargs: "TAG=${{ env.TAG }}"

- name: Deploy (dev.deepforge.org)
uses: appleboy/ssh-action@master
Expand All @@ -40,3 +60,18 @@ jobs:
script: |
rm -rf deepforge && git clone git@github.com:deepforge-dev/deepforge && cd deepforge
chmod +x ./.deployment/deploy-deepforge && ./.deployment/deploy-deepforge
if: github.event_name == 'push'

- name: Deploy (editor.deepforge.org)
uses: appleboy/ssh-action@master
with:
username: ${{ secrets.USERNAME }}
host: ${{ secrets.HOST }}
key: ${{ secrets.KEY }}
port: ${{ secrets.PORT }}
passphrase: ${{ secrets.PASSPHRASE }}
script: |
rm -rf deepforge-stable && git clone git@github.com:deepforge-dev/deepforge-stable && cd deepforge-stable
git checkout -b stable $(git tag | tail -1)
chmod +x ./.deployment/deploy-deepforge && ./.deployment/deploy-deepforge stable
if: github.event_name == 'release'
3 changes: 2 additions & 1 deletion docker/Dockerfile.kitchensink
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
FROM deepforge/server:latest
ARG TAG=latest
FROM deepforge/server:$TAG

RUN deepforge extensions add deepforge-dev/deepforge-keras && deepforge create-env -f