Image tag passing with matrix strategy + artifact method #4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Try-except block brought to the top
Removed extraneous seed_model.json
Removed test workflow execution. Added upload artifact.
Commented image build and push for testing purposes. Fixed indentation. Learnt that YAML do not permit using tabs as indentation; spaces allowed.
Temporary test file while working on automating tags modification. Checking to see if new artifacts generated even if overwrite is set to true.
Storing actual required tag with date timestamp.
Checked out from image-push-merge branch which has artifact method working. - Changed trigger branch on push to tags-dispatch - Dispatching repository event from server so that join page repo's workflow will be triggered. - Once this dispatch method works, might not even need artifacts since repository dispatch has a client_payload option to pass data. - This is helpful since only 500 artifacts are allowed in any repo: https://github.com/actions/upload-artifact#:~:text=has%20a%20limit-,of%20500%20artifacts.,-For%20assistance%20with
Trying with workflow dispatch.
Removing failed inputs
Testing triggering workflow dispatch again.
Changed branch name to new branch so it picks workflow events in join from this branch.
Testing if working with updated token scopes to have only actions: write
Testing sending docker image tags as input values in the POST request.
Sending actual date timestamp which is used as a suffix to the docker image tag.
Removed code related to artifact method for sending docker image tags. Workflow dispatch uses POST request via which input data values can be sent to the repository in which the workflow is to be triggered.
Copied over YAML file from e-mission-server repo after successfully implementing transmission of docker image tags from e-mission-server to join with artifacts and workflow dispatch. Added another dispatch job to trigger workflow via matrix strategy.
Updated branch name which was set for the workflow dispatch without matrix build.
Jobs completed successfully but nothing happens in the other three target repos.
I observed in real-time that the matrix jobs - "dispatch”, start before or at the same time the main build job and get over really quickly.
Hence, the error makes sense as the docker image tag is not generated yet:
{
"message": "Required input 'docker_image_tag' not provided",
"documentation_url": "https://docs.github.com/rest/actions/workflows#create-a-workflow-dispatch-event"
}
Hence, the timeout-minutes parameter makes sense to use as was mentioned in the blog post:
https://www.amaysim.technology/blog/using-github-actions-to-trigger-actions-across-repos
5 minutes seems too much, I’ll give it a minute for now.
Alternatively, timeout not needed; can use "needs" similar to how fetch_run_id job was run first before trying to download artifact.
Since the date is being generated in a different jobs, must access it using needs keyword.
Properly added date by setting it as an output value of the 1st build job. Then storing it as an environment variable in the 2nd job by accessing it via the needs keyword. Finally using it in the curl POST request by referencing the environment variable.
Editing sample file as a sanity check to see if matrix build works.
The bug was triggered by a .gitignore-d conf/ directory in public-dash which had a db.conf file. This was being loaded when docker-compose.dev.yml was used to test the dev version of public-dash. This was occurring since there was a nested dictionary in the db.conf.sample and db.conf files while I had initially only stored the nested keys (url, result_limit). Since it was still reading from the file, it stored the nested dictionary format with timeseries as the parent key followed by (url, result_limit) as children. Hence, fixing it by adding the same nested dictionary structure in the emission/core/config.py and emission/core/get_database.py
Added build and push to have latest image available in docker hub that is generated with updated YAML file. Removed join repo from workflow dispatch. Added artifact so that this can be used in case of push trigger event in admin-dash and public-dash repos with the help of the python script fetch_runID.py in those repos, to get the latest run_ID for a specific branch with the artifact. Workflow dispatch would still use the input parameters which contains the latest timestamp tag.
Changed branch name to tags-combo-approach; resulted in a failed workflow in admin-dash, public-dash since branch was still set to tags-matrix.
Changing step name to exclude join-page; included earlier for testing purposes but now focussing on the dashboard repos which actually use the server as the base image and hence need the workflow to be triggered.
This reverts commit 2fdb469.
This reverts commit 273c2ca.
Including the changes for the bug fix for the url KeyError for db.conf.
Reminder for things to change as per master branch of e-mission-server once changes are finalized.
Adding extra words to TODO just to trigger a workflow dispatch run and check if .env commit action works successfully for this trigger type too.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR includes @MukuFlash03 work for sharing the server's docker image build tag to the directories that build upon that image.
We use the matrix strategy to trigger the workflows in the public and admin dash repos, while using the GitHub API to send the tag to the appropriate workflows. See this comment for details, and subsequent comments for even more information.