Skip to content

Add feature to dynamically generate image tags#45

Merged
DeimerM merged 30 commits intomainfrom
dmh/create-image-tag
Jun 10, 2025
Merged

Add feature to dynamically generate image tags#45
DeimerM merged 30 commits intomainfrom
dmh/create-image-tag

Conversation

@DeimerM
Copy link
Contributor

@DeimerM DeimerM commented Jun 5, 2025

Problem description:

Initially, this new feature was proposed due to the team's concern that Cirrus Hosting clients could potentially pull images belonging to other clients stored in the shared ECR repository. This raised the need to generate image tags dynamically and, in addition, to append a random alphanumeric suffix to the tags, in order to make the images harder to identify and thus prevent unauthorized downloads.

However, during the development of this PR, a conversation took place with a edunext team, as they had also implemented a similar solution. This led to a broader scope being defined in order to cover more use cases and better meet the needs of the different teams that use the Picasso tool.

Explanation of the new feature:

When the feature developed in this PR is enabled, the image build process for the specified service will skip the tag initially configured in the strain. However, it is still necessary to define an initial image value for the service to be built, since Picasso will use this value to locate the Docker registry and repository name, and it will only modify the tag portion.

image-name:**tag**

Feature configuration and behavior:

To configure this feature, you need to pass the setting USE_DYNAMIC_IMAGE_TAG: true to the Picasso job. This will automatically enable tag overwriting and build the image using the newly generated tag.

The tag generated by default when this setting is enabled will look like the following: v19.0.3-20250606-1012. This tag is rendered using the following parameters: {tutor_version}-{timestamp}, where the timestamp follows this format: "%Y%m%d-%H%M".

However, along with this feature, it is possible to configure additional parameters in order to generate a more customized tag:

If we inspect the parameters used in the tag, we have the following structure: tutor_version-prefix + timestamp-random_part which corresponds to: {tutor_version}-{prefix}{timestamp}-{random_part}.

The prefix and timestamp parameters can be customized by passing the following inputs:

  • PREFIX: Allows you to define a custom string to appear before the timestamp.
  • TIMESTAMP_FORMAT: This parameter allows you to define the format of the timestamp. The format must be valid and supported by Python's strftime function.

The random_part parameter can be enabled by setting USE_RANDOM_SUFFIX to true. Once activated, a random alphanumeric string will be appended to the end of the tag.

Additionally, a feature was added that allows pushing a commit with the new tag to the repository where the strain is located. This functionality was designed for the Cirrus Hosting use case, in order to maintain clear control over the tag of the newly built image.

To use this feature, the job that runs the Picasso workflow must be granted contents: write permissions, and the parameter UPDATE_REMOTE_CONFIG must be set to true. It is important to note that this feature will only work if the variable USE_DYNAMIC_IMAGE_TAG is also enabled.

Considerations: It is important to ensure that the PICASSO_VERSION parameter is set to a version of Picasso that includes these changes in order for the feature to work properly.

An example of a Picasso configuration using these settings can be found below:

jobs:
  build:
    permissions:
      contents: write
    name: Build with Picasso
    uses: eduNEXT/picasso/.github/workflows/build.yml@dmh/create-image-tag
    with:
      BUILDKIT_MAX_PARALLELISM: ${{ fromJSON(inputs.BUILDKIT_MAX_PARALLELISM) }}
      STRAIN_REPOSITORY: ${{ github.repository }}
      STRAIN_REPOSITORY_BRANCH: ${{ inputs.STRAIN_REPOSITORY_BRANCH }}
      STRAIN_PATH: 'build'
      SERVICE: ${{ inputs.SERVICE }}
      USE_DYNAMIC_IMAGE_TAG: true
      UPDATE_IMAGE_TAG_IN_REPO: true
      ADD_RANDOM_SUFFIX_TO_IMAGE_TAG: true
      TIMESTAMP_FORMAT: "%Y%m%d"
      IMAGE_TAG_PREFIX: "picasso-"
      PICASSO_VERSION: dmh/create-image-tag
    secrets:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_REGION: ${{ secrets.AWS_REGION }}
      SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}

This step is in charge to create a random tag for the image service.
@DeimerM DeimerM requested a review from a team as a code owner June 5, 2025 16:44
@DeimerM DeimerM force-pushed the dmh/create-image-tag branch from 5c7d32b to eff9b1c Compare June 5, 2025 20:04
@DeimerM DeimerM force-pushed the dmh/create-image-tag branch 19 times, most recently from 7fa367a to 0ec75b0 Compare June 6, 2025 01:39
Copy link
Contributor

@magajh magajh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this, @DeimerM! I really like the logic and how it was implemented — I just have a couple of suggestions

@magajh
Copy link
Contributor

magajh commented Jun 9, 2025

Also @DeimerM, we need to update the documentation with this new feature

DeimerM and others added 7 commits June 9, 2025 12:36
Co-authored-by: Maga Giorgianni <maria.jaimes@edunext.co>
Co-authored-by: Maga Giorgianni <maria.jaimes@edunext.co>
Co-authored-by: Maga Giorgianni <maria.jaimes@edunext.co>
Co-authored-by: Maga Giorgianni <maria.jaimes@edunext.co>
Co-authored-by: Maga Giorgianni <maria.jaimes@edunext.co>
@DeimerM
Copy link
Contributor Author

DeimerM commented Jun 9, 2025

Also @DeimerM, we need to update the documentation with this new feature

Thanks a lot for your review @magajh I have created the article where I explain this new feature and how to configure it. Please let me know if you have any additional comments.

@magajh
Copy link
Contributor

magajh commented Jun 9, 2025

@DeimerM awesome! can we please also update the readme? I think we could mention this feature in the workflow-overview and also include the inputs in the configuration section

Copy link
Contributor

@magajh magajh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a couple of suggestions

@magajh magajh self-requested a review June 10, 2025 14:48
Copy link
Contributor

@magajh magajh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

DeimerM added 2 commits June 10, 2025 13:43
This changes was done due to the library that we were using is not the
library recommended by python, so we have updated it.

Also we fix some single quotes inconsistency.
@DeimerM DeimerM force-pushed the dmh/create-image-tag branch from d4d8e1f to 558d3d4 Compare June 10, 2025 20:26
This update aims to avoid file I/O operations
@DeimerM DeimerM force-pushed the dmh/create-image-tag branch from 558d3d4 to fc7010e Compare June 10, 2025 20:31
@DeimerM
Copy link
Contributor Author

DeimerM commented Jun 10, 2025

@Squirrel18 Regarding your additional comment about whether it is valuable to include the Tutor version in each of the images, I’d like to confirm that we discussed this with the team, and we believe it is indeed important for all images to include it. This is because it is the only truly mandatory variable that must be present in the config.yml, and it can be very helpful for operators to determine which version of the Open edX platform and Tutor was used to build the image.

Copy link
Member

@Squirrel18 Squirrel18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @DeimerM for this PR. I didn't have much time to review it in depth, but it covers a useful use case. Thanks for taking our case into account!

@DeimerM DeimerM merged commit 4b879dc into main Jun 10, 2025
1 check passed
@DeimerM DeimerM deleted the dmh/create-image-tag branch June 10, 2025 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants