-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
connectors-ci: publish strict-encrypt first #26644
connectors-ci: publish strict-encrypt first #26644
Conversation
Confirmed the sorts works with this pre-release workflow. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One comment
they will be sorted first with our reverse sort below. | ||
""" | ||
|
||
return sorted(contexts, key=lambda context: context.connector.technical_name, reverse=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I like this!
But, I think concurrency might still bite us
i.e. if we have concurrency set to 5 we might build source-file
and source-file-secure
at the same time
Perhaps we want the sort to pull the strict-encrypt variants to the start and their counter parts to the end of the list?
Or at the conncurreny level we explicitly try and build all strict encrypts in one block before anything else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On publish concurrency is forced to 1 for safety/stability.
What
Closes #26635
The metadata upload step on publish checks if the the connectors referenced in the metadata file are already published to DockerHub. Non strict-encrypt variant reference the strict-encrypt variant in their metadata file for cloud.
So if we publish the non strict-encrypt variant first, the metadata upload will fail if the strict-encrypt variant is not published yet.
As strict-encrypt variant are often modified in the same PR as the non strict-encrypt variant, we want to publish them first.
How
Hacky approach: as connector names with -strict-encrypt/secure prefix are longer, they will be sorted first with a reverse sort below.