Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/actions/lint-extension/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Lint Extension
description: Lint an extension for release

inputs:
extension-name:
description: The name of the extension
required: true
type: string

runs:
using: "composite"

steps:
# Ensures that the manifest.json for the given extension name
# contains all the required fields for the rest of the release workflow
- run: |
jq '
if (.extension | type) != "object" then error("Missing extension object")
elif (.extension.name | type) != "string" then error("Missing extension.name")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add additional lint around name and folder matching? Or lose name entirely and only rely on folder?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely a good idea. I'll review the workflow to see if this 1-1 match is required before adding a hard linting rule.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to do this as a follow-up for improved linting.

elif (.extension.title | type) != "string" then error("Missing extension.title")
elif (.extension.description | type) != "string" then error("Missing extension.description")
elif (.extension.homepage | type) != "string" then error("Missing extension.homepage")
elif (.extension.version | type) != "string" then error("Missing extension.version")
else . end
' ./extensions/${{ inputs.extension-name }}/manifest.json
Comment on lines +17 to +25
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a lot of inline jq. Should this, or any of the other jq scripts, move into package.json so that they can be run locally?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that is a good idea. I can look into that. It would be helpful to do a pre-push lint.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to look at this as a follow-up in combination with #35 (comment)

shell: bash

- uses: actions/setup-node@v4

- run: npm install -g semver
shell: bash

# The semver must be valid for the sorting, comparisons, and release
# process to work
- name: Check for valid semver
run: |
semver -c $(jq -c -r '.extension.version' < ./extensions/${{ inputs.extension-name }}/manifest.json)
shell: bash
45 changes: 45 additions & 0 deletions .github/actions/package-extension/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Package Extension
description: Package an extension into a tarball for release

inputs:
extension-name:
description: The name of the extension
required: true
type: string
artifact-name:
description: The name of the artifact
required: false
type: string

runs:
using: "composite"

steps:
# If we are not passed an artifact to use as the source for the tarball
# we will create a tarball from the extension's directory
Comment on lines +18 to +19
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly for my own education: it's not immediately clear to me when we would take this path versus the other path down below for these. Could you educate me?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely. Currently the only time we do this is in the publisher-command-center custom worfklow which builds the extension, and then creates an artifact to package:

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The creation and passing of the artifact to this action allows custom workflows to select which files to package.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok, so if it's a custom workflow style task, this allows us to build the artifact elsewhere (in the custom workflow) and then pass it to this workflow but/and therefore bypasses the building step in this workflow. Is that a good summary?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, even an accurate summary — doesn't need to be good 😂

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok, so if it's a custom workflow style task, this allows us to build the artifact elsewhere (in the custom workflow) and then pass it to this workflow but/and therefore bypasses the building step in this workflow. Is that a good summary?

Very close. The only correction I'll make is that this is a composite action. It allows us to build the artifact we want packaged and pass it to this action. It avoids building as part of the packaging step (for those extensions with custom workflows), and simplifies the things that need to be passed to this action.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a requested change, just a question: would it be beneficial to add additional detail such as "this is what we expect during for simple content", and in the next one "we expect artifacts to be passed and processed for complex content". Or is that getting too deep?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept the explanations in the actions a bit lighter since "simple content" probably doesn't need to interact with them. We can definitely add some deeper explanations if we get questions often.

- name: Create tar
if: ${{ inputs.artifact-name == '' }}
run: tar -czf ${{ inputs.extension-name}}.tar.gz ./extensions/${{ inputs.extension-name}}
shell: bash

# If we are passed an artifact to use as the source for the tarball
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love all of the code comments. Keep doing that. 👍

# we will download the artifact and create a tarball from it
# this avoids including extra files in the extension
- name: Download optional artifact
if: ${{ inputs.artifact-name != '' }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we offer a message to the user if an error occurs during this process? Or is that even possible?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There will be a failure if the artifact doesn't exist and the logs will give the name and that no artifact matched it. Thats all built in to the download-artifact action. Does that satisfy that you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I think that makes sense.

uses: actions/download-artifact@v4
with:
name: ${{ inputs.artifact-name }}
path: ${{ inputs.extension-name }}

- name: Create tar from artifact
if: ${{ inputs.artifact-name != '' }}
run: tar -czf ${{ inputs.extension-name }}.tar.gz ${{ inputs.extension-name }}
shell: bash

# Upload the extension's tarball for use in other actions in the workflow
- name: Upload extension tar
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.extension-name }}.tar.gz
path: ${{ inputs.extension-name }}.tar.gz
94 changes: 94 additions & 0 deletions .github/actions/release-extension/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: Release Extension
description: Release an extension as a GitHub Release

inputs:
extension-name:
description: The name of the extension
required: true
type: string

runs:
using: "composite"

steps:
- uses: actions/setup-node@v4

- run: npm install -g semver
shell: bash

- name: Get manifest extension version
run: |
MANIFEST_VERSION=$(semver -c $(jq -c -r '.extension.version' < ./extensions/${{ inputs.extension-name }}/manifest.json))
echo "MANIFEST_VERSION=$MANIFEST_VERSION" >> "$GITHUB_ENV"
shell: bash

# Grabs the latest version from the extensions.json file
# If an extension hasn't been released yet we default to `0.0.0` so any
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to allow unreleased extensions to be merged to main? It could potentially simply things if the act of merging to main implied releasing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want to:

  • remove any extensions from the repo that we didn't want initially released
  • require that extension authors keep long lived branches until the extension is ready to be released

This may not be the right choice though.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I very much see where you're coming from @cgraham-rs , that theoretically keeping unrelease out of main makes things nice and clean, but PRs into other branches and having long running branches with development versions is advanced enough git that I would like to avoid requiring it if we can.

It would be nice to have this noted prominently in a README (if we don't already) that just cause something is on main doesn't mean it's released

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have this noted prominently in a README (if we don't already) that just cause something is on main doesn't mean it's released

Absolutely. The CONTRIBUTING doc we write for adding extensions will have this callout and tips for reading the GitHub Action logs to determine if a PR will result in a release or not 👍

# version in the manifest will be higher
- name: Get lastest version from extension list
continue-on-error: true
run: |
LATEST_VERSION=$(jq -c '.extensions[] | select(.name=="${{ inputs.extension-name }}").latestVersion.version' < extensions.json)
LATEST_VERSION=$(semver -c "${LATEST_VERSION:-0.0.0}")
echo "LATEST_VERSION=$LATEST_VERSION" >> "$GITHUB_ENV"
shell: bash

# We only want to release if the manifest.json contains a newer semver
# version than the latest version in the extensions.json
# We compare that here, and echo if a release will occur
# This can be helpful when looking at Pull Request action outputs
# so it is clear what will happen on a merge to `main`
- name: Check if manifest has newer version
id: should_release
run: |
echo "The manifest version is '$MANIFEST_VERSION' and the released version is '$LATEST_VERSION'"
HIGHER_VERSION=$(semver "$MANIFEST_VERSION" "$LATEST_VERSION" | tail -n 1)
if [ "$MANIFEST_VERSION" = "$HIGHER_VERSION" ] && [ "$MANIFEST_VERSION" != "$LATEST_VERSION" ]; then
echo "🚀 Will release! The manifest version is greater than the released version."
echo "should_release=true" >> "$GITHUB_OUTPUT"
else
echo "😴 Holding back from release: The manifest version is not greater than the released version."
echo "should_release=false" >> "$GITHUB_OUTPUT"
fi
shell: bash

# Here we download the packaged extension artifact to release
- uses: actions/download-artifact@v4
if: github.ref_name == 'main' && steps.should_release.outputs.should_release == 'true'
with:
name: ${{ inputs.extension-name }}.tar.gz

# The release tag utilizes both the extension name and semver version
# to create a unique tag for the repository
- name: Release tag
if: github.ref_name == 'main' && steps.should_release.outputs.should_release == 'true'
id: release_tag
run: |
RELEASE_TAG="${{ inputs.extension-name }}@v$MANIFEST_VERSION"
echo "RELEASE_TAG=$RELEASE_TAG" >> "$GITHUB_ENV"
shell: bash

- name: Release
if: github.ref_name == 'main' && steps.should_release.outputs.should_release == 'true'
run: |
gh release create $RELEASE_TAG \
--title "${{ inputs.extension-name }} v$MANIFEST_VERSION" \
${{ inputs.extension-name }}.tar.gz
shell: bash

# We fetch the GitHub release using the GitHub API, storing the data
# for use in the extension list update.
- name: Get release data
if: github.ref_name == 'main' && steps.should_release.outputs.should_release == 'true'
run: gh api /repos/${{ github.repository }}/releases/tags/$RELEASE_TAG > release-${{ inputs.extension-name }}.json
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More for my education / curiosity: what's in the release-....json?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question. It is the response from GitHub's "Get a release by tag name" API endpoint.

Using this as opposed to gh release list allowed me to get the specific release the flow needed. The script that updates the extension list uses published_at time and the list of assets to grab the browser_download_url of the extension TAR we created.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah interesting. It might be worth stating (more?) explicitly in the comment "We fetch the full release data from the github API" That wasn't at the top of my head for where we were fetching that from, actually.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the comment in e534088 to mention that the release data is fetched from the GitHub API 👍

shell: bash

# Each release data file is uploaded as an artifact for the extension list update
# The naming convention is `release-<extension-name>.json` to easily
# download each artifact matching the pattern
- name: Upload release data
if: github.ref_name == 'main' && steps.should_release.outputs.should_release == 'true'
uses: actions/upload-artifact@v4
with:
name: release-${{ inputs.extension-name }}.json
path: release-${{ inputs.extension-name }}.json
26 changes: 0 additions & 26 deletions .github/workflows/audit-api.yaml

This file was deleted.

26 changes: 0 additions & 26 deletions .github/workflows/audit-reports.yaml

This file was deleted.

25 changes: 0 additions & 25 deletions .github/workflows/datadog-prometheus-metrics.yaml

This file was deleted.

Loading