Use separate jobs instead of child_process

[amvanbaren]

This PR introduces the use of separate jobs instead of child_process to sandbox the extension build process.

[tfroment]

+1

[tfroment]

hi @amvanbaren, do you know why this is failed https://github.com/EclipseFdn/publish-extensions/actions/runs/15301776884/job/43043958569?pr=881

[amvanbaren]

The reporting functionality is broken:

publish-extensions/.github/workflows/publish-extensions.yml

Lines 50 to 77 in 7420c17

	# TODO fix reporting
	# - name: Report results
	# run: bun run ./report-extensions.ts
	# - uses: actions/upload-artifact@v4
	# if: always()
	# with:
	# name: report
	# path: |
	# /tmp/stat.json
	# /tmp/result.md
	# - uses: actions/upload-artifact@v4
	# if: always()
	# with:
	# name: artifacts
	# path: |
	# /tmp/artifacts/*.vsix
	# - name: Upload job summary
	# if: always()
	# run: cat /tmp/result.md >> $GITHUB_STEP_SUMMARY
	# - name: Get previous job's status
	# id: lastrun
	# uses: filiptronicek/get-last-job-status@main
	# - name: Slack Notification
	# if: ${{ !github.event.inputs.extensions && ((success() && steps.lastrun.outputs.status == 'failed') || failure()) }}
	# uses: rtCamp/action-slack-notify@v2
	# env:
	# SLACK_WEBHOOK: ${{ secrets.GITPOD_SLACK_WEBHOOK }}
	# SLACK_COLOR: ${{ job.status }}

There's a check for important extensions: https://github.com/EclipseFdn/publish-extensions/blob/f7cfb5c517bbf53d772704383db8b22b4cb895aa/report-extensions.ts#L226C1-L232C2

Because the reporting functionality is broken every extension is seen as outdated and the Validate PR job fails.

[filiptronicek]

In general, I'm aligned and happy with this approach --> we never really did publishing outside of GitHub Actions anyway.

I'm curious about:

• How does our local testing story look like? I used to run node publish-extension.js with a list of extensions when debugging build issues with certain extensions
• How can we test the PR before landing it? If it's not worth the effort to do here, I'm also fine with merging it and testing then
• Do we have an idea on how this will affect timing? I know there's some concurrency limits for GHA workflows, so given there's > 400 extensions, I worry the publishing times might get lengthy.

[filiptronicek]

Does resolveExtension have a side effect we need here or can we safely remove it?

[amvanbaren]

Yes, it downloads extension files to /tmp/download/

[filiptronicek]

Can you please run npm run format here?

[amvanbaren]

Do we have an idea on how this will affect timing? I know there's some concurrency limits for GHA workflows, so given there's > 400 extensions, I worry the publishing times might get lengthy.

Yes, it's going to be lengthy and create a lot of jobs: from https://github.com/amvanbaren/publish-extensions/actions?page=2 to https://github.com/amvanbaren/publish-extensions/actions?page=18 for a single run.

[filiptronicek]

@amvanbaren do you think we could potentially hit this limitation GitHub has? As I mentioned, we have about 420 extension as of now, so maybe we wouldn't hit it today.

Workflow run queue - No more than 500 workflow runs can be queued in a 10 second interval per repository. If a workflow run reaches this limit, the workflow run is terminated and fails to complete.

From https://docs.github.com/en/actions/administering-github-actions/usage-limits-billing-and-administration#usage-limits

I guess when we do hit it, we can just add some rate limiting on our side to wait like 0.05 seconds between each trigger

[amvanbaren]

How does our local testing story look like?

I've added local-workflow.js

I guess when we do hit it, we can just add some rate limiting on our side to wait like 0.05 seconds between each trigger

I've added a rate limiter

[amvanbaren]

@filiptronicek Can we move this forward?

[filiptronicek]

LGTM. Sorry for the holdup, but let's finally give this a shot

[filiptronicek]

Running for the first time: https://github.com/EclipseFdn/publish-extensions/actions/runs/15843713746

[filiptronicek]

RequestError [HttpError]: No ref found for: build-extension [1]

@amvanbaren could you take a look?

[amvanbaren]

@filiptronicek Can you take a look at #903? It fixes the workflow ref.