Failures later than the ~100th job are ignored

[relsqui]

Describe the bug: バグの概要

We use this action in a workflow that includes a matrix of ~180 jobs. When one of the later jobs to run fails, this action reports the workflow conclusion as a success.

To Reproduce: 再現手順

Steps to reproduce the behavior:

1. Create a workflow with a matrix of many jobs.
2. Include a call to workflow-conclusion-action at the end.
3. Induce a failure in a job that will finish very late.
4. Check workflow conclusion.

Expected behavior: 期待する動作

Workflow conclusion is a failure.

Screenshots: スクリーンショット

This is the output of workflow-conclusion-action in a workflow where a late matrix job failed.

Operating environment: バグが発生した環境

• Version of software: v2
• OS: [e.g. Windows10] ubuntu-20.04
• Browser: [e.g. Chrome, Safari] n/a
• etc.

Additional context: 補足

The specific numbers involved (180ish jobs, 85 truncated in the screenshot, default page size of 100) make me suspect this is pagination-related.

[technote-space]

Not reproducible

[technote-space]

120 jobs + 2 other jobs

98 + 24 = 122

[technote-space]

workflow-conclusion-action/src/process.ts

Line 13 in 858e8f6

export const getJobs = async(octokit: Octokit, context: Context): Promise<Array<ActionsListJobsForWorkflowRunResponseData>> => octokit.paginate(

pagination

[technote-space]

Perhaps the "needs" setting is incorrect.

[technote-space]

Or same job name is used.

I don't remember why I implemented it that way at the time, but JOBs with the same name will override the status.

workflow-conclusion-action/__tests__/process.test.ts

Lines 58 to 81 in 858e8f6

	describe('getJobConclusions', () => {
	it('should get conclusions', () => {
	expect(getJobConclusions([
	{name: 'test1', conclusion: 'cancelled'},
	{name: 'test2', conclusion: 'neutral'},
	{name: 'test3', conclusion: 'failure'},
	{name: 'test4', conclusion: 'success'},
	{name: 'test5', conclusion: 'failure'},
	{name: 'test6', conclusion: 'success'},
	{name: 'test7', conclusion: 'cancelled'},
	{name: 'test8', conclusion: 'skipped'},
	{name: 'test9', conclusion: 'test1'},
	{name: 'test9', conclusion: 'test2'},
	{name: 'test9', conclusion: 'test3'},
	])).toEqual([
	'cancelled',
	'neutral',
	'failure',
	'success',
	'skipped',
	'test3',
	]);
	});
	});

This behavior is confusing so I will change so that it is not overwritten.

[relsqui]

Thanks! I'm away from work this week but I'll check if this fixes it for us when I can.

[relsqui]

TL;DR I think this is the same bug as #107 and releasing the changes from #108 will fix it.

---

Hiya, sorry to revive an old thread, but we just ran into this problem again so I spent a little time trying to reproduce it more reliably. I made this minimal workflow to create large matrix jobs with exactly one failure and then run workflow-conclusion-action to see what it says. I wasn't able to reproduce the bug consistently, but here are some illustrative results:

• the conclusions array shows only success even though there are also cancelled and failed jobs, so the overall conclusion was incorrectly success:
  • https://github.com/relsqui/workflow-testing/actions/runs/6882343735/job/18720764649
  • https://github.com/relsqui/workflow-testing/actions/runs/6882248545/job/18720450073
  • https://github.com/relsqui/workflow-testing/actions/runs/6882218413/job/18720351438
  • https://github.com/relsqui/workflow-testing/actions/runs/6882292051/job/18720682810
• the overall conclusion was correct, but the conclusions array doesn't include the cancelled jobs:
  • https://github.com/relsqui/workflow-testing/actions/runs/6882343203/job/18720762761
• behaved correctly:
  • with fast fail on (some jobs are cancelled): https://github.com/relsqui/workflow-testing/actions/runs/6882557292/job/18721401852
  • with fast fail off (no jobs are cancelled): https://github.com/relsqui/workflow-testing/actions/runs/6882558836/job/18721439305

I never saw it fail on a run with fewer than 200 jobs, and never in this test repo when "fast fail" was turned off -- but in our actual work repo, we did see this error with fast fail off (and a very large matrix job). In the logs for that run, I see that the failed jobs are still marked as "in_progress," which led me to #107 .

I see that #107 was marked as completed after #108 was closed and a new version of the action was released ... but I don't see the changes from that PR anywhere in the code, in any version tag or on main. Did you intend to release that change? I think it would solve our problem.