fix(plugin-circleci): populate workflow id for unfinished-job collection (#8907)#8912
Open
zahorniak wants to merge 2 commits into
Open
fix(plugin-circleci): populate workflow id for unfinished-job collection (#8907)#8912zahorniak wants to merge 2 commits into
zahorniak wants to merge 2 commits into
Conversation
… a helper Signed-off-by: Volodymyr Zahorniak <v.zahorniak@gmail.com>
…ion (apache#8907) The collectJobs 'unfinished details' collector built its URL from '/v2/workflow/{{ .Input.Id }}/job' but its iterator selected 'DISTINCT workflow_id' into a models.CircleciJob, leaving .Id empty and producing '/v2/workflow//job' (HTTP 500) whenever a job was running/queued/on_hold. Alias the projection to 'workflow_id AS id' so .Id carries the workflow id, mirroring the new-records collector. Adds an e2e regression test. Signed-off-by: Volodymyr Zahorniak <v.zahorniak@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pr-type/bug-fix.Summary
The CircleCI
collectJobssubtask runs two API collectors that share the URL template/v2/workflow/{{ .Input.Id }}/job:idfrom_tool_circleci_workflows, so.Input.Idis populated and the URL is well-formed.DISTINCT workflow_idfrom_tool_circleci_jobsand scanned each row into amodels.CircleciJob. Since the selected column isworkflow_id, the scan filledCircleciJob.WorkflowIdand leftCircleciJob.Idempty — so the shared template collapsed to/v2/workflow//job, which the CircleCI API answers with HTTP 500.This triggers whenever a job is in a non-terminal status (
running/not_running/queued/on_hold) — e.g. an approval-gated workflow sittingon_hold. The 500 aborts the whole CircleCI collection, so all CI/CD data (and the DORA metrics derived from it) goes stale.Fix
Alias the projection to
DISTINCT workflow_id AS idso the scannedCircleciJob.Idcarries the workflow id, mirroring the new-records collector that already selectsid. This is the minimal change and keeps both collectors consistent.To make the behaviour testable, the "unfinished details" DAL clauses are extracted into an exported helper,
tasks.UnfinishedJobsInputClauses(...), which both the collector and the new test call — so the regression test exercises the production query rather than a copy of it.Tests
Adds
TestCircleciUnfinishedJobsInputIterator(e2e, real DB). It:tasks.UnfinishedJobsInputClauses) through the realapi.DalCursorIterator— the same path the collector uses;.Idequals the workflow id, that results are DISTINCT per workflow, and that the status/connection filters hold.The test fails before the fix (
.Idis empty →["", ""]) and passes after. Because it calls the exported production helper, a future revert of theSelectre-breaks the test.Does this close any open issues?
Closes #8907
Screenshots
N/A — backend collector fix, no UI changes.
Other Information
workflow_id→workflow_id AS id) plus a test-enabling extraction. No domain-layer, schema, migration, or snapshot changes; existing CircleCI e2e (Extract/Convert) tests are unaffected.git commit -s).go build ./plugins/circleci/...,go vet ./plugins/circleci/..., and the full./plugins/circleci/e2e/...suite pass against a real MySQL 8./v2/workflow//job500 — so the bug reliably breaks CircleCI collection at scale, not just in edge cases.