Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/workflows/pull_requests_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ on:
- reopened
- synchronize

permissions:
contents: read

jobs:
static-checks:
name: Static Checks
Expand All @@ -27,3 +30,19 @@ jobs:
GOLANGCI_LINT_VERSION: "1.44.2"
- name: Run pre-commit
uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1

unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Configure Go
uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: 1.16.x
- name: Build
run: go build ./...
- name: Vet
run: go vet ./...
- name: Test
run: go test ./...
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
42 changes: 42 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
- [github-flow-manager](#github-flow-manager)
- [Help](#help)
- [Example](#example)
- [How specific check names are evaluated](#how-specific-check-names-are-evaluated)
- [Pre commit](#pre-commit)
- [Expressions](#expressions)
- [Available variables](#available-variables)
Expand Down Expand Up @@ -37,6 +38,8 @@ Usage:
github-flow-manager [OWNER] [REPOSITORY] [SOURCE_BRANCH] [DESTINATION_BRANCH] [EXPRESSION] [SPECIFIC_COMMIT_CHECK_NAME - Optional] [flags]

Flags:
--accept-skipped-checks
Accept a required check GitHub reports as SKIPPED as satisfied
-c, --commits-number int Number of commits to get under evaluation (>0, <=100) (default 100)
-d, --dry-run Don't modify repository
-f, --force Use the force Luke... - Changes branch HEAD with force
Expand All @@ -61,6 +64,45 @@ GITHUB_TOKEN=xxx github-flow-manager octocat Hello-World test master "StatusSucc
GITHUB_TOKEN=xxx github-flow-manager octocat Hello-World test master "StatusSuccess == false" "pipeline-1-name-to-be-checked,pipeline-2-name-to-be-checked" --verbose --dry-run
```

## How specific check names are evaluated

When `SPECIFIC_COMMIT_CHECK_NAME` is given, every name in it is evaluated
**separately**, and `StatusSuccess` is true only when **all** of them are
satisfied. A name is satisfied when:

- at least one result on the commit carries that name - otherwise the check
never ran and the commit is not promotable;
- no result for that name is still queued or in progress, so a promotion can
never pre-empt a verdict; and
- every concluded result for that name concluded `SUCCESS`.

Only `SUCCESS` satisfies a required check. `NEUTRAL` deliberately does **not**:
`devops-pipelines`' `hotfix_aware_skip_check` publishes `hotfix-skip-tests` as
`SUCCESS` to mean "authorised to promote without tests" and `NEUTRAL` to mean
"not a hotfix", and repositories gate their hotfix promote on that single name,
so accepting `NEUTRAL` would turn a fail-closed gate into a fail-open one.

A name may legitimately be reported **more than once** - a merge queue builds a
commit twice, once for the queue run and once for the push run, and a reusable
workflow called by several callers publishes its check once per caller. Several
results for one name are fine as long as they are all green. A name can also be
reported as a commit status context, as the name of a workflow, or as the name of
a check run; all three are searched.

`SKIPPED` does not satisfy a required check either, unless
`--accept-skipped-checks` is passed. Prefer solving a legitimate skip *upstream*,
by publishing one aggregator check that is only ever `success` or `failure` -
`dbt-app`'s `dbt-app CI merge readiness` and `noa-whisper-app`'s
`all_builds_passed` both do this - so the promotion gate never has to interpret a
skip.

Without `SPECIFIC_COMMIT_CHECK_NAME`, `StatusSuccess` comes from GitHub's own
status check rollup for the commit, unchanged.

With `--verbose`, any commit held back by its required checks is listed under the
table with one line per required name, so a stalled promotion can be diagnosed
from the job log.

## Pre commit

This repo leverage pre commit to lint, secure, document the IaaC codebase. The pre-commit configuration require the following dependencies:
Expand Down
31 changes: 24 additions & 7 deletions cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,13 @@ import (
)

var (
commitsNumber *int
githubToken *string
force *bool
verbose *bool
dryRun *bool
separator *string
commitsNumber *int
githubToken *string
force *bool
verbose *bool
dryRun *bool
separator *string
acceptSkippedChecks *bool
)

const (
Expand Down Expand Up @@ -68,7 +69,7 @@ If a SPECIFIC_COMMIT_CHECK_NAME is specified, the StatusSuccess will be calculat
*githubToken = os.Getenv("GITHUB_TOKEN")
}

results, err := flow_manager.Manage(*githubToken, owner, repo, sourceBranch, destinationBranch, expression, specificChecksNames, *separator, *commitsNumber, *force, *dryRun)
results, err := flow_manager.Manage(*githubToken, owner, repo, sourceBranch, destinationBranch, expression, specificChecksNames, *separator, *acceptSkippedChecks, *commitsNumber, *force, *dryRun)
if err != nil {
fmt.Println(err.Error())
os.Exit(1)
Expand Down Expand Up @@ -105,6 +106,21 @@ If a SPECIFIC_COMMIT_CHECK_NAME is specified, the StatusSuccess will be calculat

table.Render()

// Without this, a commit held back by its required checks looks
// identical to one held back by the expression, which makes a stalled
// promotion very hard to diagnose from the job log alone.
for _, res := range results {
c := res.Commit
if c.StatusSuccess || len(c.ChecksSummary) == 0 {
continue
}

fmt.Println("Required checks not satisfied for " + c.SHA + ":")
for _, line := range c.ChecksSummary {
fmt.Println("\t" + line)
}
}

endingMessage := "THERE IS NO COMMITS PASSING EVALUATION"
if results[len(results)-1].Result {
endingMessage = "NO MORE COMMITS WERE EXAMINED BECAUSE LAST ONE EVALUATED SUCCESSFULLY"
Expand All @@ -127,4 +143,5 @@ func init() {
verbose = rootCmd.Flags().BoolP("verbose", "v", false, "Print table with commits evaluation status")
dryRun = rootCmd.Flags().BoolP("dry-run", "d", false, "Don't modify repository")
separator = rootCmd.Flags().StringP("separator", "s", ",", "Set string separator of status checks")
acceptSkippedChecks = rootCmd.Flags().Bool("accept-skipped-checks", false, "Accept a required check GitHub reports as SKIPPED as satisfied (off by default: only SUCCESS satisfies a required check)")
}
225 changes: 225 additions & 0 deletions github/checks.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
package github

import (
"fmt"
"strings"

"github.com/shurcooL/githubv4"
)

// checkOutcome is the normalised outcome of a single result that GitHub
// reported for a required check name.
type checkOutcome int

const (
// outcomePassed means the result is green, or green enough to promote.
outcomePassed checkOutcome = iota
// outcomePending means the result has not concluded yet, so promoting now
// would pre-empt its verdict.
outcomePending
// outcomeFailed means the result concluded in a state that is not green.
outcomeFailed
)

// checkResult is one result on a commit that carries a required check name,
// together with the raw state GitHub reported, kept for the operator-facing
// summary.
type checkResult struct {
outcome checkOutcome
verdict string
}

// noVerdictYet is shown for a result GitHub has not concluded yet.
const noVerdictYet = "NO_CONCLUSION_YET"

// splitCheckNames splits the required check names supplied on the command line,
// dropping whitespace and empty entries so that a stray separator (`a,b,` or
// `a, b`) cannot silently add a name that can never be satisfied.
func splitCheckNames(specificChecksNames, sep string) []string {
var names []string
for _, name := range strings.Split(specificChecksNames, sep) {
if trimmed := strings.TrimSpace(name); trimmed != "" {
names = append(names, trimmed)
}
}

return names
}

// classifyConclusion maps the conclusion of a check suite or a check run onto a
// checkOutcome.
//
// Only SUCCESS satisfies a required check, which is exactly the acceptance rule
// the previous implementation applied - this change fixes how results are
// counted, deliberately without widening what counts as green.
//
// NEUTRAL in particular must not satisfy anything. devops-pipelines'
// hotfix_aware_skip_check workflow publishes its `hotfix-skip-tests` check as
// SUCCESS to mean "this commit is authorised to promote without tests" and
// NEUTRAL to mean "not a hotfix", and repositories gate their hotfix promote on
// that single name. Accepting NEUTRAL would turn that gate from fail-closed into
// fail-open and let any commit promote untested.
//
// SKIPPED satisfies only when acceptSkipped is set, for repositories that skip a
// required job on purpose - see the --accept-skipped-checks flag.
func classifyConclusion(conclusion githubv4.String, acceptSkipped bool) checkOutcome {
if conclusion == "" {
// GitHub reports a null conclusion until the suite or run completes.
return outcomePending
}

switch githubv4.CheckConclusionState(conclusion) {
case githubv4.CheckConclusionStateSuccess:
return outcomePassed
case githubv4.CheckConclusionStateSkipped:
if acceptSkipped {
return outcomePassed
}
return outcomeFailed
default:
return outcomeFailed
}
}

// classifyCheckRun classifies a single check run. Its status wins over its
// conclusion: a run that has not completed cannot have a trustworthy verdict.
func classifyCheckRun(checkRun CheckRunNodes, acceptSkipped bool) checkResult {
if checkRun.Status != "" && githubv4.CheckStatusState(checkRun.Status) != githubv4.CheckStatusStateCompleted {
return checkResult{outcome: outcomePending, verdict: string(checkRun.Status)}
}

return checkResult{outcome: classifyConclusion(checkRun.Conclusion, acceptSkipped), verdict: verdictOf(checkRun.Conclusion)}
}

// verdictOf renders a conclusion for the summary, naming the empty case.
func verdictOf(conclusion githubv4.String) string {
if conclusion == "" {
return noVerdictYet
}

return string(conclusion)
}

// classifyStatusContext classifies a single commit status context. Commit
// statuses carry no conclusion, only a state.
func classifyStatusContext(ctx Context) checkResult {
result := checkResult{verdict: string(ctx.State)}

switch githubv4.StatusState(ctx.State) {
case githubv4.StatusStateSuccess:
result.outcome = outcomePassed
case githubv4.StatusStatePending, githubv4.StatusStateExpected:
result.outcome = outcomePending
default:
// ERROR, FAILURE, or a state this build does not recognise.
result.outcome = outcomeFailed
}

return result
}

// collectResults gathers every result on the commit that carries the given
// required check name.
//
// The same name can reach us through three different GitHub concepts - a commit
// status context, the name of a workflow run, and the name of a check run - so
// all three are searched. One name legitimately matching several results is
// normal: while a merge queue is active, a commit is built twice, once for the
// queue run and once for the push run, and both report the same workflow name.
func collectResults(name string, edge Edge, acceptSkipped bool) []checkResult {
var results []checkResult

// A commit status set directly on the commit (the classic statuses API).
for _, ctx := range edge.Node.Status.Contexts {
if githubv4.String(name) == ctx.Context {
results = append(results, classifyStatusContext(ctx))
}
}

for _, checkSuite := range edge.Node.CheckSuites.Nodes {
// The required name can be the name of the workflow itself, in which
// case the suite's conclusion is the workflow run's verdict.
if (checkSuite.WorkflowRun != WorkflowRun{}) && githubv4.String(name) == checkSuite.WorkflowRun.Workflow.Name {
conclusion := checkSuite.WorkflowRun.CheckSuite.Conclusion
results = append(results, checkResult{
outcome: classifyConclusion(conclusion, acceptSkipped),
verdict: verdictOf(conclusion),
})
}

// Or the name of an individual check run inside the suite, which is how
// a required "workflow / job" check appears.
for _, checkRun := range checkSuite.CheckRuns.Nodes {
if githubv4.String(name) == checkRun.Name {
results = append(results, classifyCheckRun(checkRun, acceptSkipped))
}
}
}

return results
}

// evaluateCheckName applies "nothing missing, nothing pending, nothing red" to a
// single required check name and returns whether it is satisfied plus a line for
// the summary.
func evaluateCheckName(name string, edge Edge, acceptSkipped bool) (bool, string) {
results := collectResults(name, edge, acceptSkipped)
if len(results) == 0 {
return false, fmt.Sprintf("%s: NEVER RAN - no commit status, workflow run or check run carries this name", name)
}

verdicts := make([]string, 0, len(results))
var pending, failed int
for _, result := range results {
verdicts = append(verdicts, result.verdict)
switch result.outcome {
case outcomePending:
pending++
case outcomeFailed:
failed++
case outcomePassed:
}
}
reported := fmt.Sprintf("%s: %d result(s) [%s]", name, len(results), strings.Join(verdicts, " "))

switch {
case failed > 0:
return false, reported + fmt.Sprintf(" - %d did not pass", failed)
case pending > 0:
return false, reported + fmt.Sprintf(" - %d still running", pending)
}

return true, "OK " + reported
}

// evaluateSpecificChecks reports whether every required check name is satisfied
// on the commit, along with a per-name summary.
//
// Every name is evaluated on its own. An earlier implementation summed the
// successful results across all names and compared that total to the number of
// names, which meant a surplus on one name could silently cover a deficit on
// another - promoting commits whose required checks had never gone green - while
// a name reporting more than one successful result pushed the total past the
// exact-equality test and blocked commits where everything had in fact passed.
// Duplicate results per name are routine, which is what made both directions of
// that bug reachable: a merge queue builds a commit twice, and a reusable
// workflow called by several callers reports its check once per caller.
func evaluateSpecificChecks(edge Edge, checkNames []string, acceptSkipped bool) (bool, []string) {
if len(checkNames) == 0 {
// Names were asked for but none are usable. Fail closed: an empty
// requirement set would make every commit vacuously promotable.
return false, []string{"no usable required check names were supplied - refusing to promote"}
}

statusSuccess := true
summary := make([]string, 0, len(checkNames))
for _, name := range checkNames {
passed, line := evaluateCheckName(name, edge, acceptSkipped)
if !passed {
statusSuccess = false
}
summary = append(summary, line)
}

return statusSuccess, summary
}
Loading
Loading