-
Notifications
You must be signed in to change notification settings - Fork 0
init: import whichtests sources #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
ethanndickson
wants to merge
8
commits into
main
Choose a base branch
from
init
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+5,316
−0
Draft
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
09252a2
init: import scripts/testselect from coder/coder and split main.go
ethanndickson f8dc245
split: organize tests along source-file boundaries
ethanndickson 22fb9e9
chore: import boilerplate from coder/paralleltestctx
ethanndickson b925314
fix(testselect): address mechanical lint findings
ethanndickson 87aa54d
rename: testselect -> whichtests
ethanndickson 38707a6
fix: address gosec findings
ethanndickson 4acef74
fix: address whichtests review feedback
ethanndickson f3da5e5
refactor: simplify whichtests after round 2 review
ethanndickson File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| name: "Setup Go" | ||
| description: | | ||
| Sets up the Go environment for tests, builds, etc. | ||
| inputs: | ||
| version: | ||
| description: "The Go version to use." | ||
| default: "1.26.2" | ||
| use-cache: | ||
| description: "Whether to use the cache." | ||
| default: "true" | ||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - name: Setup Go | ||
| uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0 | ||
| with: | ||
| go-version: ${{ inputs.version }} | ||
| cache: ${{ inputs.use-cache }} | ||
|
|
||
| # It isn't necessary that we ever do this, but it helps separate the "setup" | ||
| # from the "run" times. | ||
| - name: go mod download | ||
| shell: bash | ||
| run: go mod download -x |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| name: quality | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| pull_request: | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| # Cancel in-progress runs for pull requests when developers push additional | ||
| # changes. | ||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: ${{ github.event_name == 'pull_request' }} | ||
|
|
||
| jobs: | ||
| fmt: | ||
| name: fmt | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: make fmt | ||
| run: make fmt | ||
|
|
||
| - name: Check unstaged | ||
| run: | | ||
| if [[ -n $(git ls-files --other --modified --exclude-standard) ]]; then | ||
| echo "Unexpected difference in directories after formatting. Run 'make fmt' and include the output in the commit." | ||
| exit 1 | ||
| fi | ||
|
|
||
| lint: | ||
| name: lint | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Setup Go | ||
| uses: ./.github/actions/setup-go | ||
|
|
||
| - name: make lint | ||
| run: make lint | ||
|
|
||
| test: | ||
| name: test | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Setup Go | ||
| uses: ./.github/actions/setup-go | ||
|
|
||
| - name: make test | ||
| run: make test |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # Binaries for programs and plugins | ||
| *.exe | ||
| *.exe~ | ||
| *.dll | ||
| *.so | ||
| *.dylib | ||
|
|
||
| # Test binary, built with `go test -c` | ||
| *.test | ||
|
|
||
| # Code coverage profiles and other test artifacts | ||
| *.out | ||
| coverage.* | ||
| *.coverprofile | ||
| profile.cov | ||
|
|
||
| # Go workspace file | ||
| go.work | ||
| go.work.sum | ||
|
|
||
| # env file | ||
| .env | ||
|
|
||
| # Editor/IDE | ||
| .idea/ | ||
| .vscode/ | ||
|
|
||
| # Key files | ||
| *.key | ||
| *.pub | ||
| *.pem | ||
|
|
||
| # Output directory | ||
| build/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| version: "2" | ||
|
|
||
| linters: | ||
| exclusions: | ||
| rules: | ||
| - path: _test\.go | ||
| linters: | ||
| - gosec | ||
| text: "G304: Potential file inclusion via variable" | ||
| enable: | ||
| - goconst | ||
| - gocritic | ||
| - gosec | ||
| - misspell | ||
| - nakedret | ||
| - revive | ||
| - unconvert | ||
| - unparam | ||
| settings: | ||
| govet: | ||
| enable: | ||
| - shadow | ||
| misspell: | ||
| locale: US | ||
| revive: | ||
| rules: | ||
| - name: package-comments | ||
| disabled: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| { | ||
| "printWidth": 120, | ||
| "semi": false, | ||
| "trailingComma": "all", | ||
| "overrides": [ | ||
| { | ||
| "files": ["./*.md", "./**/*.md"], | ||
| "options": { | ||
| "printWidth": 80, | ||
| "proseWrap": "always" | ||
| } | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| FIND_EXCLUSIONS= \ | ||
| -not \( \( -path '*/.git/*' -o -path './build/*' -o -path './vendor/*' -o -path '*/.terraform/*' \) -prune \) | ||
| GO_SRC_FILES := $(shell find . $(FIND_EXCLUSIONS) -type f -name '*.go' -not -name '*_test.go') | ||
| GO_FMT_FILES := $(shell find . $(FIND_EXCLUSIONS) -type f -name '*.go' -print0 | xargs -0 grep -E --null -L '^// Code generated .* DO NOT EDIT\.$$' | tr '\0' ' ') | ||
|
|
||
| default: build | ||
|
|
||
| build/whichtests: $(GO_SRC_FILES) go.mod go.sum | ||
| mkdir -p ./build | ||
| go build -o ./build/whichtests . | ||
|
|
||
| build: build/whichtests | ||
| .PHONY: build | ||
|
|
||
| fmt: | ||
| go mod tidy | ||
| go run golang.org/x/tools/cmd/goimports@v0.35.0 -w $(GO_FMT_FILES) | ||
| go run mvdan.cc/gofumpt@v0.8.0 -w -l $(GO_FMT_FILES) | ||
| .PHONY: fmt | ||
|
|
||
| lint: | ||
| go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.4.0 run ./... | ||
| .PHONY: lint | ||
|
|
||
| test: | ||
| go test -test.v -timeout 30s -cover ./... | ||
| .PHONY: test |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # whichtests | ||
|
|
||
| `whichtests` is the Go test-plan generator that drives the `flake-go` CI | ||
| workflow in `coder/coder`. Given a base/head git revision pair (or a | ||
| GitHub Actions event), it walks the diff, parses each changed test | ||
| file, picks the smallest set of tests to rerun, and emits a workflow | ||
| matrix plus a human-readable Markdown summary. | ||
|
|
||
| ## Building and running | ||
|
|
||
| ```sh | ||
| go build ./ | ||
| ./whichtests --help | ||
| ``` | ||
|
|
||
| Typical invocation against the local working tree: | ||
|
|
||
| ```sh | ||
| ./whichtests \ | ||
| --repo-root . \ | ||
| --base-sha origin/main \ | ||
| --head-sha HEAD \ | ||
| --out-matrix ./flake-matrix.json \ | ||
| --out-summary - | ||
| ``` | ||
|
|
||
| In GitHub Actions: | ||
|
|
||
| ```sh | ||
| go run ./ \ | ||
| --repo-root . \ | ||
| --github-actions \ | ||
| --out-matrix "$RUNNER_TEMP/flake-matrix.json" | ||
| ``` | ||
|
|
||
| For `pull_request` events, checkout must use the PR head SHA, for example `github.event.pull_request.head.sha`. The default synthetic merge ref is rejected because the checked-out `HEAD` must match `pull_request.head.sha`. | ||
|
|
||
| The matrix JSON contains `include` rows with `package`, `run_regex`, and `test_count`. `package` is normally one safe Go package pattern. If the matrix cap is hit, the final overflow row stores a space-separated list of safe package tokens in `package`, leaves `run_regex` empty, and sets `test_count` to `1`; this is the contract consumed by the current `flake-go` workflow. | ||
|
|
||
| ## File layout | ||
|
|
||
| The binary is a single `package main`, split into focused files: | ||
|
|
||
| | File | Responsibility | | ||
| | --------------- | ------------------------------------------------------------------- | | ||
| | `cli.go` | `main`, flag parsing, command orchestration (`runCommand`). | | ||
| | `config.go` | `config` / `commandConfig` types and defaults. | | ||
| | `request.go` | `runRequest`, `diffRange`, revision validation. | | ||
| | `gitexec.go` | `gitRunner` / `gitFetcher` types and the real `exec.Command` impl. | | ||
| | `diff.go` | Reading and parsing `git diff`, change kinds, hunks, line ranges. | | ||
| | `snapshot.go` | AST snapshot parsing, `fileSnapshot`, and `sharedDecl`. | | ||
| | `broadening.go` | Per-kind broadening rules (`broadeningScope`). | | ||
| | `selection.go` | Per-change selection logic (`selectChange`, broaden vs narrow). | | ||
| | `inventory.go` | `inventoryCache` for package/directory test discovery. | | ||
| | `plan.go` | Plan construction, matrix and summary rendering (`buildExecutionPlan`, `selectTestPlan`). | | ||
| | `githubactions.go` | GitHub Actions request builder and history preparation. | | ||
| | `publish.go` | Single sink for matrix and summary outputs. | | ||
|
|
||
| ## Testing | ||
|
|
||
| ```sh | ||
| go test ./... | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| package main | ||
|
|
||
| type broadeningScope uint8 | ||
|
|
||
| const ( | ||
| broadeningNone broadeningScope = iota | ||
| broadeningPackage | ||
| broadeningDirectory | ||
| ) | ||
|
|
||
| func broadeningScopeForOldHunk(decls []sharedDecl, candidate lineRange) broadeningScope { | ||
| scope := broadeningNone | ||
| for _, decl := range decls { | ||
| if !decl.Range.overlaps(candidate) { | ||
| continue | ||
| } | ||
| scope = max(scope, decl.broadeningScope()) | ||
| } | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit Small cleanups across the rewritten code, none blocking. (Style Reviewer, Modernization Reviewer, Duplication Checker, Go Architect)
|
||
| return scope | ||
| } | ||
|
|
||
| func broadeningScopeForNewHunk(decls []sharedDecl, oldSnapshot *fileSnapshot, candidate lineRange) broadeningScope { | ||
| scope := broadeningNone | ||
| for _, decl := range decls { | ||
| if !decl.Range.overlaps(candidate) { | ||
| continue | ||
| } | ||
| scope = max(scope, decl.broadeningScopeOnNewSide(oldSnapshot)) | ||
| } | ||
| return scope | ||
| } | ||
|
|
||
| func (decl sharedDecl) broadeningScope() broadeningScope { | ||
| switch decl.Kind { | ||
| case sharedDeclInit, sharedDeclTestMain: | ||
| // Go builds package and package_test files into one test binary. | ||
| // Init and TestMain changes can affect every test in the directory. | ||
| return broadeningDirectory | ||
| case sharedDeclImport, sharedDeclVar, sharedDeclConst, sharedDeclType, sharedDeclHelper: | ||
| return broadeningPackage | ||
| } | ||
| return broadeningNone | ||
| } | ||
|
|
||
| func (decl sharedDecl) broadeningScopeOnNewSide(oldSnapshot *fileSnapshot) broadeningScope { | ||
| switch decl.Kind { | ||
| // TODO: Decide whether new imports should narrow to tests that still | ||
| // reference package-local declarations. Today any import edit broadens | ||
| // the package. | ||
| case sharedDeclImport: | ||
| return broadeningPackage | ||
|
ethanndickson marked this conversation as resolved.
|
||
| case sharedDeclInit, sharedDeclTestMain: | ||
| return broadeningDirectory | ||
| case sharedDeclVar, sharedDeclConst, sharedDeclType, sharedDeclHelper: | ||
| if oldSnapshot != nil && oldSnapshot.hasSharedKey(decl.Keys) { | ||
| return broadeningPackage | ||
| } | ||
| } | ||
| return broadeningNone | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| func TestBroadeningScopeForOldHunkChoosesMaxOverlappingScope(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| data := []byte(`package sample | ||
|
|
||
| import "testing" | ||
|
|
||
| func init() { | ||
| register() | ||
| } | ||
|
|
||
| func TestAlpha(t *testing.T) {} | ||
| `) | ||
| snapshot, err := parseFileSnapshot(data) | ||
| require.NoError(t, err) | ||
| candidate := rangeSpan( | ||
| singleLineRange(t, string(data), `import "testing"`), | ||
| singleLineRange(t, string(data), "register()"), | ||
| ) | ||
| require.Equal(t, broadeningDirectory, broadeningScopeForOldHunk(snapshot.shared, candidate)) | ||
| } | ||
|
|
||
| func TestBroadeningScopeForNewHunkChoosesMaxOverlappingScope(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| data := []byte(`package sample | ||
|
|
||
| import "testing" | ||
|
|
||
| func TestMain(m *testing.M) { | ||
| m.Run() | ||
| } | ||
|
|
||
| func TestAlpha(t *testing.T) {} | ||
| `) | ||
| snapshot, err := parseFileSnapshot(data) | ||
| require.NoError(t, err) | ||
| candidate := rangeSpan( | ||
| singleLineRange(t, string(data), `import "testing"`), | ||
| singleLineRange(t, string(data), "m.Run()"), | ||
| ) | ||
| require.Equal(t, broadeningDirectory, broadeningScopeForNewHunk(snapshot.shared, nil, candidate)) | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.