chore: add flakinmess#356
Merged
Merged
Conversation
Contributor
Reviewer's guide (collapsed on small PRs)Reviewer's GuideAdds pytest-flakiness support to the Python CI workflow by scoping OIDC permissions and flakiness-related environment to the test job only, and updates the architecture docs to describe the new permissions model for the test matrix. Sequence diagram for test job running pytest with pytest-flakiness via GitHub OIDCsequenceDiagram
actor Developer
participant GitHubActions as GitHubActionsWorkflow
participant TestJob as PythonTestJob
participant Pytest as pytest
participant Flakiness as pytest_flakiness
participant OIDC as GitHubOIDC
participant Service as FlakinessService
Developer->>GitHubActions: push to repository
GitHubActions->>TestJob: start test job (id-token: write, FLAKINESS_PROJECT)
TestJob->>Pytest: run pytest
Pytest->>Flakiness: load pytest-flakiness plugin
Flakiness->>OIDC: request id-token for FLAKINESS_PROJECT
OIDC-->>Flakiness: return OIDC id-token
Flakiness->>Service: submit flakiness report with id-token
Service-->>Flakiness: acknowledge report
Flakiness-->>Pytest: mark test results with flakiness data
Pytest-->>TestJob: exit with test and flakiness results
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Contributor
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- Consider deriving
FLAKINESS_PROJECTfrom the repository context (e.g.,github.repository) rather than hardcodingvinitkumar/json2xmlso the workflow stays accurate if the repo is renamed or forked. - The architecture note says the test job "scopes
id-token: writeandFLAKINESS_PROJECT" to the pytest matrix, butFLAKINESS_PROJECTis an environment variable rather than a permission; you might reword that sentence to avoid conflating env vars with permissions.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Consider deriving `FLAKINESS_PROJECT` from the repository context (e.g., `github.repository`) rather than hardcoding `vinitkumar/json2xml` so the workflow stays accurate if the repo is renamed or forked.
- The architecture note says the test job "scopes `id-token: write` and `FLAKINESS_PROJECT`" to the pytest matrix, but `FLAKINESS_PROJECT` is an environment variable rather than a permission; you might reword that sentence to avoid conflating env vars with permissions.
## Individual Comments
### Comment 1
<location path="lat.md/architecture.md" line_range="79" />
<code_context>
+The `.github/workflows/` files declare the minimum `permissions:` scopes needed by each workflow, with CodeQL retaining `security-events: write` for result upload and TestPyPI retaining `id-token: write` for explicit trusted-publishing runs. The Python test job also scopes `id-token: write` and `FLAKINESS_PROJECT` to its pytest matrix so the flakiness reporter can authenticate through GitHub OIDC without exposing that permission to lint. Release-branch pushes build distributions and run Twine checks; TestPyPI upload is a manual opt-in because that external registry requires separate publisher configuration. Action references are pinned to immutable commits with the upstream tag retained in comments for reviewability, and `.github/dependabot.yml` checks the `github-actions` ecosystem weekly so those pins do not silently age. The Python test matrix pins its PyPy 3.11 job to an explicit PyPy release so CI keeps exercising the intended CPython 3.11.15-compatible runtime instead of silently drifting with runner cache updates. It also exercises regular CPython 3.15.0b3 while leaving that beta's free-threaded builds out of CI until the runner support is less brittle.
</code_context>
<issue_to_address>
**issue (typo):** Fix the subject–verb agreement in the phrase about release-branch pushes and Twine checks.
In the sentence beginning "Release-branch pushes build distributions and run Twine checks", "pushes" is singular while "run" is plural. Please align the verbs, e.g. "Release-branch pushes build distributions and runs Twine checks."
```suggestion
The `.github/workflows/` files declare the minimum `permissions:` scopes needed by each workflow, with CodeQL retaining `security-events: write` for result upload and TestPyPI retaining `id-token: write` for explicit trusted-publishing runs. The Python test job also scopes `id-token: write` and `FLAKINESS_PROJECT` to its pytest matrix so the flakiness reporter can authenticate through GitHub OIDC without exposing that permission to lint. Release-branch pushes build distributions and runs Twine checks; TestPyPI upload is a manual opt-in because that external registry requires separate publisher configuration. Action references are pinned to immutable commits with the upstream tag retained in comments for reviewability, and `.github/dependabot.yml` checks the `github-actions` ecosystem weekly so those pins do not silently age. The Python test matrix pins its PyPy 3.11 job to an explicit PyPy release so CI keeps exercising the intended CPython 3.11.15-compatible runtime instead of silently drifting with runner cache updates. It also exercises regular CPython 3.15.0b3 while leaving that beta's free-threaded builds out of CI until the runner support is less brittle.
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #356 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 7 7
Lines 833 833
=========================================
Hits 833 833
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by Sourcery
Integrate pytest-flakiness reporting into the Python test workflow while tightening GitHub Actions permissions and documenting the new CI scope.
Enhancements:
Documentation: