-
Notifications
You must be signed in to change notification settings - Fork 0
docs: harden OSS repo posture and add OMO setup guidance #33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
bd01975
docs: add MAINTAINERS.md with roles and triage expectations
aryeko ec0fbbb
docs: add OMO setup guide with tmux and category config
aryeko 781e928
chore(repo): add license and baseline trust signals
aryeko b3ddc24
docs(repo): strengthen positioning and publication governance
aryeko b762ba5
docs(repo): strengthen positioning and publication governance
aryeko 6a29ad3
docs(repo): strengthen positioning and publication governance
aryeko baef7ae
docs(repo): improve docs IA and cross-link integrity
aryeko 95f56ff
docs: add rollout checklist and completion summary
aryeko 70d5e7e
chore(git): ignore local sisyphus state files
aryeko aa91455
docs: remove OMO setup guide and references
aryeko File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,3 +13,6 @@ coverage.out | |
|
|
||
| # Local worktrees | ||
| .worktrees/ | ||
|
|
||
| # Local OMO planning/state artifacts | ||
| .sisyphus/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| MIT License | ||
|
|
||
| Copyright (c) 2026 go-modkit authors | ||
|
|
||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
|
|
||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
|
|
||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| # Maintainers | ||
|
|
||
| This document outlines the maintainership roles, triage expectations, and response windows for the `benchmarks` repository. | ||
|
|
||
| ## Roles | ||
|
|
||
| - **Lead Maintainers**: Responsible for the overall direction, architecture, and final approval of major changes. | ||
| - **Maintainers**: Responsible for reviewing pull requests, triaging issues, and maintaining specific components (e.g., parity runner, benchmark scripts). | ||
| - **Triage**: Responsible for initial issue screening, labeling, and routing questions to [Discussions](https://github.com/go-modkit/benchmarks/discussions). | ||
|
|
||
| ## Triage Expectations | ||
|
|
||
| We aim to provide timely feedback on community contributions while balancing other project priorities. | ||
|
|
||
| | Activity | Initial Response Window | Goal | | ||
| |----------|-------------------------|------| | ||
| | **New Issues** | 2-3 business days | Acknowledge, label, and request missing info. | | ||
| | **Pull Requests** | 3-5 business days | Provide initial review or feedback. | | ||
| | **Discussions** | Best effort | Community-driven; maintainers participate as time permits. | | ||
|
|
||
| *Note: Response windows are targets, not guarantees. We appreciate your patience.* | ||
|
|
||
| ## Escalation Paths | ||
|
|
||
| ### Security Vulnerabilities | ||
| Do not report security vulnerabilities via public issues. Follow the [Security Policy](SECURITY.md) for private disclosure. | ||
|
|
||
| ### Urgent Issues | ||
| For critical regressions or build failures impacting the main branch, please mention a maintainer in the relevant issue or PR. | ||
|
|
||
| ## Governance | ||
|
|
||
| Decisions are made through consensus among maintainers. For major architectural changes, a design document (RFC) should be submitted to `docs/design/` for review. | ||
|
|
||
| See [CONTRIBUTING.md](CONTRIBUTING.md) for the technical contribution process. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # OSS Hardening and OMO Enablement - Rollout Checklist | ||
|
|
||
| ## Summary | ||
| All 16 tasks completed successfully. Work is ready for PR. | ||
|
|
||
| ## Branch | ||
| - **Branch**: feat/oss-hardening-and-omo-enablement | ||
| - **Worktree**: .worktrees/feat-oss-hardening/ | ||
| - **Commits**: 7 commits ahead of main | ||
|
|
||
| ## Files Changed | ||
| 1. LICENSE (new) - MIT License | ||
| 2. MAINTAINERS.md (new) - Governance doc | ||
| 3. README.md - Badge row, modkit narrative, trust section | ||
| 4. docs/guides/AGENTS.md - Guide index updates | ||
| 5. docs/guides/benchmark-publication-policy.md (new) - Publication policy | ||
| 6. docs/guides/repository-metadata.md (new) - Metadata profile | ||
|
|
||
| ## Suggested PR Sequence | ||
|
|
||
| ### PR 1: Legal and Baseline Trust | ||
|
|
||
|
|
||
| ### PR 2: Governance and Positioning | ||
|
|
||
|
|
||
| ### PR 3: OMO Setup Guide | ||
|
|
||
|
|
||
| ## Post-Merge Actions | ||
| 1. Apply GitHub metadata via commands in docs/guides/repository-metadata.md | ||
| 2. Verify badges render correctly on README | ||
| 3. Confirm docs/guides navigation works | ||
|
|
||
| ## Evidence | ||
| See .sisyphus/evidence/task-8-final-gate.txt for verification output. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # Benchmark Publication and Reproducibility Policy | ||
|
|
||
| This document defines the minimum disclosure requirements and operational bounds for publishable benchmark results generated by this harness. Adherence to this policy ensures that published claims are reproducible, statistically sound, and fair. | ||
|
|
||
| ## Minimum Disclosure Requirements | ||
|
|
||
| Any public claim or report referencing benchmarks from this repository must include or link to the following artifacts from the `results/latest/` directory: | ||
|
|
||
| ### 1. Environment Manifest and Fingerprint | ||
| - **Artifacts**: `environment.manifest.json`, `environment.fingerprint.json` | ||
| - **Requirement**: Disclosure of the runner metadata, toolchain versions (Go, Python, Docker), and host environment characteristics. | ||
| - **Purpose**: Ensures results can be contextualized against the hardware and software stack used during the run. | ||
|
|
||
| ### 2. Schema Validation | ||
| - **Requirement**: All artifacts must pass `make benchmark-schema-validate`. | ||
| - **Purpose**: Guarantees that the raw metrics and summaries adhere to the versioned data contract. | ||
|
|
||
| ### 3. Quality Policy Compliance | ||
| - **Requirement**: The run must pass `make ci-benchmark-quality-check` as defined in `stats-policy.yaml`. | ||
| - **Metrics**: Must include at least RPS (median) and Latency (P50, P95, P99). | ||
| - **Variance**: Coefficient of Variation (CV) must stay within the thresholds defined in `stats-policy.yaml` (e.g., <10% for RPS). | ||
|
|
||
| ### 4. Parity and Skip Transparency | ||
| - **Requirement**: Any target implementation that failed the parity gate must be disclosed as "Skipped (Parity Failure)". | ||
| - **Purpose**: Prevents "cherry-picking" results from implementations that do not yet meet the functional contract. | ||
|
|
||
| ## Operational Bounds | ||
|
|
||
| To maintain comparability and prevent resource exhaustion in CI/CD environments, publishable runs must stay within the following bounds (aligned with `benchmark-manual` workflow): | ||
|
|
||
| | Parameter | Range | Default | | ||
| |-----------|-------|---------| | ||
| | **Runs per target** | 1 .. 10 | 3 | | ||
| | **Requests per run** | 50 .. 1000 | 300 | | ||
| | **Frameworks** | Subset of `modkit,nestjs,baseline,wire,fx,do` | - | | ||
|
|
||
| Runs exceeding these bounds are considered "Experimental" and should not be used for official framework comparisons or public performance claims. | ||
|
|
||
| ## Reproducibility Workflow | ||
|
|
||
| To generate a publishable report, follow the standard workflow: | ||
|
|
||
| 1. **Clean Environment**: Ensure a clean working tree and stable runtime versions. | ||
| 2. **Execution**: | ||
| ```bash | ||
| make benchmark | ||
| ``` | ||
| 3. **Validation**: | ||
| ```bash | ||
| make benchmark-schema-validate | ||
| make ci-benchmark-quality-check | ||
| ``` | ||
| 4. **Reporting**: | ||
| ```bash | ||
| make report | ||
| ``` | ||
|
|
||
| The resulting `results/latest/report.md` and `results/latest/summary.json` are the authoritative sources for publication. | ||
|
|
||
| ## Fairness and Interpretation | ||
|
|
||
| - **Fairness**: All runs must adhere to the principles in [METHODOLOGY.md](../../METHODOLOGY.md). | ||
| - **Disclaimer**: Every published report must include the fairness disclaimer regarding cross-language comparisons and runtime effects. | ||
| - **Correctness First**: Performance data from a run with parity failures is invalid for comparative claims. | ||
|
|
||
| ## Policy Enforcement | ||
|
|
||
| The `make publication-sync-check` command (and its CI equivalent) verifies that published artifacts align with the current methodology and versioned policy. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # Repository Metadata Profile | ||
|
|
||
| This document defines the GitHub repository metadata for the `benchmarks` project. Maintaining accurate metadata ensures discoverability and clear positioning within the `go-modkit` ecosystem. | ||
|
|
||
| ## Metadata Profile | ||
|
|
||
| | Field | Value | | ||
| |-------|-------| | ||
| | **Description** | Parity-gated benchmark harness for API framework performance comparison. Ensures correctness via declarative contract fixtures before measuring throughput and latency. | | ||
| | **Topics** | `go`, `benchmark`, `api-parity`, `performance-testing`, `reproducibility`, `modkit`, `framework-comparison`, `api-contract`, `docker-orchestration`, `quality-gates` | | ||
| | **Homepage** | `https://github.com/go-modkit/benchmarks` | | ||
|
|
||
| ## Application Checklist | ||
|
|
||
| ### Automated (via GitHub CLI) | ||
|
|
||
| If you have the `gh` CLI installed and authenticated, run the following command to apply the profile: | ||
|
|
||
| ```bash | ||
| gh repo edit go-modkit/benchmarks \ | ||
| --description "Parity-gated benchmark harness for API framework performance comparison. Ensures correctness via declarative contract fixtures before measuring throughput and latency." \ | ||
| --add-topic "go,benchmark,api-parity,performance-testing,reproducibility,modkit,framework-comparison,api-contract,docker-orchestration,quality-gates" \ | ||
| --homepage "https://github.com/go-modkit/benchmarks" | ||
| ``` | ||
|
|
||
| ### Manual Fallback | ||
|
|
||
| If the GitHub CLI is unavailable, follow these steps: | ||
|
|
||
| 1. Navigate to the repository on GitHub: [go-modkit/benchmarks](https://github.com/go-modkit/benchmarks) | ||
| 2. Click on the **Settings** gear icon (or the "About" section edit icon on the main page). | ||
| 3. In the **Description** field, paste: | ||
| > Parity-gated benchmark harness for API framework performance comparison. Ensures correctness via declarative contract fixtures before measuring throughput and latency. | ||
| 4. In the **Website** field, paste: | ||
| > https://github.com/go-modkit/benchmarks | ||
| 5. In the **Topics** section, add the following tags one by one: | ||
| - `go` | ||
| - `benchmark` | ||
| - `api-parity` | ||
| - `performance-testing` | ||
| - `reproducibility` | ||
| - `modkit` | ||
| - `framework-comparison` | ||
| - `api-contract` | ||
| - `docker-orchestration` | ||
| - `quality-gates` | ||
| 6. Click **Save changes**. | ||
|
|
||
| ## Verification | ||
|
|
||
| To verify the current metadata, run: | ||
|
|
||
| ```bash | ||
| gh repo view go-modkit/benchmarks --json description,repositoryTopics,homepage | ||
| ``` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.