Skip to content

specs: implement SPDD daily work plan 2026-07-27 (9 tasks across 5 spec files) #48447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 3 commits into from
Jul 28, 2026
Merged

specs: implement SPDD daily work plan 2026-07-27 (9 tasks across 5 spec files) #48447

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
376ecfa
Initial plan
Copilot Jul 27, 2026
191e081
chore: initial plan for daily SPDD spec work
Copilot Jul 27, 2026
3fd2678
feat(specs): implement SPDD daily work plan 2026-07-27
Copilot Jul 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/smoke-copilot-auto.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

124 changes: 66 additions & 58 deletions specs/awf-config-sources-spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,15 +40,67 @@ The following documents are authoritative and MUST be consulted together:
- `docs/authentication-architecture.md` — credential isolation architecture
- `schemas/README.md` — schema directory overview

## 3. Required coverage checks
## 3. Data Model

This section defines the canonical data entities used in the drift detection procedure. The `DriftRecord` entity is the primary structured output of drift detection.

### 3.1 DriftRecord

A `DriftRecord` represents a single detected schema drift item. All automation and agents that produce or consume drift reports **MUST** use this schema for structured drift output. See Section 7.5 for how `DriftRecord` objects are produced and consumed within the drift detection procedure.

#### 3.1.1 Formal Schema (JSON Schema)

```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "DriftRecord",
"description": "A single detected configuration drift item between gh-aw-firewall canonical sources and gh-aw implementation.",
"type": "object",
"required": ["property_path", "drift_category", "suggested_action", "detected_at"],
"properties": {
"property_path": {
"type": "string",
"description": "Dot-notation path to the drifted configuration property (e.g., 'apiProxy.anthropicAutoCache').",
"examples": ["apiProxy.anthropicAutoCache", "container.dockerHostPathPrefix"]
},
"drift_category": {
"type": "string",
"enum": ["missing_in_ghaw", "missing_in_schema", "spec_mismatch"],
"description": "Classification of the drift condition. 'missing_in_ghaw': property exists in canonical schema but gh-aw has no coverage. 'missing_in_schema': gh-aw generates a field not present in either schema. 'spec_mismatch': CLI mapping in gh-aw disagrees with the normative spec description."
},
"suggested_action": {
"type": "string",
"description": "Human-readable remediation recommendation for this drift item (e.g., 'Add coverage for apiProxy.anthropicAutoCache in pkg/workflow/ and reconcile with docs/awf-config-spec.md CLI mapping table').",
"minLength": 1
},
"detected_at": {
"type": "string",
"format": "date-time",
"description": "ISO 8601 timestamp (UTC) when this drift item was first detected in the current run."
}
},
"additionalProperties": false
}
```

#### 3.1.2 Field Reference and Implementation Sync Notes

| Property | Type | Required | Description | Implementation file |
|----------|------|----------|-------------|---------------------|
| `property_path` | `string` | **MUST** | Dot-notation config property path (e.g., `apiProxy.anthropicAutoCache`) | `pkg/workflow/awf_config_drift_formal_test.go` (`FormalDriftRecord.PropertyPath`); production emission target: `pkg/workflow/` drift detection logic |
| `drift_category` | `enum` | **MUST** | One of `missing_in_ghaw`, `missing_in_schema`, or `spec_mismatch` (see Section 7.2, Step 4) | `pkg/workflow/awf_config_drift_formal_test.go` (`FormalDriftRecord.DriftCategory`, `formalDriftCategoryExhaustiveness`); production emission target: `pkg/workflow/` drift detection logic |
| `suggested_action` | `string` | **MUST** | Actionable remediation text; **MUST NOT** be empty | `pkg/workflow/awf_config_drift_formal_test.go` (`FormalDriftRecord.SuggestedAction`, `formalDriftRecordStructuralValidity`); production emission target: `pkg/workflow/` drift detection logic |
| `detected_at` | `string` (ISO 8601) | **MUST** | UTC timestamp of detection; filesystem-safe format **SHOULD** use `YYYY-MM-DDTHH:MM:SSZ` | `pkg/workflow/awf_config_drift_formal_test.go` (`FormalDriftRecord.DetectedAt`); production emission target: `pkg/workflow/` drift detection logic |

## 4. Required coverage checks

When updating AWF config generation, schema sync, or validation in gh-aw, agents MUST verify:

1. Every relevant property in `docs/awf-config.schema.json` is represented in gh-aw logic.
2. CLI mapping behavior in `docs/awf-config-spec.md` is reconciled with schema-defined properties.
3. Config-only fields (without CLI flags) are still modeled where required by runtime behavior.

## 4. Known drift example (apiProxy)
## 5. Known drift example (apiProxy)

The following fields previously existed in schema but were missed in spec CLI mapping checks:

Expand All @@ -72,7 +124,7 @@ Agents SHOULD treat this class of mismatch as a regression signal and open a cor

---

## 5. Conformance Requirements
## 6. Conformance Requirements

The key words **MUST**, **MUST NOT**, **SHOULD**, and **MAY** in this section are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119).

Expand All @@ -92,19 +144,19 @@ The key words **MUST**, **MUST NOT**, **SHOULD**, and **MAY** in this section ar

---

## 6. Drift Detection Procedure
## 7. Drift Detection Procedure

This section describes the concrete steps for detecting schema drift between `gh-aw-firewall` and `gh-aw`.

### 6.1 When to Run
### 7.1 When to Run

Drift detection MUST be triggered when:

1. A pull request modifies `docs/awf-config.schema.json`, `src/awf-config-schema.json`, or `docs/awf-config-spec.md` in `github/gh-aw-firewall`.
2. A scheduled workflow runs the reconciliation check (RECOMMENDED: daily or weekly).
3. An agent is asked to generate or validate AWF config behavior.

### 6.2 Step-by-Step Procedure
### 7.2 Step-by-Step Procedure

1. **Fetch the canonical sources** from `github/gh-aw-firewall`:
- `docs/awf-config.schema.json` — published schema
Expand Down Expand Up @@ -132,7 +184,7 @@ Drift detection MUST be triggered when:

6. **Open a corrective PR** when any drift of category "missing in gh-aw" or "spec mismatch" is found. The PR description MUST include the drift report and reference this procedure.

### 6.3 Example Drift Check (CLI)
### 7.3 Example Drift Check (CLI)

```bash
# Requires GH_TOKEN (or GITHUB_TOKEN) with repo read access
Expand Down Expand Up @@ -164,7 +216,7 @@ rg --no-heading --no-filename --only-matching 'apiProxy\.[A-Za-z0-9_.]+' pkg/wor
diff -u /tmp/schema-keys.txt /tmp/ghaw-refs.txt || true
```

### 6.4 Automation
### 7.4 Automation

A scheduled GitHub Actions workflow in `github/gh-aw` SHOULD automate this procedure. The workflow SHOULD:

Expand All @@ -175,7 +227,7 @@ A scheduled GitHub Actions workflow in `github/gh-aw` SHOULD automate this proce

Current implementation reference: [`/.github/workflows/schema-consistency-checker.md`](../.github/workflows/schema-consistency-checker.md) (scheduled daily) is the tracked drift-detection workflow path for schema consistency checks and SHOULD include AWF config source drift checks from this section.

#### 6.4.1 Drift SLA tracking (CR-06)
#### 7.4.1 Drift SLA tracking (CR-06)

To satisfy CR-06 tracking obligations, drift escalation records SHOULD use:

Expand All @@ -198,57 +250,13 @@ To satisfy CR-06 tracking obligations, drift escalation records SHOULD use:

The scheduled schema consistency workflow SHOULD open or update one such issue when drift remains unresolved beyond 5 business days.

### 6.5 DriftRecord Entity Schema

A `DriftRecord` represents a single detected schema drift item produced by the drift detection procedure (Section 6.2, Step 5). All automation and agents that produce or consume drift reports **MUST** use this schema for structured drift output.

#### 6.5.1 Formal Schema (JSON Schema)

```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "DriftRecord",
"description": "A single detected configuration drift item between gh-aw-firewall canonical sources and gh-aw implementation.",
"type": "object",
"required": ["property_path", "drift_category", "suggested_action", "detected_at"],
"properties": {
"property_path": {
"type": "string",
"description": "Dot-notation path to the drifted configuration property (e.g., 'apiProxy.anthropicAutoCache').",
"examples": ["apiProxy.anthropicAutoCache", "container.dockerHostPathPrefix"]
},
"drift_category": {
"type": "string",
"enum": ["missing_in_ghaw", "missing_in_schema", "spec_mismatch"],
"description": "Classification of the drift condition. 'missing_in_ghaw': property exists in canonical schema but gh-aw has no coverage. 'missing_in_schema': gh-aw generates a field not present in either schema. 'spec_mismatch': CLI mapping in gh-aw disagrees with the normative spec description."
},
"suggested_action": {
"type": "string",
"description": "Human-readable remediation recommendation for this drift item (e.g., 'Add coverage for apiProxy.anthropicAutoCache in pkg/workflow/ and reconcile with docs/awf-config-spec.md CLI mapping table').",
"minLength": 1
},
"detected_at": {
"type": "string",
"format": "date-time",
"description": "ISO 8601 timestamp (UTC) when this drift item was first detected in the current run."
}
},
"additionalProperties": false
}
```

#### 6.5.2 Field Reference
### 7.5 DriftRecord Entity Schema

| Property | Type | Required | Description |
|----------|------|----------|-------------|
| `property_path` | `string` | **MUST** | Dot-notation config property path (e.g., `apiProxy.anthropicAutoCache`) |
| `drift_category` | `enum` | **MUST** | One of `missing_in_ghaw`, `missing_in_schema`, or `spec_mismatch` (see Section 6.2, Step 4) |
| `suggested_action` | `string` | **MUST** | Actionable remediation text; **MUST NOT** be empty |
| `detected_at` | `string` (ISO 8601) | **MUST** | UTC timestamp of detection; filesystem-safe format **SHOULD** use `YYYY-MM-DDTHH:MM:SSZ` |
A `DriftRecord` represents a single detected schema drift item produced by the drift detection procedure (Section 7.2, Step 5). The canonical schema definition, field reference, and implementation sync notes for `DriftRecord` are in **Section 3.1**. All automation and agents that produce or consume drift reports **MUST** use the schema defined in Section 3.1 for structured drift output.
Comment on lines +253 to +255

#### 6.5.3 Usage
#### 7.5.1 Usage

The drift detection procedure (Section 6.2, Step 5) **MUST** produce a list of zero or more `DriftRecord` objects. When any record has `drift_category` of `missing_in_ghaw` or `spec_mismatch`, the detecting automation **MUST** open a corrective PR (CR-05) and, if the SLA window is exceeded, an escalation issue (CR-06). The corrective PR description **MUST** embed the full `DriftRecord` list as JSON.
The drift detection procedure (Section 7.2, Step 5) **MUST** produce a list of zero or more `DriftRecord` objects (schema: Section 3.1). When any record has `drift_category` of `missing_in_ghaw` or `spec_mismatch`, the detecting automation **MUST** open a corrective PR (CR-05) and, if the SLA window is exceeded, an escalation issue (CR-06). The corrective PR description **MUST** embed the full `DriftRecord` list as JSON.

**Example output (Step 5 of the drift detection procedure):**

Expand All @@ -263,7 +271,7 @@ The drift detection procedure (Section 6.2, Step 5) **MUST** produce a list of z
]
```

## 7. Safeguards
## 8. Safeguards

When canonical sources in `github/gh-aw-firewall` are unavailable (GitHub outage, auth failure, transient fetch errors), agents and automation MUST apply the following safeguards:

Expand Down
28 changes: 28 additions & 0 deletions specs/compiler-threat-detection-spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,34 @@ New threat categories do not immediately become normative rules. This section de

3. **Normative stage**: The threat class is formally added to Section 5.1 and Section 8.1 via a pull request that includes: the CTR rule definition, the implementation mapping in Section 7.1, at least one test ID in Section 8.1, and a change-log entry in Section 10. The pull request **MUST** be reviewed by at least one security-focused maintainer. Once merged, the rule **MUST** be enforced by all conforming implementations. Any feature flag used during Candidate stage **MUST** be removed in the same pull request that adds the Normative definition.

### 6.6 Optimizer Failure Safeguards

The daily optimizer process (Section 6) is itself subject to failure. This section specifies normative behavior for the three principal failure modes: API unavailability, runner timeout, and rate-limit or quota exhaustion. These safeguards mirror the pattern established in the AWF Config Canonical Sources Specification §7.
Comment on lines +235 to +237

**Failure Mode 1 — API Unavailability**

When the GitHub API or any external service required by the optimizer (for example, code-scanning results, issue search, or PR listing endpoints) is unavailable during the threat-coverage check:

1. The optimizer **MUST** not emit false noop reports. When authoritative data cannot be retrieved, the optimizer **MUST** emit an `OPTIMIZER_DEGRADED` diagnostic entry in its daily output that records the failing endpoint(s), the HTTP status or error class, and the UTC timestamp of the failure.
2. The optimizer **MUST NOT** open a pull request or update spec artifacts based on incomplete threat-coverage data obtained during a degraded API run.
3. The optimizer **SHOULD** retry failed API calls with an exponential back-off policy (initial delay: 10 seconds; maximum delay: 5 minutes; maximum attempts: 3) before declaring the run degraded.

**Failure Mode 2 — Runner Timeout**

When the optimizer job is cancelled or exceeds its allotted execution time before completing the threat-coverage check:

1. The optimizer job **MUST** emit a structured `OPTIMIZER_TIMEOUT` output entry before termination, recording the last completed step and the set of CTR rules that had not yet been evaluated at the time of cancellation.
2. The optimizer **MUST NOT** produce a partial noop report or a partial PR when the timeout occurs mid-evaluation; any in-progress artifacts **MUST** be discarded.
3. The optimizer workflow **SHOULD** be configured with an explicit `timeout-minutes` value and **SHOULD** schedule a follow-up retry run within the same calendar day when a timeout is detected.

**Failure Mode 3 — Rate-Limit or Quota Exhaustion**

When the GitHub API returns secondary rate-limit (`403` with `Retry-After` header) or primary rate-limit (`429`) responses during the optimization run:

1. The optimizer **MUST** apply the `RATE_LIMIT_RETRY_CONFIG` retry policy (as defined in `actions/setup/js/error_recovery.cjs`) before emitting a terminal failure.
2. If all retries are exhausted and the rate limit is not recovered, the optimizer **MUST** emit an `OPTIMIZER_RATE_LIMITED` diagnostic entry recording the affected endpoints and the `Retry-After` or `x-ratelimit-reset` value.
3. The optimizer **MUST NOT** count a rate-limited run as a completed threat-coverage cycle; the run **MUST** be retried in the next scheduled window.

---

## 7. Implementation Mapping
Expand Down
1 change: 1 addition & 0 deletions specs/otel-observability-spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -873,6 +873,7 @@ attribute-level checks mapped directly to Section 10 requirements:
| **T-OT-008** | §10.2 `gh-aw.job.name` on built-in job spans | The setup span includes `gh-aw.job.name` equal to the GitHub Actions job name | Decode `/tmp/gh-aw/otel.jsonl` or captured OTLP payloads and assert the setup span attribute value exactly matches the workflow job name |
| **T-OT-009** | §10.3 `gen_ai.system` on built-in agent spans | A known engine emits a non-empty normalized provider/system value | Run a workflow with a built-in agent span, decode exported spans, and assert `gen_ai.system` is present whenever the engine mapping is known |
| **T-OT-010** | §13.3 `gh-aw.outcome.type` on outcome-evaluation spans | Outcome-evaluation spans emit the safe-output type being evaluated | Execute the outcome collector, inspect the `gh-aw.outcome.evaluate` span, and assert `gh-aw.outcome.type` matches the evaluated manifest item type |
| **T-OT-011** | §17 Backward-compatibility: v0.3.0 fields present in v0.4.0 export | All span attribute keys that were present in v0.3.0 (`gh-aw.*`, `github.*`, `gen_ai.system`, `gen_ai.usage.total_tokens`, `OTEL_EXPORTER_OTLP_ENDPOINT`, built-in setup/conclusion span names) **MUST** also be present in a v0.4.0 export at the same semantic positions | Produce a v0.4.0 compliant export from the current implementation; compare the set of attribute keys against the v0.3.0 attribute inventory (defined in Version 0.3.0 change-log entry in Section 19); assert no v0.3.0 attribute has been removed or renamed; assert no built-in span name has changed |


### 17.2 Optional Extension Tests
Expand Down
Loading