Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 95 additions & 1 deletion .github/workflows/q.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,45 @@ This workflow was triggered from a comment on discussion #${{ github.event.discu
- Is this a general optimization request?
3. **Identify Target Workflows**: Determine which workflows to analyze (specific ones or all)

#### Cache Initialization

The cache root for this workflow is `/tmp/gh-aw/cache-memory` (restored by the `cache-memory: true` setup step).
Q's state lives in a workflow-specific subdirectory: `/tmp/gh-aw/cache-memory/q/`.

Load prior state from cache at startup so this run can build on previous findings:

```bash
# Cache root: /tmp/gh-aw/cache-memory (restored by the cache-memory setup step)
# Q-specific state lives in the /q/ subdirectory inside that root
CACHE_DIR=/tmp/gh-aw/cache-memory/q
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot validate the cache path matches the actual cache folder location

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validated in commit 85cfe1d. The compiled lock file confirms the cache save/restore operates on path: /tmp/gh-aw/cache-memory (the root). The prompt now uses /tmp/gh-aw/cache-memory/q/ — a workflow-specific subdirectory nested inside that root, which follows the same convention as daily-compiler-quality (/tmp/gh-aw/cache-memory/compiler-quality/). Both the Phase 0 and Phase 5.5 snippets now include inline comments making this relationship explicit.

mkdir -p "$CACHE_DIR"

if [ -f "$CACHE_DIR/state.json" ]; then
echo "Cache restored — loading prior Q state"
cat "$CACHE_DIR/state.json" | jq '{workflows_analyzed: (.workflows_analyzed | length), patterns_count: (.patterns | length), last_run: .last_run}'
Copy link

Copilot AI Apr 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the cache-restore log summary, workflows_analyzed is being replaced with a count (length) while patterns_count is explicitly named as a count. To avoid confusing the meaning of workflows_analyzed (array in state.json vs. number in this output), rename the output field to something like workflows_analyzed_count (or similar).

Suggested change
cat "$CACHE_DIR/state.json" | jq '{workflows_analyzed: (.workflows_analyzed | length), patterns_count: (.patterns | length), last_run: .last_run}'
cat "$CACHE_DIR/state.json" | jq '{workflows_analyzed_count: (.workflows_analyzed | length), patterns_count: (.patterns | length), last_run: .last_run}'

Copilot uses AI. Check for mistakes.
else
echo "No prior cache found — starting fresh"
fi
```

The `state.json` file (if present) has the following shape:

```json
{
"last_run": "2026-04-25-14-30-00",
"workflows_analyzed": ["workflow-a", "workflow-b"],
"patterns": [
{"name": "pattern-name", "description": "brief description", "workflows": ["wf-a", "wf-b"]}
],
"processed_run_ids": ["24929839999", "24928863169"]
}
```

Use this state to:
- Skip re-analysis of workflows that were already optimized recently
- Reuse identified patterns to accelerate the current session
- Determine which run IDs have already been reviewed so you focus on new data

### Phase 1: Gather Live Data

**NEVER EVER make up logs or data - always pull from live sources.**
Expand Down Expand Up @@ -174,7 +213,7 @@ Use internal resources to research solutions:

1. **Repository Documentation**: Read documentation files in `docs/` to understand best practices
2. **Workflow Examples**: Examine successful workflows in `.github/workflows/` as reference
3. **Cache Memory**: Check cache-memory for patterns and solutions from previous analyses
3. **Cache Memory**: Check `/tmp/gh-aw/cache-memory/q/state.json` for patterns and solutions from previous analyses — if the file exists, load it and reuse known patterns to avoid duplicate research
4. **GitHub Issues**: Search closed issues for similar problems and their resolutions

### Phase 4: Workflow Improvements
Expand Down Expand Up @@ -256,6 +295,61 @@ General optimizations:
3. **Validate Syntax**: Confirm the workflow is syntactically correct
4. **Review Generated YAML**: Check that .lock.yml files are properly generated

### Phase 5.5: Persist Cache State

**ALWAYS** write updated state to cache-memory before creating the PR (or before finishing if no PR is needed). This ensures future runs start warm and avoid re-scanning the same logs.

The cache save step persists the entire `/tmp/gh-aw/cache-memory` tree, so files written to `/tmp/gh-aw/cache-memory/q/` are automatically included.

```bash
# Cache root: /tmp/gh-aw/cache-memory (saved by the cache-memory save step)
# Q-specific state lives in the /q/ subdirectory inside that root
CACHE_DIR=/tmp/gh-aw/cache-memory/q
mkdir -p "$CACHE_DIR"

# Build the new state.json — use filesystem-safe timestamp (no colons)
NOW=$(date -u +"%Y-%m-%d-%H-%M-%S")

# Load previously processed run IDs from existing state so we can merge them
PREV_RUN_IDS="[]"
PREV_WORKFLOWS="[]"
PREV_PATTERNS="[]"
if [ -f "$CACHE_DIR/state.json" ]; then
PREV_RUN_IDS=$(jq '.processed_run_ids // []' "$CACHE_DIR/state.json")
PREV_WORKFLOWS=$(jq '.workflows_analyzed // []' "$CACHE_DIR/state.json")
PREV_PATTERNS=$(jq '.patterns // []' "$CACHE_DIR/state.json")
fi

# Write merged state — THIS_RUN_* variables must be set by the agent
# to JSON arrays, e.g. THIS_RUN_WORKFLOWS='["workflow-a","workflow-b"]'
jq -n \
--arg now "$NOW" \
--argjson prev_wf "$PREV_WORKFLOWS" \
--argjson new_wf "${THIS_RUN_WORKFLOWS:-[]}" \
--argjson prev_pat "$PREV_PATTERNS" \
--argjson new_pat "${THIS_RUN_PATTERNS:-[]}" \
--argjson prev_ids "$PREV_RUN_IDS" \
--argjson new_ids "${THIS_RUN_RUN_IDS:-[]}" \
'{
last_run: $now,
workflows_analyzed: ($prev_wf + $new_wf | unique),
patterns: ($prev_pat + $new_pat | unique_by(.name)),
Copy link

Copilot AI Apr 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

patterns: ($prev_pat + $new_pat | unique_by(.name)) does not actually merge pattern content across runs: if a pattern with the same name appears again (e.g., with additional workflows), unique_by keeps only one copy and can drop the new data. If you want patterns to accumulate, consider grouping by .name and merging fields (at least unioning the workflows arrays, and deciding whether to prefer the newest description).

Suggested change
patterns: ($prev_pat + $new_pat | unique_by(.name)),
patterns: (
($prev_pat + $new_pat)
| group_by(.name)
| map({
name: .[0].name,
description: (map(.description // empty) | last // ""),
workflows: (map(.workflows // []) | add | unique)
})
),

Copilot uses AI. Check for mistakes.
processed_run_ids: ($prev_ids + $new_ids | unique)
}' > "$CACHE_DIR/state.json"

echo "Cache state saved to $CACHE_DIR/state.json"
ls -lh "$CACHE_DIR/"
```

**What to include in the saved state — populate the shell variables before running the snippet above:**

- `THIS_RUN_WORKFLOWS`: JSON array of workflow names examined this run, e.g. `'["q","archie"]'`
- `THIS_RUN_PATTERNS`: JSON array of patterns found, e.g. `'[{"name":"missing-cache-write","description":"workflow enables cache-memory but never writes to it","workflows":["q"]}]'`
- `THIS_RUN_RUN_IDS`: JSON array of run IDs you pulled logs for, e.g. `'["24929839999","24928863169"]'`
- `last_run`: Filesystem-safe UTC timestamp (`YYYY-MM-DD-HH-MM-SS`) — no colons
Copy link

Copilot AI Apr 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list of variables to populate includes last_run, but the snippet sets NOW=... and writes last_run: $now inside state.json. As written, there is no last_run shell variable for the agent to populate—this bullet should refer to NOW (the shell variable) or to the last_run JSON field, otherwise it’s misleading.

Suggested change
- `last_run`: Filesystem-safe UTC timestamp (`YYYY-MM-DD-HH-MM-SS`) — no colons
- `NOW`: Filesystem-safe UTC timestamp (`YYYY-MM-DD-HH-MM-SS`) — no colons; this is written to the `last_run` field in `state.json`

Copilot uses AI. Check for mistakes.

The `jq` merge step deduplicates across runs so history accumulates without growing unbounded.
Copy link

Copilot AI Apr 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The text says the merge "accumulates history across runs without growing unbounded", but processed_run_ids (and potentially workflows_analyzed) will still grow indefinitely as new unique values appear over time. If bounding size is a goal, add an explicit retention policy (e.g., keep only the most recent N run IDs / patterns, or prune by age using last_run).

Copilot uses AI. Check for mistakes.

### Phase 6: Create Pull Request (Only if Changes Exist)

**IMPORTANT**: Only create a pull request if you have made actual changes to workflow files. If no changes are needed, explain your findings in a comment instead.
Expand Down
Loading