Skip to content

docs: Add CI/CD integration guide (GitHub Actions, Azure DevOps)#100

Open
spboyer wants to merge 2 commits intomainfrom
docs/ci-integration-guide
Open

docs: Add CI/CD integration guide (GitHub Actions, Azure DevOps)#100
spboyer wants to merge 2 commits intomainfrom
docs/ci-integration-guide

Conversation

@spboyer
Copy link
Member

@spboyer spboyer commented Mar 5, 2026

Closes #89

Adds CI/CD integration guide to the GitHub Pages docs site.

Replaces #97 (closed — had leaked .squad/ files).

@spboyer spboyer requested a review from chlowell as a code owner March 5, 2026 16:01
Copilot AI review requested due to automatic review settings March 5, 2026 16:01
@github-actions github-actions bot enabled auto-merge (squash) March 5, 2026 16:01
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces #97 (which had leaked .squad/ files) and adds a comprehensive CI/CD integration guide to the waza docs site at site/src/content/docs/guides/ci-cd.mdx. The guide covers installing waza in CI, running evaluations in GitHub Actions, Azure DevOps, and GitLab CI, along with best practices and troubleshooting tips.

Changes:

  • Rewrites the existing ci-cd.mdx guide from a simple GitHub Actions-focused page into a comprehensive multi-platform CI/CD integration guide covering GitHub Actions, Azure DevOps, and GitLab CI
  • Adds sections for token budget checks, caching, path filtering, advanced workflows (baseline comparison, multi-stage pipelines), and troubleshooting
  • Adds tabbed UI components for cross-platform secret configuration examples
Comments suppressed due to low confidence (2)

site/src/content/docs/guides/ci-cd.mdx:267

  • This uses waza tokens diff with non-existent --ref1 and --ref2 flags. The correct command is waza tokens compare, which takes positional arguments: waza tokens compare origin/main HEAD --format json > token-diff.json. Additionally, the jq '.exceeded' check is wrong — the JSON output has no top-level exceeded field. The correct check would be jq '.summary.exceededCount' (an integer, not a boolean), so the condition should be something like if [ "$EXCEEDED" -gt 0 ].
```yaml
- script: |
    waza tokens diff \
      --ref1 origin/main \
      --ref2 HEAD \
      --format json > token-diff.json
    
    EXCEEDED=$(jq '.exceeded' token-diff.json)
    if [ "$EXCEEDED" == "true" ]; then
      echo "##vso[task.logissue type=error;]Token budget exceeded"
      exit 1
    fi
  displayName: 'Check token budget'
**site/src/content/docs/guides/ci-cd.mdx:379**
* Same `waza tokens diff` issue here — the correct command is `waza tokens compare --strict`.
- name: Check token budget
  run: waza tokens diff --strict
</details>

spboyer and others added 2 commits March 5, 2026 12:46
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix waza tokens diff -> waza tokens compare (3 occurrences)
- Fix JSON field paths (results.passed -> results.summary.succeeded)
- Fix waza compare example (takes JSON files, not YAML)
- Remove incorrect Windows PowerShell install example
- Fix waza tokens check to use directory path not eval YAML
- Remove unnecessary Python dependency from Azure DevOps example
- Remove unused Aside import

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@spboyer spboyer force-pushed the docs/ci-integration-guide branch from ca73446 to ccea6a2 Compare March 5, 2026 17:46
Copilot AI review requested due to automatic review settings March 5, 2026 17:46
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

chlowell pushed a commit to chlowell/waza that referenced this pull request Mar 5, 2026
Closes microsoft#98

Implements the behavior grader type for validating agent behavior
metrics (tool usage, token consumption, forbidden/required tools,
duration limits).

## Changes
- New `internal/graders/behavior_grader.go` with BehaviorGrader
- Added `TypeBehavior` constant and factory registration
- Extended `graders.Context` with `Session` field for SessionDigest data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: Add CI/CD integration guide (GitHub Actions, Azure DevOps)

2 participants