Skip to content

docs(readme): add parallel WAA evaluation, fix build badge#19

Merged
abrichr merged 2 commits intomainfrom
feature/readme-parallel-waa
Feb 5, 2026
Merged

docs(readme): add parallel WAA evaluation, fix build badge#19
abrichr merged 2 commits intomainfrom
feature/readme-parallel-waa

Conversation

@abrichr
Copy link
Member

@abrichr abrichr commented Feb 5, 2026

Summary

  • Fix broken build status badge (referenced publish.yml but file is release.yml)
  • Add prominent "Parallel WAA Benchmark Evaluation" section near the top highlighting the new v0.3.0 feature
  • Add detailed "WAA Benchmark Workflow" section (refactor(cli): minimal WAA CLI with vanilla image support #14) with complete setup instructions

Changes

Badge Fix

The build status badge was showing a broken image because it referenced a non-existent workflow file.

New Documentation

Near the top (high visibility):

  • Quick start commands for pool-create, pool-wait, pool-run
  • Performance estimates table (1 worker vs 5 vs 10)
  • Key features summary

Detailed workflow section (#14):

  • Prerequisites (Azure CLI, API keys, quota)
  • Single VM workflow for quick testing
  • Parallel pool workflow for full evaluations
  • VNC access instructions for viewing Windows VMs
  • Architecture diagram
  • Cost estimates table

Test plan

  • Verify badge displays correctly on GitHub
  • Verify anchor link from top section works
  • Review documentation accuracy

🤖 Generated with Claude Code

- Fix broken build badge (publish.yml → release.yml)
- Add prominent "Parallel WAA Benchmark Evaluation" section near top
- Add detailed "WAA Benchmark Workflow" section (#14) with:
  - Single VM and parallel pool workflows
  - VNC access instructions
  - Architecture diagram
  - Cost estimates
- Update section numbering (Limitations → 15, Roadmap → 16)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Member Author

@abrichr abrichr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-Review

✅ What's Good

  1. Badge fix is correct - publish.ymlrelease.yml matches the actual workflow file
  2. Prominent placement - New parallel WAA section at top gives immediate visibility to the key new feature
  3. Complete workflow documentation - Section 14 covers prerequisites, single VM, parallel pool, VNC access, architecture, and costs
  4. Accurate commands - All CLI commands match the actual implementation
  5. Helpful performance table - Gives users clear expectations for time savings

⚠️ Suggestions for Improvement

  1. Anchor placement - The <a id="waa-benchmark-workflow"></a> is after the heading. Move it before for proper anchor behavior:

    <a id="waa-benchmark-workflow"></a>
    ## 14. WAA Benchmark Workflow
  2. Missing pool-delete command - Section 14.3 references pool-delete -y but this command may not exist yet (need to verify). Should either:

    • Add the command to CLI if missing
    • Change to existing cleanup approach
  3. VNC tunnel ports - The manual SSH tunnel examples show hardcoded IPs (<worker-0-ip>). Could mention how to get IPs:

    # Get worker IPs from pool-status
    uv run python -m openadapt_ml.benchmarks.cli pool-status
  4. Cost estimate accuracy - The $0.38/hr is approximate. Consider adding "prices vary by region" caveat.

  5. Missing error handling docs - No mention of what to do if pool-create fails mid-way (use pool-cleanup?)

🔍 Verification Needed

  • Confirm pool-delete command exists
  • Test anchor link from top section actually navigates to section 14
  • Verify badge now renders correctly on GitHub

Verdict

Approve with minor suggestions - The core documentation is valuable and accurate. The suggestions above are enhancements, not blockers.

- Fix anchor placement (move before heading for proper navigation)
- Correct pool-delete → pool-cleanup (actual command name)
- Add pool-status example for getting worker IPs
- Add "prices vary by region" caveat

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@abrichr
Copy link
Member Author

abrichr commented Feb 5, 2026

Fixed in Latest Push (5d3f4f8)

Addressed the self-review feedback:

  • Anchor placement - Moved <a id="waa-benchmark-workflow"></a> before the heading
  • Command name - Changed pool-delete -ypool-cleanup (the actual command)
  • Worker IPs - Added pool-status example before SSH tunnel commands
  • Price caveat - Added "Prices vary by Azure region" note

Ready for final review.

@abrichr abrichr merged commit fea0a10 into main Feb 5, 2026
0 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant