ARGUS

Autonomous visual QA agent for the Global AI Hackathon with Qwen Cloud — Track 4: Autopilot Agent.

ARGUS explores any web app the way a human QA tester would — it sees the screen, decides what to test, clicks, types, and judges whether what happened was a bug. It confirms every bug by replaying it, files a structured report with a video and a runnable repro script, and grows a regression suite as it goes. Named for the hundred-eyed watchman of Greek myth: nothing on the screen escapes it.

---

The problem

Every 2026 post-mortem on agentic AI says the same thing: agents don't fail because models are weak, they fail at the system layer — compound step failure, context drift, tool-call cascades, no observability, no recovery. Gartner projects 40%+ of agentic projects get scrapped by 2027 for exactly this. Manual QA, meanwhile, is slow, expensive, and never covers enough. ARGUS attacks both: a genuinely useful autonomous QA system and a showcase of how to engineer a long-horizon agent that survives contact with the real world.

What makes it a real system (not a demo)

• Vision-grounded action — it acts on what it sees (pixel grounding), not brittle DOM crawling. This is Qwen's headline capability and almost no competing team will use it.
• Reliability-engineered — durable execution with per-step checkpointing, action verification, a reproduction gate that kills false positives, full reasoning traces, and human gates on destructive actions.
• Coverage-driven — a frontier priority queue + screen-novelty scoring, so exploration is an algorithm with a measurable coverage number, not random clicking.
• MCP-native — exposed as an MCP server (trigger a QA sweep from any client) and consumes GitHub/filesystem as MCP tools. Directly hits the rubric's "sophisticated use of QwenCloud APIs (custom skills, MCP integrations)."
• Self-growing — every confirmed bug's repro script accumulates into a Playwright suite ARGUS re-runs on the next commit. One-shot QA becomes continuous QA.

Architecture at a glance

A mission charter feeds a checkpointed agent loop. Each step: perceive (screenshot + DOM + console + network) -> plan (long-horizon strategy) -> ground + act (pixel target -> Playwright) -> judge (oracle ensemble) -> reproduce + report. The loop repeats until a coverage target is met. Every model call flows through a single model gateway (routing, retries, fallback, caching, budget governor). State lives in Postgres (graph + bugs), Tair/Redis (checkpoints + queue), and OSS (evidence).

See docs/ARCHITECTURE.md and the system diagram in docs/diagram.svg for the full layered view.

Qwen Cloud usage

All calls go through the OpenAI-compatible endpoint https://dashscope-intl.aliyuncs.com/compatible-mode/v1 (Singapore region), keyed by DASHSCOPE_API_KEY.

Sub-task	Model	Why
Long-horizon planning, coverage strategy	qwen-max (dev: qwen-plus)	1M context, expensive — called once per step, not per pixel
Per-step perception + pixel grounding	qwen-vl-plus (fallback qwen-vl-max)	GUI grounding + cheap input; agent loops are input-bound
Fast visual oracle (layout/error checks)	qwen-vl-plus	Low latency, runs every step within budget
Repro script + final test export	qwen-coder-plus	Generates runnable Playwright/pytest
Bug dedup + screen novelty	text-embedding-v3	Cheap vector ops, no LLM call

(Live model strings, routed exclusively through gateway.py — see MODEL_* env vars in .env.example / orchestrator/config.py. Verify current snapshot names in the Model Studio console before the final demo run per AGENT.md §0.5.)

The architectural justification — input price dominates agent cost, so perception runs on Plus/Flash while only strategic planning hits Max — is documented in AGENT.md and should be stated in the submission.

Alibaba Cloud services (deployment proof requirement)

Service	Role
Model Studio / DashScope	Qwen inference (primary Alibaba Cloud API usage)
OSS	Screenshots, videos, repro scripts, evidence bundles
SAE (Serverless App Engine) / ECS	Orchestrator + Playwright workers (need a real browser)
ApsaraDB RDS for PostgreSQL	State graph + bug DB
Tair (Redis-compatible)	Checkpoint cache + job queue
Function Compute	Lightweight webhook handlers (GitHub/Jira filing)

The "proof of Alibaba Cloud deployment" file is orchestrator/evidence.py (calls OSS) plus orchestrator/gateway.py (calls DashScope), running on SAE/ECS.

Tech stack

• Backend: Python 3.12, FastAPI (async), Pydantic v2
• Browser automation: Playwright (Chromium) in containers
• Data: PostgreSQL, Redis/Tair, OSS
• Frontend: Next.js 14 live dashboard (SSE)
• Infra: Terraform
• MCP: Python MCP SDK (server + client)

Repo structure

argus/
├── orchestrator/          # control plane (FastAPI)
│   ├── gateway.py         # model gateway: routing, retry, cache, budget
│   ├── perception.py      # screenshot + a11y tree + console + network
│   ├── explorer.py        # the agent loop
│   ├── planner.py         # qwen3.7-max strategy, goal pinning, summarization
│   ├── coverage.py        # state graph, frontier queue, novelty scoring
│   ├── oracle.py          # deterministic + visual + spec oracles
│   ├── reproduction.py    # replay-from-checkpoint flake gate
│   ├── reporter.py        # severity, dedup, repro-script gen, suite growth
│   ├── checkpoint.py      # durable execution (PG + Redis)
│   ├── evidence.py        # OSS writer  <- Alibaba proof
│   ├── trace.py           # reasoning-trace observability store
│   ├── gates.py           # 3-tier human-in-loop
│   ├── mcp_server.py      # exposes ARGUS as an MCP server
│   ├── mcp_client.py      # consumes GitHub as an MCP tool
│   ├── memory.py          # cross-run warm start + diff-aware prioritization
│   ├── wiring.py          # adapts oracle/reporter/reproduction/evidence into the loop's hooks
│   └── autofix.py         # flag-gated draft-PR stretch goal (AUTOFIX_ENABLED)
├── web/                   # Next.js 14 dashboard
├── infra/                 # Terraform: SAE, OSS, RDS, Tair
├── docs/                  # ARCHITECTURE.md, diagram.svg, phase briefs
├── .qwen/settings.json    # Qwen Code CLI config (MCP servers, no secrets)
└── tests/                 # pytest with mocked/replayed model calls

Setup

cp .env.example .env        # fill DASHSCOPE_API_KEY, ALI_AK_ID/SECRET, OSS_BUCKET, DB urls
uv sync                     # or pip install -r requirements.txt
playwright install chromium
docker compose up -d        # local postgres + redis

Required env vars: DASHSCOPE_API_KEY, ALI_AK_ID, ALI_AK_SECRET, OSS_BUCKET, OSS_ENDPOINT, DATABASE_URL, REDIS_URL. Never commit .env.

Running

uvicorn orchestrator.api:app --reload
curl -X POST localhost:8000/runs -H 'Content-Type: application/json' -d '{"url":"https://demo.app","charter":"test checkout"}'
# watch live at http://localhost:3000

Build phases

Build in order. Each brief is an agent-ready spec with acceptance criteria. Do not start a phase until the previous one's acceptance criteria pass.

0. docs/QWEN-CODE-SETUP.md — install & authenticate Qwen Code CLI, wire it to the repo's MCP tools (build with Qwen Cloud, not just for it)
1. docs/PHASE-1-perception-grounding-gateway.md — perception, model gateway, grounded action, act-and-verify
2. docs/PHASE-2-agent-loop-coverage-durability.md — the loop, coverage engine, durable execution
3. docs/PHASE-3-oracles-reproduction-reporting.md — oracle ensemble, reproduction gate, reporter, self-growing suite, eval harness
4. docs/PHASE-4-mcp-evidence-observability.md — MCP layer, OSS evidence, reasoning trace, human gates, CI/PR webhook trigger, cross-run memory
5. docs/PHASE-5-dashboard-deploy-demo.md — live dashboard, Alibaba Cloud deploy, demo hardening, auto-fix stretch goal

See AGENT.md for the engineering standards every phase must meet.

Submission checklist

• Public repo, OSS license file detectable in the About section
• Proof of Alibaba Cloud deployment: link to a code file using Alibaba Cloud services + a short recording of the backend running on Alibaba Cloud
• Architecture diagram (docs/diagram.svg)
• ~3-minute demo video on YouTube/Vimeo/Facebook, public
• Text description of features and functionality
• Track identified: Track 4 — Autopilot Agent
• (Optional) Blog/social post about building with Qwen Cloud -> Blog Post Prize

How this maps to judging

• Technical Depth & Engineering (30%): model gateway, MCP integration, pixel grounding, durable execution, coverage algorithm.
• Innovation & AI Creativity (30%): vision-grounded autonomous QA, reproduction gate, multi-model routing, novelty-driven exploration.
• Problem Value & Impact (25%): real, expensive pain (QA); self-growing regression suite = productization + scalability story.
• Presentation & Documentation (15%): live dashboard visualizes the reasoning; this docs package + architecture diagram.