AMD Hackathon ACT II — Validator-Gated AI Agent

An accuracy-first agent for the AMD Developer Hackathon. It combines zero-token deterministic solving, category-aware routing, Fireworks model escalation, and strict output validation within the competition's container contract.

Highlights

Capability	Implementation
Accuracy	86.36% (38/44) on a frozen unseen evaluation
Container contract	Reads /input/tasks.json; writes /output/results.json
Reliability	Validator-gated answers with retry and safe fallback paths
Efficiency	Local deterministic solutions use zero inference tokens
Routing	Category-aware model selection without a paid classification call
Runtime	linux/amd64, 600-second global deadline

Evaluation results are from an internal frozen dataset. Hidden leaderboard performance may vary.

Architecture

Input tasks
    |
    v
Deterministic classifier and context trimmer
    |
    +--> High-confidence local solver --> Validator --+
    |                                                 |
    +--> Fireworks model routing ------> Validator --+--> Normalized results
                                      |               |
                                      +--> Retry -----+

The runtime performs five main steps:

1. Classify each task locally using deterministic signals.
2. Solve high-confidence arithmetic and exact tasks without an API call.
3. Batch and route remaining tasks to an allowed Fireworks model.
4. Validate syntax, structure, labels, and task-specific constraints.
5. Normalize the accepted answer and always emit the required JSON shape.

Submission artifacts

Competition image

ghcr.io/jaswanth422/amd-hackathon-agent@sha256:5618ae7333244350b835eace6be76e82c5ac11b2bb130975b4b3046a378bd9fd

The verified image is public, targets linux/amd64, is approximately 2.9 GB, and produced 9/9 valid, non-blank results in the container smoke test.

Mentor demo image

ghcr.io/jaswanth422/amd-hackathon-demo:latest

The demo runs the Streamlit interface in demo_app.py and compares the fine-tuned router with a prompt-based routing baseline.

Quick start

Local Python

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

Add your own Fireworks credentials and permitted model identifiers to .env, then run:

set -a
source .env
set +a

export TASKS_PATH=./input/tasks.json
export RESULTS_PATH=./output/results.json
export LOCAL_TIER=0

mkdir -p output
python main.py

Competition container

docker pull --platform linux/amd64 \
  ghcr.io/jaswanth422/amd-hackathon-agent@sha256:5618ae7333244350b835eace6be76e82c5ac11b2bb130975b4b3046a378bd9fd

./scripts/run_local.sh

To reuse an existing local build:

SKIP_BUILD=1 ./scripts/run_local.sh

Streamlit demo

pip install -r requirements-demo.txt
streamlit run demo_app.py

The demo displays the selected route, chosen model tier, generated answer, per-query tokens, and running session savings.

Configuration

Runtime configuration is supplied only through environment variables.

Variable	Purpose
FIREWORKS_API_KEY	Fireworks authentication secret
FIREWORKS_BASE_URL	Approved Fireworks-compatible endpoint
ALLOWED_MODELS	Comma-separated model allowlist
MODEL_CHEAP	Lower-cost routing tier
MODEL_EXPENSIVE	Higher-capability routing tier
LOCAL_TIER	Enables or disables optional local inference
USE_TOON	Selects TOON or plain-JSON batch transport

Never commit .env. The file is ignored by Git and is not copied into the competition image.

Validation and testing

Run the unit suite:

python -m unittest discover -s tests -v

Run the benchmark harness:

python benchmark.py
python benchmark.py --modes router,cheap_only,expensive_only

The validators check, where applicable:

• required JSON structure and non-blank answers;
• requested function names and Python syntax;
• named-entity labels and multi-part output constraints;
• bullet counts and length limits;
• malformed, truncated, uncertain, or meta-analytic responses.

Training and evaluation

The project follows the AMD query-router workflow:

Stage	Project implementation
Model tiers	Environment-controlled cheap and expensive Fireworks models
Difficulty labels	Empirical cheap-vs-expensive evaluation with an independent judge
Router training	DistilBERT easy/hard classifier with class weighting
Baselines	Always-cheap, always-expensive, and prompt-based routing
Final policy	Accuracy-first local solving plus validator-gated escalation

Train or evaluate the router with:

pip install -r router/requirements.txt
python3 router/train_router.py
python3 router/evaluate_heldout.py \
  --router-report router/checkpoints/router-distilbert-best/report.json

Large binary checkpoints and generated benchmark artifacts are intentionally excluded from the lightweight source repository. The published Docker images are the reproducible artifacts for the full submission and hosted demo.

Compliance

• All remote inference calls go through FIREWORKS_BASE_URL.
• Only identifiers in ALLOWED_MODELS may be called.
• Secrets are injected at runtime and are never embedded in the image.
• The process exits non-zero on fatal input, output, or configuration errors.
• Token usage is recorded in /output/token_usage.json.
• Remote failures retain a valid output contract through controlled fallbacks.

Repository map

Path	Purpose
main.py	Competition entry point
classifier.py	Deterministic category classification
fireworks_client.py	Batched Fireworks inference and retry handling
validator.py	Task-specific acceptance gate
normalizer.py	Output canonicalization
router/	Router training and evaluation
demo_app.py	Mentor-facing Streamlit application
tests/	Unit and contract tests
scripts/	Local execution and packaging helpers