Track 1: Dynamic Agentic Workflow

Overview

This starter kit provides a scaffold for building an agentic LLM engine. We have already provided all necessary parsing logics and API connection, and your goal is to optimize the engine for the agentic workflow scenario.

Project Structure

.
├── app/
│   ├── main.py          # FastAPI entry point.
│   ├── constants.py          # Necessary constants
│   ├── schemas.py       # Pydantic models for request/response.
│   └── agent_engine.py  # MAIN LOGIC GOES HERE.
├── Dockerfile           # Environment definition.
├── pyproject.toml       # Python dependencies.
└── README.md

Your Task

Modify app/agent_engine.py to optimize the LLM inference for the agentic workflow scenario. The benchmark evaluates throughput, latency, perplexity (output quality), and trace length (efficiency).

Local Development & Testing

1. Setup

The docker-compose.yaml in the repository specifies the AMD64 platform to align with the test environment. If you are developing locally on a non-AMD64 machine, please comment out the platform: linux/amd64 line in docker-compose.yaml to avoid architecture errors.

services:
  agent-engine:
    # ...
    # platform: linux/amd64  <-- Comment out for local ARM64 testing

2. Build and Run

We recommend using Docker Compose for local testing. The service will listen on http://localhost:8000.

# Build the image, and start a container in detach mode
docker compose up --build -d

To shutdown the engine,

docker compose stop

3. Check Status

• Liveness: GET http://localhost:8000/health
• Readiness: GET http://localhost:8000/ready

Deployment on Vast.ai

To test your engine on Vast.ai, you need to push your Docker image to a container registry (GHCR) and create a Template.

1. Push Docker Image to GHCR (Private)

1. Login to GHCR Create a GitHub Personal Access Token under https://github.com/settings/tokens with write:packages and read:packages permissions.

export CR_PAT=YOUR_TOKEN
export GITHUB_USERNAME=YOUR_GITHUB_USERNAME
echo $CR_PAT | docker login ghcr.io -u $GITHUB_USERNAME --password-stdin

2. Build & Push

docker compose build
docker tag agent-engine:latest ghcr.io/$GITHUB_USERNAME/agent-engine:latest
docker push ghcr.io/$GITHUB_USERNAME/agent-engine:latest

We suggest remembering the SHA256 digest of the image to ensure reproducibility. You can find it at the end of output of docker push, or use the following command:

docker inspect --format='{{index .RepoDigests 0}}' ghcr.io/$GITHUB_USERNAME/agent-engine:latest

2. Create a Private Template on Vast.ai

1. Log in to Vast.ai and go to the Templates page (https://cloud.vast.ai/templates/).
2. Click Create New Template, set the relevant fields as following:
   • Image Path: ghcr.io/YOUR_GITHUB_USERNAME/agent-engine@sha256:YOUR_DIGEST
   • Docker Options: Add -p 8000:8000 to expose the port.
   • Launch Mode: Docker ENTRYPOINT.
   • Docker Repository Authentication: Set Server as ghcr.io and input your github username and PAT accordingly.
   • Disk Space: We suggest 128GB as the image and weights take around 40GB.

3. Launch & Test

Rent a GPU instance using your template. We suggest using NVIDIA RTX 5080 as this aligns with the test environment. Note that if you want to use Flash attention, please use Vast AI instances with Max CUDA: 13.0.

Once running, you can open the IP & Port Info tab of the instance by clicking on the IP address, which provide the external port $VAST_PORT mapping to the internal 8000 port (which the engine is listening on). Then you should be able to access the engine at http://$VAST_IP:$VAST_PORT.

Submission Instructions

Submit the following items to Canvas:

1. Full Image Name with Digest: (e.g., ghcr.io/username/track1-agent@sha256:abcdef...)
2. Read-Only PAT and the related Github username: A GitHub Personal Access Token with read:packages permission, plus the associated Github username.
3. Archived Code: A .zip file of your project code.

1. Get Image Digest & Check Architecture

Ensure your image is built for linux/amd64 and get its digest.

# Pull the image (if not local)
docker pull ghcr.io/${GITHUB_USERNAME}/agent-engine:latest

# Inspect to get Digest and Architecture
docker inspect ghcr.io/${GITHUB_USERNAME}/agent-engine:latest --format 'Digest: {{.RepoDigests}} | Arch: {{.Architecture}}'

• Verify: Architecture must be amd64.
• Copy: The full string ghcr.io/YOUR_GITHUB_USERNAME/agent-engine@sha256:YOUR_DIGEST.

2. Generate Read-Only PAT

To allow TAs to grade your submission without full access to your account:

1. Go to GitHub Settings -> Developer Settings -> Personal access tokens (Classic).
2. Generate new token (classic).
3. Scopes: Select ONLY read:packages.
4. Expiration: Set to at least 2 weeks after the submission deadline.
5. Generate and copy the token.

3. Test Your Submission Credentials

Crucial: Verify that the PAT works for pulling the image.

# 1. Logout of your current session
docker logout ghcr.io

# 2. Login with the READ-ONLY PAT
echo "YOUR_READ_ONLY_PAT" | docker login ghcr.io -u YOUR_USERNAME --password-stdin

# 3. Try to pull your image
docker pull ghcr.io/YOUR_USERNAME/track1-agent@sha256:<YOUR_DIGEST>

If the pull succeeds, your credentials are correct.

4. Archive Code

Zip your track1_agent folder (excluding .venv, __pycache__, and large model files).

zip -r track1_agent_submission.zip . -x "*.venv*" -x "*__pycache__*"

Submit the Image URI (with digest), the Read-Only PAT, and the Zip file to Canvas.