Skip to content

serverless-dna/code-harness

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

3 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Code Harness

A simple coding agent CLI built with AWS Strands SDK β€” designed as a teaching example of how to build a coding harness from scratch.

There is a complete writeup on my Serverless DNA blog here

What is this?

This is a minimal but functional AI coding assistant that runs in your terminal. You type a request (e.g. "create a Python fibonacci function"), and the agent reads files, writes code, and runs commands to fulfill it.

The agent streams its output in real-time and shows its thinking process as it works, giving you visibility into how it makes decisions.

The entire implementation is ~375 lines of TypeScript across two files.

Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  User   β”‚ ─────▢ β”‚   Agent   β”‚ ─────▢ β”‚    Tools    β”‚
β”‚ (stdin) β”‚ ◀───── β”‚  (Strands)β”‚ ◀◀──── β”‚  (fs/shell) β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜        β””β”€β”€β”€β”€β”€β–²β”€β”€β”€β”€β”€β”˜        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                         β”‚  β”‚
                    send  β”‚  β”‚ respond
                    msg   β”‚  β”‚ + tool calls
                         β”‚  β–Ό
                   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                   β”‚  Bedrock  β”‚
                   β”‚  (Claude) β”‚
                   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

A single user message can trigger multiple LLM ↔ tool round-trips before you see a final response. That multi-turn cycle β€” running automatically inside agent.stream() β€” is what makes this an agent rather than a plain chatbot.

How the Agent Loop Works

This is the most important thing to understand. When you call agent.stream(userInput), Strands runs this cycle internally β€” potentially many times β€” before yielding your final answer:

  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚                    agent.stream(userInput)                   β”‚
  β”‚                                                              β”‚
  β”‚  1. Your message is sent to the LLM (Claude on Bedrock)      β”‚
  β”‚                                                              β”‚
  β”‚  2. The LLM streams back a response containing:              β”‚
  β”‚       β€’ reasoning text  β†’  you see: reasoningContentDelta    β”‚
  β”‚       β€’ response text   β†’  you see: textDelta events         β”‚
  β”‚       β€’ tool call(s)    β†’  you see: toolUseStart events      β”‚
  β”‚                                                              β”‚
  β”‚  3. If the LLM requested tool calls, Strands executes them:  β”‚
  β”‚       β€’ runs your callback functions (read file, run cmd…)   β”‚
  β”‚       β€’ you see: toolResultBlock events                      β”‚
  β”‚                                                              β”‚
  β”‚  4. The tool results are sent back to the LLM as a new       β”‚
  β”‚     message, and the LLM responds again  (back to step 2)    β”‚
  β”‚                                                              β”‚
  β”‚  5. The loop ends when the LLM returns a response with       β”‚
  β”‚     no tool calls β€” meaning it has enough information        β”‚
  β”‚     to give you a final answer.                              β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Example: You ask "add tests to my utils file". The agent might:

  1. Call list_directory to explore the project β†’ LLM sees the result
  2. Call read_file on utils.ts β†’ LLM sees the file contents
  3. Call read_file on the existing test file β†’ LLM sees the test structure
  4. Call write_file to write the new tests β†’ LLM sees the confirmation
  5. Call run_command to run the test suite β†’ LLM sees the output
  6. Return a final text response summarising what it did

All of that happens inside a single agent.stream() call. You didn't orchestrate any of it β€” the LLM decided which tools to call and in what order.

Features

  • Real-time streaming β€” See the agent's response as it's generated
  • Extended thinking β€” The model reasons internally before responding (shown in gray text)
  • Tool visibility β€” See which tools are being used in real-time
  • Conversation memory β€” The agent remembers the full conversation history
  • Simple configuration β€” One JSON file with AWS credentials and model settings

Project Structure

src/
β”œβ”€β”€ index.ts          # CLI entry point β€” agent setup + interactive loop
└── tools.ts          # Four coding tools the agent can use
config.json.example   # Template config β€” copy to config.json and fill in your settings
build-sea.sh          # Shell script to build the SEA binary
sea-config.json       # Node.js SEA configuration

The Tools

Tool What it does
read_file Read file contents from disk
write_file Create or overwrite files
list_directory List files/folders in a directory
run_command Execute a shell command (with timeout)

That's it. These four tools are enough for an agent to read code, write code, and verify its work.

Setup

Prerequisites

  • Node.js 20+
  • An AWS account with Bedrock model access enabled
  • AWS CLI installed

1. Configure AWS Authentication

The agent uses AWS credentials to call Bedrock. How you get those credentials depends on your setup β€” pick the method that matches your situation.

Install AWS CLI

# macOS
brew install awscli

# Linux
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

# Windows β€” download and run:
# https://awscli.amazonaws.com/AWSCLIV2.msi

Option A β€” AWS SSO (recommended for organisations)

If your company uses AWS SSO / IAM Identity Center:

# First-time setup: configure the SSO profile
aws configure sso

# Then log in whenever your session expires
aws sso login --profile your-profile

This stores temporary credentials under the profile name you chose. Use that same name in config.json (see step 3).

Option B β€” Static credentials (personal AWS account)

If you have a personal AWS account with an IAM user:

aws configure

This prompts for your access key ID, secret access key, and default region, and stores them in ~/.aws/credentials under [default].

What your credentials file looks like

After authenticating, ~/.aws/credentials will contain something like:

[default]
aws_access_key_id = AKIA...          # permanent key (Option B)
aws_secret_access_key = ...

[my-sso-profile]
aws_access_key_id = ASIA...          # temporary key β€” notice the ASIA prefix
aws_secret_access_key = ...
aws_session_token = IQoJb3JpZ2lu...  # session token required for temporary creds

Temporary credentials (SSO / STS) expire after 1–12 hours depending on your organisation's settings. When they expire, run aws sso login again.

2. Enable Model Access in Bedrock

Go to the AWS Bedrock console β†’ Model access β†’ Request access to the models you want to use.

3. Configure the Agent

Copy the example config and fill in your settings:

cp config.json.example config.json

Then edit config.json:

{
  "aws_profile": "default",
  "aws_region": "us-east-1",
  "model_id": "global.anthropic.claude-sonnet-4-6",
  "thinking_budget_tokens": 5000
}
Field Description
aws_profile The AWS CLI profile name to use (default, or a named profile like my-sso-profile)
aws_region AWS region where Bedrock is available
model_id The Bedrock model identifier
max_tokens (optional) Maximum tokens in the model's response. Defaults to 16,000 when thinking is enabled.
thinking_budget_tokens (optional) How many tokens the model can spend on internal reasoning. Set to 0 to disable thinking. Defaults to 5000.

Note: temperature is not a config option here β€” when thinking is enabled the Anthropic API requires temperature: 1 exactly, so it is set automatically. When thinking is disabled, 0.3 is used. See the walkthrough below for details.


Available Models on Bedrock

Extended thinking requires Claude Sonnet 4 or newer. Older models (Sonnet 3.5, Haiku) will ignore the thinking config or return an error.

🟣 Anthropic β€” Claude

Model ID Thinking
Claude Sonnet 4.6 (default) global.anthropic.claude-sonnet-4-6 βœ…
Claude Sonnet 4 anthropic.claude-sonnet-4-20250514-v1:0 βœ…
Claude Sonnet 3.5 v2 anthropic.claude-3-5-sonnet-20241022-v2:0 ❌
Claude Haiku 3.5 anthropic.claude-3-5-haiku-20241022-v1:0 ❌

If you switch to a model that doesn't support thinking, set thinking_budget_tokens to 0 in config.json.

πŸ”΅ Zhipu AI β€” GLM

Model ID Notes
GLM-4 zhipu.glm-4 Flagship GLM model β€” strong at reasoning & code
GLM-4V zhipu.glm-4v Multimodal variant with vision support

Note: GLM models are available via Bedrock in supported regions. Check the Bedrock model catalog for availability in your region.

🟒 Alibaba β€” Qwen

Model ID Notes
Qwen 2.5 72B Instruct qwen.qwen2-5-72b-instruct Latest flagship Qwen β€” top-tier reasoning & coding
Qwen 2.5 Coder 32B qwen.qwen2-5-coder-32b-instruct Specialized for code generation & completion
Qwen 2.5 7B Instruct qwen.qwen2-5-7b-instruct Lightweight, fast, cost-effective

Note: Qwen models are available via Bedrock Marketplace. Check the Bedrock model catalog for availability and subscription requirements.


4. Environment Variable Overrides

You can override any config value with environment variables (useful for CI or switching between profiles quickly):

# Override the profile
AWS_PROFILE=my-other-profile npm run dev

# Override the region
AWS_REGION=eu-west-1 npm run dev

# Override the model (disable thinking if switching to a non-Claude-4 model)
MODEL_ID=anthropic.claude-3-5-haiku-20241022-v1:0 npm run dev

# Use Qwen instead of Claude
MODEL_ID=qwen.qwen2-5-72b-instruct npm run dev

# Combine overrides
AWS_PROFILE=prod AWS_REGION=us-west-2 npm run dev

Priority order: Environment variable > config.json value

5. Install & Run

npm install
npm run dev

You should see:

πŸ”§ Code Harness β€” a simple coding agent
   Model:    global.anthropic.claude-sonnet-4-6
   Region:   us-east-1
   Profile:  default
   Thinking: enabled (budget: 5000 tokens)
   Type your request and press Enter. Type 'exit' to quit.

>

As the agent works, you'll see:

  • Gray text β€” The model's internal reasoning (thinking), streamed before the response
  • Cyan text β€” Tool usage notifications (e.g., [Using tool: read_file])
  • Normal text β€” The agent's final response to you

Building a Standalone Binary (SEA)

You can compile the entire agent into a single executable β€” no Node.js installation required to run it:

./build-sea.sh

This produces dist/code-harness (~115MB, includes the Node.js runtime). Copy it anywhere and run:

./dist/code-harness

Note: The binary still reads config.json from the current working directory at runtime. Make sure the config file is present and that you have valid AWS credentials.

How the SEA build works:

  1. Bundle β€” esbuild compiles all TypeScript + dependencies into one dist/bundle.js file
  2. Generate blob β€” node --experimental-sea-config creates a binary blob from the bundle
  3. Inject β€” The blob is injected into a copy of the node binary using postject
  4. Sign β€” On macOS, the binary is re-signed with an ad-hoc signature

See build-sea.sh for the full process (~50 lines of bash).

How It Works β€” A Walkthrough

1. Loading Config & Creating the Model (src/index.ts)

import { Agent, BedrockModel } from "@strands-agents/sdk";

const config = loadConfig();
process.env.AWS_PROFILE = config.aws_profile;

const thinkingBudget = config.thinking_budget_tokens ?? 5000;
const thinkingEnabled = thinkingBudget > 0;

// When thinking is on, Anthropic requires max_tokens > budget_tokens.
// We default to 16,000 which gives the model plenty of room to write
// large files or long responses after spending its thinking budget.
const DEFAULT_MAX_TOKENS_WITH_THINKING = 16000;

const model = new BedrockModel({
  region: config.aws_region,
  modelId: config.model_id,
  // Anthropic requires temperature: 1 when thinking is enabled.
  temperature: thinkingEnabled ? 1 : 0.3,
  // When thinking is off: leave maxTokens unset so Bedrock uses the model's
  // own maximum. When thinking is on: we must provide a value greater than
  // budget_tokens β€” Anthropic rejects the request otherwise.
  maxTokens: config.max_tokens ?? (thinkingEnabled ? DEFAULT_MAX_TOKENS_WITH_THINKING : undefined),
  // Thinking is passed via additionalRequestFields β€” there is no dedicated
  // thinking option on BedrockModel. This maps to Bedrock's
  // additionalModelRequestFields and is forwarded to the API as-is.
  ...(thinkingEnabled && {
    additionalRequestFields: {
      thinking: {
        type: "enabled",
        budget_tokens: thinkingBudget,
      },
    },
  }),
});

The BedrockModel uses the standard AWS SDK credential chain. By setting AWS_PROFILE, the SDK automatically loads credentials from ~/.aws/credentials for that profile β€” whether they're permanent keys or temporary STS tokens.

How thinking works in Strands

Strands has no dedicated thinking: true flag. Extended thinking is enabled by passing the thinking config through additionalRequestFields, which BedrockModel forwards directly to Bedrock's additionalModelRequestFields on every request.

Two hard requirements from Anthropic when thinking is on:

  1. temperature must be exactly 1 β€” any other value is rejected by the API
  2. max_tokens must exceed budget_tokens β€” the budget is consumed first, then the response is generated within the remaining token allowance

When thinking is enabled, the model reasons internally before writing its response. That reasoning is streamed back as reasoningContentDelta events, which we display in gray text. The thinking content is also included in the conversation history so the model can refer back to its earlier reasoning on follow-up turns.

2. Defining Tools (src/tools.ts)

Each tool uses the Strands tool() helper with a Zod schema:

import { tool } from "@strands-agents/sdk";
import { z } from "zod";

const readFile = tool({
  name: "read_file",
  description: "Read the contents of a file at the given path. Returns the file text.",
  inputSchema: z.object({
    filePath: z.string().describe("Path to the file to read (relative or absolute)"),
  }),
  callback: (input) => {
    return fs.readFileSync(input.filePath, "utf-8");
  },
});

Three things make a tool:

  • description β€” the LLM reads this to decide when to call the tool. Write it clearly.
  • inputSchema β€” a Zod schema that defines what arguments the LLM must provide. Strands validates the LLM's output against this before calling your function.
  • callback β€” the function that actually runs when the tool is called. Whatever it returns is sent back to the LLM as the tool result.

The LLM never calls your code directly. It emits a structured tool_use block in its response (e.g. { name: "read_file", input: { filePath: "src/index.ts" } }), and Strands is the one that validates it, calls your callback, and feeds the result back.

3. Creating the Agent

const agent = new Agent({
  model,                         // Bedrock provider
  systemPrompt: SYSTEM_PROMPT,   // Personality & instructions
  tools: codingTools,            // What it can do
});

The agent maintains conversation history automatically across stream() calls. We handle streaming output ourselves in the loop below, so we can display thinking in gray text and tool calls in cyan.

4. The Streaming Loop

This is where Strands does its work. A single call to agent.stream() runs the full multi-turn agent loop and surfaces each step as a typed event:

for await (const event of agent.stream(userInput)) {

  // Token-by-token streaming deltas β€” text output and thinking output.
  if (event.type === "modelContentBlockDeltaEvent") {
    const { delta } = event;

    if (delta.type === "textDelta") {
      // A chunk of the LLM's text response β€” print as it arrives.
      process.stdout.write(delta.text);

    } else if (delta.type === "reasoningContentDelta" && delta.text) {
      // The LLM's internal reasoning β€” shown in dim/italic gray.
      // Only fires when thinking is enabled in the model config.
      process.stdout.write(`\x1b[2m\x1b[3m${delta.text}\x1b[0m`);
    }

  // The LLM has decided to call a tool. Fires BEFORE the tool runs.
  // event.start.name = tool name.
  } else if (event.type === "modelContentBlockStartEvent") {
    if (event.start?.type === "toolUseStart") {
      process.stdout.write(`\n\x1b[36m[Using tool: ${event.start.name}]\x1b[0m\n`); // cyan
    }

  // The tool has finished running. Fires AFTER the callback returns.
  // Strands automatically sends this result back to the LLM.
  } else if (event.type === "toolResultBlock") {
    if (event.status === "error") {
      console.log(`\x1b[31m[Tool error]\x1b[0m`);
    }
  }
}

Event order within a single agent.stream() call (the tool section may repeat many times):

reasoningContentDelta  β†’  toolUseStart  β†’  toolResultBlock  β†’  (repeat)  β†’  final textDelta

You don't call the LLM again yourself. You don't collect tool results and pass them back. Strands handles all of that β€” the for await loop just keeps yielding events until the agent is done.

Key Concepts for Learners

  1. Tools are just functions β€” There's no magic. A tool is a function with a name, description, and typed inputs. The LLM reads the descriptions and decides which ones to call.

  2. The LLM doesn't run your code β€” Strands does β€” The LLM emits a structured tool call request. Strands validates it against your Zod schema, runs your callback, and sends the result back to the LLM. Your function is never called directly by the model.

  3. agent.stream() is the whole agent loop β€” One call can trigger many LLM ↔ tool round-trips. The loop ends automatically when the LLM stops requesting tools and returns a plain text response.

  4. Thinking has no dedicated flag β€” use additionalRequestFields β€” Strands passes additionalRequestFields directly to Bedrock's additionalModelRequestFields. This is the correct way to enable any model feature that isn't a first-class SDK option, including thinking.

  5. Streaming shows the process β€” Unlike invoke() which only returns the final result, stream() lets you observe each step: reasoning, tool calls, tool results, and the final answer β€” all as they happen.

  6. The system prompt matters β€” It shapes how the agent behaves. Our prompt tells it to read before writing and to test its changes. The LLM follows these instructions when deciding which tools to call and in what order.

  7. Conversation memory is built-in β€” The agent maintains the full message history across stream() calls. Each turn appends to that history, so the LLM always has context from earlier in the session.

Troubleshooting

Error Fix
config.json not found Copy config.json.example to config.json and fill in your settings
AccessDeniedException Your profile doesn't have Bedrock permissions, or credentials have expired β€” re-authenticate with aws sso login
ResourceNotFoundException The model ID is wrong or not available in your region
ValidationException (model access) Enable model access in the Bedrock console
ValidationException (thinking + temperature) Thinking requires temperature: 1 β€” this is set automatically, but check you haven't overridden it
ValidationException (thinking + max_tokens) max_tokens must be greater than thinking_budget_tokens β€” increase max_tokens or reduce the budget
Thinking not appearing Check the model supports thinking (Claude Sonnet 4+) and thinking_budget_tokens is greater than 0
ExpiredTokenException Your temporary credentials expired β€” run aws sso login --profile your-profile
The security token included in the request is invalid Credentials are invalid or expired β€” re-authenticate
Credentials not found Run aws configure (static keys) or aws sso login (SSO) to set up credentials

Extending This

Ideas for making this more capable:

  • Add a search_files tool β€” grep/regex search across the codebase
  • Add a patch_file tool β€” edit specific sections instead of rewriting entire files
  • Add conversation persistence β€” save/load chat history to disk
  • Add guardrails β€” restrict which directories can be written to
  • Customize streaming display β€” add progress bars, spinners, or other UI elements
  • Add error recovery β€” handle tool failures more gracefully

License

MIT

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors