Skip to content

feat: add Bedrock Mantle API format support for harness #1412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
notgitika wants to merge 3 commits into
base: main
Choose a base branch
from
Open

feat: add Bedrock Mantle API format support for harness #1412

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2f921a6
feat: add Bedrock Mantle API format support for harness model config
notgitika May 28, 2026
e9896c7
test: add unit tests for Bedrock Mantle apiFormat support
notgitika May 28, 2026
ffa1953
fix: use BedrockApiFormatSchema constants and add apiFormat validatio…
notgitika Jun 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ async function main() {
codeLocation?: string;
tools?: { type: string; name: string }[];
apiKeyArn?: string;
apiFormat?: 'converse_stream' | 'responses' | 'chat_completions';
}[] = [];
for (const entry of specAny.harnesses ?? []) {
const harnessDir = path.resolve(projectRoot, entry.path);
Expand All @@ -127,6 +128,7 @@ async function main() {
codeLocation: harnessSpec.dockerfile ? harnessDir : undefined,
tools: harnessSpec.tools,
apiKeyArn: harnessSpec.model?.apiKeyArn,
apiFormat: harnessSpec.model?.apiFormat,
});
} catch (err) {
throw new Error(
Expand Down
2 changes: 2 additions & 0 deletions src/assets/cdk/bin/cdk.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ async function main() {
codeLocation?: string;
tools?: { type: string; name: string }[];
apiKeyArn?: string;
apiFormat?: 'converse_stream' | 'responses' | 'chat_completions';
}[] = [];
for (const entry of specAny.harnesses ?? []) {
const harnessDir = path.resolve(projectRoot, entry.path);
Expand All @@ -82,6 +83,7 @@ async function main() {
codeLocation: harnessSpec.dockerfile ? harnessDir : undefined,
tools: harnessSpec.tools,
apiKeyArn: harnessSpec.model?.apiKeyArn,
apiFormat: harnessSpec.model?.apiFormat,
});
} catch (err) {
throw new Error(
Expand Down
1 change: 1 addition & 0 deletions src/cli/aws/agentcore-harness.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ export type HarnessStatus = 'CREATING' | 'READY' | 'UPDATING' | 'DELETING' | 'DE

export interface BedrockModelConfig {
modelId: string;
apiFormat?: 'converse_stream' | 'responses' | 'chat_completions';
temperature?: number;
topP?: number;
maxTokens?: number;
Expand Down
1 change: 1 addition & 0 deletions src/cli/commands/add/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ export interface AddHarnessCliOptions {
name?: string;
modelProvider?: string;
modelId?: string;
apiFormat?: string;
apiKeyArn?: string;
container?: string;
memory?: boolean;
Expand Down
15 changes: 15 additions & 0 deletions src/cli/commands/add/validate.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { ConfigIO, findConfigRoot } from '../../../lib';
import {
AgentNameSchema,
BedrockApiFormatSchema,
BuildTypeSchema,
DatasetNameSchema,
DatasetSchemaTypeSchema,
Expand Down Expand Up @@ -892,6 +893,20 @@ const VALID_HARNESS_TOOLS = [
const VALID_GATEWAY_OUTBOUND_AUTH = ['awsIam', 'none', 'oauth'] as const;

export function validateAddHarnessOptions(options: AddHarnessCliOptions): ValidationResult {
if (options.apiFormat) {
const validFormats = BedrockApiFormatSchema.options;
if (!validFormats.includes(options.apiFormat as (typeof validFormats)[number])) {
return {
valid: false,
error: `Invalid API format: ${options.apiFormat}. Use ${validFormats.join(', ')}`,
};
}
const provider = options.modelProvider ?? 'bedrock';
if (provider !== 'bedrock') {
return { valid: false, error: '--api-format is only supported for the bedrock provider' };
Copy link
Copy Markdown

@davisarthur davisarthur Jun 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe out of scope for this PR, but OpenAI also supports responses/chat completions API formats

}
}

Comment on lines +896 to +909
Copy link
Copy Markdown
Contributor

@padmak30 padmak30 Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should extract the common validation logic across create/add into a single util. I see that create/add harness validations have diverged, something to fix later.

if (options.tools) {
const toolNames = options.tools.split(',').map(s => s.trim());
for (const tool of toolNames) {
Expand Down
49 changes: 49 additions & 0 deletions src/cli/commands/create/__tests__/harness-validate.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import { validateCreateHarnessOptions } from '../harness-validate';
import { describe, expect, it } from 'vitest';

describe('validateCreateHarnessOptions', () => {
const validOptions = {
name: 'MyHarness',
modelProvider: 'bedrock',
modelId: 'anthropic.claude-v2',
};

describe('apiFormat validation', () => {
it('accepts valid apiFormat for bedrock provider', () => {
const result = validateCreateHarnessOptions({ ...validOptions, apiFormat: 'responses' });
expect(result.valid).toBe(true);
});

it('accepts chat_completions format', () => {
const result = validateCreateHarnessOptions({ ...validOptions, apiFormat: 'chat_completions' });
expect(result.valid).toBe(true);
});

it('accepts converse_stream format', () => {
const result = validateCreateHarnessOptions({ ...validOptions, apiFormat: 'converse_stream' });
expect(result.valid).toBe(true);
});

it('rejects invalid apiFormat value', () => {
const result = validateCreateHarnessOptions({ ...validOptions, apiFormat: 'invalid_format' });
expect(result.valid).toBe(false);
expect(result.error).toContain('Invalid API format');
});

it('rejects apiFormat for non-bedrock provider', () => {
const result = validateCreateHarnessOptions({
...validOptions,
modelProvider: 'open_ai',
apiKeyArn: 'arn:aws:secretsmanager:us-east-1:123:secret:key',
apiFormat: 'responses',
});
expect(result.valid).toBe(false);
expect(result.error).toContain('only supported for the bedrock provider');
});

it('passes when apiFormat is not specified', () => {
const result = validateCreateHarnessOptions(validOptions);
expect(result.valid).toBe(true);
});
});
});
4 changes: 3 additions & 1 deletion src/cli/commands/create/harness-action.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { CONFIG_DIR } from '../../../lib';
import type { HarnessModelProvider, NetworkMode } from '../../../schema';
import type { BedrockApiFormat, HarnessModelProvider, NetworkMode } from '../../../schema';
import { harnessPrimitive } from '../../primitives/registry';
import { type ProgressCallback, createProject } from './action';
import type { CreateResult } from './types';
Expand All @@ -12,6 +12,7 @@ export interface CreateHarnessProjectOptions {
cwd: string;
modelProvider: HarnessModelProvider;
modelId: string;
apiFormat?: BedrockApiFormat;
Copy link
Copy Markdown

@davisarthur davisarthur Jun 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, why do we nest API format under model configuration in some places (eg), but not in others (above).

From Harness API perspective, API format will always be nested under the model provider configuration since different model providers support a different set of API formats.

apiKeyArn?: string;
skipMemory?: boolean;
containerUri?: string;
Expand Down Expand Up @@ -57,6 +58,7 @@ export async function createProjectWithHarness(options: CreateHarnessProjectOpti
name: options.name,
modelProvider: options.modelProvider,
modelId: options.modelId,
apiFormat: options.apiFormat,
apiKeyArn: options.apiKeyArn,
containerUri: options.containerUri,
dockerfilePath: options.dockerfilePath,
Expand Down
16 changes: 15 additions & 1 deletion src/cli/commands/create/harness-validate.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
import { HarnessNameSchema, ProjectNameSchema } from '../../../schema';
import { BedrockApiFormatSchema, HarnessNameSchema, ProjectNameSchema } from '../../../schema';
import { validateFolderNotExists } from './validate';

export interface CreateHarnessCliOptions {
name?: string;
projectName?: string;
modelProvider?: string;
modelId?: string;
apiFormat?: string;
apiKeyArn?: string;
container?: string;
noMemory?: boolean;
Expand Down Expand Up @@ -90,5 +91,18 @@ export function validateCreateHarnessOptions(options: CreateHarnessCliOptions, c
return { valid: false, error: `--api-key-arn is required for ${options.modelProvider} provider` };
}

if (options.apiFormat) {
const validFormats = BedrockApiFormatSchema.options;
if (!validFormats.includes(options.apiFormat as (typeof validFormats)[number])) {
return {
valid: false,
error: `Invalid API format: ${options.apiFormat}. Use ${validFormats.join(', ')}`,
};
}
if (options.modelProvider !== 'bedrock') {
return { valid: false, error: '--api-format is only supported for the bedrock provider' };
}
}
Comment thread
notgitika marked this conversation as resolved.

return { valid: true };
}
45 changes: 45 additions & 0 deletions src/cli/operations/deploy/imperative/deployers/__tests__/harness-mapper.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,51 @@ describe('mapHarnessSpecToCreateOptions', () => {
});
});

it('maps bedrock with apiFormat responses', async () => {
const opts = baseOptions({
harnessSpec: {
name: 'h',
model: { provider: 'bedrock', modelId: 'openai.gpt-oss-120b', apiFormat: 'responses' },
tools: [],
skills: [],
} as any,
});
const result = await mapHarnessSpecToCreateOptions(opts);
expect(result.model).toEqual({
bedrockModelConfig: { modelId: 'openai.gpt-oss-120b', apiFormat: 'responses' },
});
});

it('maps bedrock with apiFormat chat_completions', async () => {
const opts = baseOptions({
harnessSpec: {
name: 'h',
model: { provider: 'bedrock', modelId: 'openai.gpt-oss-120b', apiFormat: 'chat_completions' },
tools: [],
skills: [],
} as any,
});
const result = await mapHarnessSpecToCreateOptions(opts);
expect(result.model).toEqual({
bedrockModelConfig: { modelId: 'openai.gpt-oss-120b', apiFormat: 'chat_completions' },
});
});

it('omits apiFormat when converse_stream (default)', async () => {
const opts = baseOptions({
harnessSpec: {
name: 'h',
model: { provider: 'bedrock', modelId: 'claude', apiFormat: 'converse_stream' },
tools: [],
skills: [],
} as any,
});
const result = await mapHarnessSpecToCreateOptions(opts);
expect(result.model).toEqual({
bedrockModelConfig: { modelId: 'claude' },
});
});

it('includes optional model params when set', async () => {
const opts = baseOptions({
harnessSpec: {
Expand Down
3 changes: 2 additions & 1 deletion src/cli/operations/deploy/imperative/deployers/harness-mapper.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,13 +152,14 @@ export async function mapHarnessSpecToCreateOptions(options: MapHarnessOptions):
// ============================================================================

function mapModel(model: HarnessSpec['model']): HarnessModelConfiguration {
const { provider, modelId, apiKeyArn, temperature, topP, topK, maxTokens } = model;
const { provider, modelId, apiKeyArn, apiFormat, temperature, topP, topK, maxTokens } = model;

switch (provider) {
case 'bedrock':
return {
bedrockModelConfig: {
modelId,
...(apiFormat && apiFormat !== 'converse_stream' && { apiFormat }),
...(temperature !== undefined && { temperature }),
...(topP !== undefined && { topP }),
...(maxTokens !== undefined && { maxTokens }),
Expand Down
14 changes: 12 additions & 2 deletions src/cli/primitives/HarnessPrimitive.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { APP_DIR, ConfigIO, type Result, findConfigRoot } from '../../lib';
import type {
BedrockApiFormat,
HarnessGatewayOutboundAuth,
HarnessModelProvider,
HarnessSpec,
Expand Down Expand Up @@ -27,6 +28,7 @@ export interface AddHarnessOptions {
name: string;
modelProvider: HarnessModelProvider;
modelId: string;
apiFormat?: BedrockApiFormat;
Copy link
Copy Markdown

@davisarthur davisarthur Jun 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this be extensible for non-Bedrock model providers? For example, Harness now supports OpenAI with responses and chat completions.

apiKeyArn?: string;
systemPrompt?: string;
skipMemory?: boolean;
Expand Down Expand Up @@ -149,6 +151,7 @@ export class HarnessPrimitive extends BasePrimitive<AddHarnessOptions, Removable
model: {
provider: options.modelProvider,
modelId: options.modelId,
...(options.apiFormat && { apiFormat: options.apiFormat }),
...(options.apiKeyArn && { apiKeyArn: options.apiKeyArn }),
},
tools,
Expand Down Expand Up @@ -345,6 +348,7 @@ export class HarnessPrimitive extends BasePrimitive<AddHarnessOptions, Removable
.option('--name <name>', 'Harness name (start with letter, alphanumeric + underscores, max 48 chars)')
.option('--model-provider <provider>', 'Model provider: bedrock, open_ai, gemini')
.option('--model-id <id>', 'Model ID (e.g., anthropic.claude-3-5-sonnet-20240620-v1:0)')
.option('--api-format <format>', 'API format for Bedrock: converse_stream, responses, chat_completions')
.option('--api-key-arn <arn>', 'API key ARN for non-Bedrock providers')
.option('--container <uri-or-path>', 'Container image URI or path to a Dockerfile')
.option('--no-memory', 'Skip auto-creating memory')
Expand Down Expand Up @@ -390,6 +394,7 @@ export class HarnessPrimitive extends BasePrimitive<AddHarnessOptions, Removable
name?: string;
modelProvider?: string;
modelId?: string;
apiFormat?: string;
apiKeyArn?: string;
container?: string;
memory?: boolean;
Expand Down Expand Up @@ -454,16 +459,21 @@ export class HarnessPrimitive extends BasePrimitive<AddHarnessOptions, Removable
process.exit(1);
}

const { DEFAULT_MODEL_IDS } = await import('../tui/screens/harness/types');
const { DEFAULT_BEDROCK_MANTLE_MODEL_ID, DEFAULT_MODEL_IDS } =
await import('../tui/screens/harness/types');
const provider = (cliOptions.modelProvider ?? 'bedrock') as HarnessModelProvider;
const modelId = cliOptions.modelId ?? DEFAULT_MODEL_IDS[provider];
const isMantleFormat =
cliOptions.apiFormat === 'responses' || cliOptions.apiFormat === 'chat_completions';
const modelId =
cliOptions.modelId ?? (isMantleFormat ? DEFAULT_BEDROCK_MANTLE_MODEL_ID : DEFAULT_MODEL_IDS[provider]);

const containerOption = this.parseContainerFlag(cliOptions.container);

const result = await this.add({
name: cliOptions.name,
modelProvider: provider,
modelId,
apiFormat: cliOptions.apiFormat as BedrockApiFormat | undefined,
apiKeyArn: cliOptions.apiKeyArn,
containerUri: containerOption.containerUri,
dockerfilePath: containerOption.dockerfilePath,
Expand Down
1 change: 1 addition & 0 deletions src/cli/tui/screens/create/useCreateFlow.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -591,6 +591,7 @@ export function useCreateFlow(cwd: string): CreateFlowState {
name: addHarnessConfig.name,
modelProvider: addHarnessConfig.modelProvider,
modelId: addHarnessConfig.modelId,
apiFormat: addHarnessConfig.apiFormat,
apiKeyArn: addHarnessConfig.apiKeyArn,
skipMemory: addHarnessConfig.skipMemory,
containerUri: addHarnessConfig.containerUri,
Expand Down
1 change: 1 addition & 0 deletions src/cli/tui/screens/harness/AddHarnessFlow.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ export function AddHarnessFlow({ isInteractive = true, onExit, onBack, onDev, on
name: config.name,
modelProvider: config.modelProvider,
modelId: config.modelId,
apiFormat: config.apiFormat,
apiKeyArn: config.apiKeyArn,
skipMemory: config.skipMemory,
containerUri: config.containerUri,
Expand Down
Loading
Loading