Skip to content

fix: recursive array/object JSON Schema in tool params across all runtimes#332

Merged
sethjuarez merged 5 commits into
mainfrom
sethjuarez/fix-array-item-schema-support
Apr 10, 2026
Merged

fix: recursive array/object JSON Schema in tool params across all runtimes#332
sethjuarez merged 5 commits into
mainfrom
sethjuarez/fix-array-item-schema-support

Conversation

@sethjuarez

Copy link
Copy Markdown
Member

Problem

Tool parameter schemas with nested arrays and objects produced flat JSON Schema like {"type": "array"} without items, and {"type": "object"} without nested properties. This caused LLMs to invent their own field names inside array items, leading to undefined values in the caller's code.

Example — this .prompty tool definition:

parameters:
  - name: encounters
    kind: array
    items:
      kind: object
      properties:
        - name: title
          kind: string
        - name: difficulty
          kind: integer

Before (sent to LLM):

{ "encounters": { "type": "array" } }

After (sent to LLM):

{
  "encounters": {
    "type": "array",
    "items": {
      "type": "object",
      "properties": {
        "title": { "type": "string" },
        "difficulty": { "type": "integer" }
      },
      "required": ["title", "difficulty"],
      "additionalProperties": false
    }
  }
}

Root Cause

The schema emitter types (ArrayProperty.items, ObjectProperty.properties) were already correct — ArrayProperty has items: Property and ObjectProperty has properties: list[Property]. The bug was in the hand-written wire-format conversion functions that mapped these to flat JSON Schema without recursing.

Fix

Replace flat schema conversion with recursive property→JSON Schema logic in all runtimes:

Runtime File What Changed
Python OpenAI/Azure/Foundry providers/openai/executor.py _schema_to_wire delegates to _property_to_json_schema
Python Anthropic providers/anthropic/executor.py Same pattern
TypeScript OpenAI packages/openai/src/wire.ts schemaToWire delegates to propertyToJsonSchema
TypeScript Anthropic packages/anthropic/src/wire.ts Same pattern
C# Prompty.Core/SchemaHelpers.cs New recursive PropertyToJsonSchema() method
Rust OpenAI prompty-openai/src/wire.rs New property_to_json_schema() for tools + outputs
Rust Anthropic prompty-anthropic/src/wire.rs Same for tools

Note: Structured output was already correct in Python and TypeScript (they had a separate recursive path). C# and Rust structured output was also flat and is now fixed.

Tests

  • Python: 43 OpenAI executor tests + 36 Anthropic tests pass (3 + 2 new nested schema tests)
  • C#: 21 schema helper tests pass (3 new: ArrayOfObjects, NestedObject, DeeplyNested)
  • TypeScript: 3 new tests added (same pre-existing @prompty/core resolution issue blocks test runner)
  • Rust: Compiles clean (cargo check) for both prompty-openai and prompty-anthropic

…times

The wire-format conversion functions (_schema_to_wire, schemaToWire,
PropertiesToJsonSchema, parameters_to_json_schema) were producing flat
schemas like {"type": "array"} with no "items" for array properties and
no nested "properties" for object properties. This caused LLMs to
invent their own field names inside array items and objects.

The schema emitter types (ArrayProperty.items, ObjectProperty.properties)
were already correct — the bug was purely in the hand-written wire
conversion code that didn't recurse into nested types.

Fix: Replace flat schema conversion with recursive property-to-JSON-Schema
logic in all 5 runtimes:

- Python OpenAI/Azure/Foundry: _schema_to_wire delegates to _property_to_json_schema
- Python Anthropic: same pattern
- TypeScript OpenAI: schemaToWire delegates to propertyToJsonSchema
- TypeScript Anthropic: same pattern
- C#: new recursive PropertyToJsonSchema method in SchemaHelpers
- Rust OpenAI: new property_to_json_schema fn for tools + structured output
- Rust Anthropic: same for tools

Structured output (_property_to_json_schema / propertyToJsonSchema) was
already correct in Python and TypeScript. C# and Rust were also fixed
for structured output paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sethjuarez
sethjuarez force-pushed the sethjuarez/fix-array-item-schema-support branch from 3035812 to 9d9ce99 Compare April 10, 2026 21:15
sethjuarez and others added 4 commits April 10, 2026 14:32
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sethjuarez
sethjuarez merged commit 40aad82 into main Apr 10, 2026
19 checks passed
@sethjuarez
sethjuarez deleted the sethjuarez/fix-array-item-schema-support branch April 10, 2026 21:58
sethjuarez added a commit that referenced this pull request Apr 10, 2026
Bump Python (2.0.0a8), TypeScript (2.0.0-alpha.8), C# (2.0.0-alpha.8),
and VS Code extension packages for release including:
- fix: recursive array/object JSON Schema in tool params (#332)
- feat: Rust runtime parity + optional authenticationMode (#333)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sethjuarez added a commit that referenced this pull request Apr 11, 2026
* chore: bump all runtimes to alpha.8

Bump Python (2.0.0a8), TypeScript (2.0.0-alpha.8), C# (2.0.0-alpha.8),
and VS Code extension packages for release including:
- fix: recursive array/object JSON Schema in tool params (#332)
- feat: Rust runtime parity + optional authenticationMode (#333)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: nonce type coercion bug in parser strict mode across all runtimes

When randomBytes/token_hex generates an all-digit hex nonce (e.g.
'1234567890123456'), parseAttrs type coercion converts it to a number.
The subsequent strict comparison (number !== string) always fails,
causing a spurious 'Nonce mismatch' error.

Fix: compare nonces as strings in buildMessage/build_message across
Python, TypeScript, and Rust parsers.

Probability of all-digit 16-char hex: (10/16)^16 ≈ 0.01% — matches
the observed flaky CI failure rate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: add conceptual documentation for thread inputs (conversation history)

Adds a new core-concepts page explaining how thread inputs work in Prompty:
- What threads are and why they use nonce-based expansion
- How to declare kind: thread inputs and place them in templates
- Data format with multi-language examples (Python, TypeScript, C#, Rust)
- Internal pipeline mechanics (render → parse → expand)
- Best practices: token budgets, sliding windows, stateless design
- Integration with agent mode

Also adds cross-references from the file-format doc, core-concepts index,
and the chat-assistant tutorial.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant