Schema Flattening

[ngoiyaeric]

Recon schema flattening techniques achieved internally to resolve model compatibility issues and find other optimization opportunities for schema flattening in the codebase.

[ngoiyaeric]

@coderabbitai plan

[coderabbitai]

Coding Plan

Summary

• Document the existing flattening patterns (flat optional fields, avoid z.literal()/z.any(), runtime reconstruction) as reusable guidelines to prevent future UX regressions
• Flatten the nested coordinates object in geospatialQuerySchema to top-level latitude/longitude fields, reducing map interaction errors for users
• Simplify relatedSchema by removing the unnecessary wrapper object, improving streaming latency for related query suggestions
• Preserve inquirySchema as-is since its shallow nesting works with current models and its object shape provides streaming ergonomics that benefit incremental UI rendering

Design Choices

Design Choice 1: Should the `inquirySchema` options array be flattened to parallel arrays?

Options Considered:

1. Flatten to parallel arrays (optionValues: string[], optionLabels: string[]) for maximum compatibility
2. Keep current structure since it works and DeepPartial streaming benefits from the object shape

Chosen Option: 2

Rationale: Keep current structure—the schema is shallow enough that xAI accepts it, and the object shape provides better streaming ergonomics for the UI. With DeepPartial, each option streams as a complete {value, label} unit, allowing the followup-panel component to render options incrementally as they arrive. Flattening to parallel arrays would break this streaming granularity, potentially causing UI flicker as mismatched value/label pairs arrive asynchronously.

Design Choice 2: Should the `relatedSchema` items array be simplified?

Options Considered:

1. Simplify to items: z.array(z.string()) since the wrapper object adds no semantic value
2. Keep current structure to maintain consistency with other schemas and streaming behavior

Chosen Option: 1

Rationale: Simplify to z.array(z.string())—this is a straightforward optimization that reduces schema complexity without compatibility risk. The wrapper object provides no streaming benefit here (unlike inquirySchema where value/label pairing matters), and removing it reduces parsing overhead, allowing related query suggestions to appear faster in the search-related components.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Audit and Document Existing Flattening Patterns

Catalog all schema flattening techniques currently implemented in the codebase, extracting the compatibility rules and rationale into a centralized reference that can guide future schema design.

UX Benefit: Documented guidelines prevent future regressions where new schemas inadvertently use incompatible patterns (like z.literal() or deeply nested objects), which would cause model failures and degrade user-facing features. This protects the reliability improvements already achieved.

Task 1: Catalog Existing Flattened Schemas

Review and document the flattening patterns already implemented in the two primary flattened schemas.

• Review lib/schema/geospatial.tsx and extract the flat schema pattern: optional fields with queryType discriminant, avoidance of nested required objects, and reliance on field descriptions to convey per-queryType requirements
• Review lib/schema/resolution-search.ts and document the explicit avoidance of z.literal() (generates JSON Schema const) and z.any(), plus the flattened GeoJSON feature structure
• Document the post-processing reconstruction pattern in app/actions.tsx (lines 117-133) where flat schema output is transformed back to standard GeoJSON

Task 2: Create Schema Compatibility Guidelines

Establish documented rules for schema design that ensures xAI/OpenAI-compatible endpoint compatibility.

• Create a guidelines document in docs/ or as a comment block in lib/schema/README.md covering: (1) avoid z.literal(), (2) avoid z.any(), (3) prefer flat optional fields over discriminated unions, (4) use field descriptions to convey conditional requirements, (5) implement runtime reconstruction for nested output formats
• Reference the existing schema files as canonical examples of the pattern

🤖 Prompt for AI agents

This phase is about auditing the existing codebase and creating documentation
for schema compatibility guidelines — no application code changes are made.

- Review `lib/schema/geospatial.tsx` and extract the flat schema pattern:
  - Optional fields with `queryType` discriminant
  - Avoidance of nested required objects
  - Reliance on field descriptions to convey per-queryType requirements
- Review `lib/schema/resolution-search.ts` and document:
  - Explicit avoidance of `z.literal()` (which generates JSON Schema `const`)
  - Avoidance of `z.any()`
  - Flattened GeoJSON feature structure
- Review `app/actions.tsx` (lines 117-133) to document the post-processing
reconstruction pattern where flat schema output is transformed back to standard
GeoJSON
- Create a guidelines document in `docs/` or as a comment block in
`lib/schema/README.md` covering the following rules:
  1. Avoid `z.literal()`
  2. Avoid `z.any()`
  3. Prefer flat optional fields over discriminated unions
  4. Use field descriptions to convey conditional requirements
  5. Implement runtime reconstruction for nested output formats
- Reference `lib/schema/geospatial.tsx` and `lib/schema/resolution-search.ts` as
canonical examples in the guidelines document

Phase 2: Flatten Geospatial Coordinates Schema

Promote the nested coordinates.latitude and coordinates.longitude fields to top-level schema fields, following the established flattening pattern and updating all consumers.

UX Benefit: The nested coordinates object is a known source of model parsing errors, particularly with xAI. When the model fails to correctly structure the nested object, reverse geocoding and map centering operations fail silently or return incorrect locations. Flattening to top-level latitude/longitude fields reduces these errors, resulting in more accurate map pin placements and location lookups for users.

Task 1: Update geospatialQuerySchema Definition

Refactor the schema to eliminate the nested coordinates object.

• In lib/schema/geospatial.tsx, replace coordinates: z.object({ latitude: z.number(), longitude: z.number() }).optional() with two top-level fields: latitude: z.number().min(-90).max(90).optional() and longitude: z.number().min(-180).max(180).optional()
• Update the field descriptions to indicate these are used for reverse geocoding and map centering operations
• Add or update the compatibility comment at the top of the file to note this change follows the established flattening pattern

Task 2: Update Geospatial Tool Execute Function

Refactor the tool's execute function to consume the flattened coordinate fields.

• In lib/agents/tools/geospatial.tsx, update the parameter destructuring to extract latitude and longitude directly instead of coordinates.latitude and coordinates.longitude
• Update any conditional logic that checks for coordinate presence (e.g., if (coordinates) becomes if (latitude !== undefined && longitude !== undefined))
• Ensure all downstream usages within the execute function (MCP tool calls, Gemini grounding, etc.) receive the coordinates in whatever format they require

Task 3: Verify Tool Registration Unchanged

Confirm that the tool registration in the index file requires no changes.

• Verify lib/agents/tools/index.tsx imports and exports the geospatial tool without explicit schema type references that would need updating
• The Zod schema type inference should propagate automatically; no changes expected

🤖 Prompt for AI agents

This phase flattens the nested `coordinates` object in `geospatialQuerySchema`
to top-level fields and updates all consumers.

**Update `lib/schema/geospatial.tsx`:**
- Replace `coordinates: z.object({ latitude: z.number(), longitude: z.number()
}).optional()` with two separate top-level optional fields:
  - `latitude: z.number().min(-90).max(90).optional()`
  - `longitude: z.number().min(-180).max(180).optional()`
- Update field descriptions to indicate they are used for reverse geocoding and
map centering operations
- Add or update the compatibility comment at the top of the file noting this
follows the established flattening pattern

**Update `lib/agents/tools/geospatial.tsx`:**
- Update parameter destructuring to extract `latitude` and `longitude` directly
rather than from a nested `coordinates` object
- Replace any conditional logic like `if (coordinates)` with `if (latitude !==
undefined && longitude !== undefined)`
- Ensure all downstream usages (MCP tool calls, Gemini grounding, etc.) receive
coordinates in their required format

**Verify `lib/agents/tools/index.tsx`:**
- Confirm imports and exports of the geospatial tool do not contain explicit
schema type references that need updating
- No changes should be required here since Zod schema type inference propagates
automatically

Phase 3: Simplify Related Queries Schema

Streamline the relatedSchema by removing the unnecessary wrapper object in the items array, and update all consumers.

UX Benefit: The current { query: string } wrapper adds parsing overhead without providing streaming benefits. Simplifying to string[] reduces the token count the model must generate for each suggestion, resulting in faster time-to-render for the related queries panel. Users see follow-up suggestions appear more quickly after their initial search completes.

Task 1: Simplify relatedSchema Definition

Remove the wrapper object from the items array structure.

• In lib/schema/related.tsx, change items: z.array(z.object({ query: z.string() })).length(3) to items: z.array(z.string()).length(3)
• Update the PartialRelated and Related type exports to reflect the new shape
• Add a comment noting this simplification reduces schema complexity for better model compatibility

Task 2: Update Query Suggestor Agent

Update the agent to work with the simplified array structure.

• In lib/agents/query-suggestor.tsx, verify the streamObject call and any post-processing of the result handles string[] instead of { query: string }[]
• Update any result mapping that previously accessed .query on each item

Task 3: Update Search Related Components

Update the UI components that consume the related queries.

• In components/search-related.tsx, update the iteration over related.items to treat each item as a plain string rather than an object with a .query property
• In components/search-related-optimized.tsx, apply the same change
• Verify click handlers and display logic work with the simplified string items

🤖 Prompt for AI agents

This phase simplifies the `relatedSchema` by removing the unnecessary wrapper
object from the items array and updates all consumers to handle plain strings.

**Update `lib/schema/related.tsx`:**
- Change `items: z.array(z.object({ query: z.string() })).length(3)` to `items:
z.array(z.string()).length(3)`
- Update the `PartialRelated` and `Related` type exports to reflect the new
shape
- Add a comment noting this simplification reduces schema complexity for better
model compatibility

**Update `lib/agents/query-suggestor.tsx`:**
- Verify the `streamObject` call handles the updated schema
- Update any post-processing or result mapping that previously accessed `.query`
on each item — items are now plain strings (`string[]` instead of `{ query:
string }[]`)

**Update `components/search-related.tsx` and
`components/search-related-optimized.tsx`:**
- Update iteration over `related.items` to treat each item as a plain string
instead of an object with a `.query` property
- Verify click handlers and display logic correctly work with the simplified
string items

Research

The project uses Zod schemas throughout lib/schema/ for AI tool parameters and structured outputs, consumed by Vercel AI SDK's streamText, streamObject, and generateObject calls. Two schemas (geospatialQuerySchema and resolutionSearchSchema) already implement explicit flattening patterns with documented compatibility comments explaining avoidance of z.literal(), z.any(), and discriminated unions for xAI endpoint compatibility. Post-processing reconstruction in app/actions.tsx demonstrates the pattern of flattening schemas for AI consumption, then rebuilding nested structures at runtime. The model factory in lib/utils/index.ts supports xAI, Gemini, OpenAI, and Bedrock providers with a fallback chain.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

This phase is about auditing the existing codebase and creating documentation
for schema compatibility guidelines — no application code changes are made.

- Review `lib/schema/geospatial.tsx` and extract the flat schema pattern:
  - Optional fields with `queryType` discriminant
  - Avoidance of nested required objects
  - Reliance on field descriptions to convey per-queryType requirements
- Review `lib/schema/resolution-search.ts` and document:
  - Explicit avoidance of `z.literal()` (which generates JSON Schema `const`)
  - Avoidance of `z.any()`
  - Flattened GeoJSON feature structure
- Review `app/actions.tsx` (lines 117-133) to document the post-processing
reconstruction pattern where flat schema output is transformed back to standard
GeoJSON
- Create a guidelines document in `docs/` or as a comment block in
`lib/schema/README.md` covering the following rules:
  1. Avoid `z.literal()`
  2. Avoid `z.any()`
  3. Prefer flat optional fields over discriminated unions
  4. Use field descriptions to convey conditional requirements
  5. Implement runtime reconstruction for nested output formats
- Reference `lib/schema/geospatial.tsx` and `lib/schema/resolution-search.ts` as
canonical examples in the guidelines document
===============================================================================

Task: 2

This phase flattens the nested `coordinates` object in `geospatialQuerySchema`
to top-level fields and updates all consumers.

**Update `lib/schema/geospatial.tsx`:**
- Replace `coordinates: z.object({ latitude: z.number(), longitude: z.number()
}).optional()` with two separate top-level optional fields:
  - `latitude: z.number().min(-90).max(90).optional()`
  - `longitude: z.number().min(-180).max(180).optional()`
- Update field descriptions to indicate they are used for reverse geocoding and
map centering operations
- Add or update the compatibility comment at the top of the file noting this
follows the established flattening pattern

**Update `lib/agents/tools/geospatial.tsx`:**
- Update parameter destructuring to extract `latitude` and `longitude` directly
rather than from a nested `coordinates` object
- Replace any conditional logic like `if (coordinates)` with `if (latitude !==
undefined && longitude !== undefined)`
- Ensure all downstream usages (MCP tool calls, Gemini grounding, etc.) receive
coordinates in their required format

**Verify `lib/agents/tools/index.tsx`:**
- Confirm imports and exports of the geospatial tool do not contain explicit
schema type references that need updating
- No changes should be required here since Zod schema type inference propagates
automatically
===============================================================================

Task: 3

This phase simplifies the `relatedSchema` by removing the unnecessary wrapper
object from the items array and updates all consumers to handle plain strings.

**Update `lib/schema/related.tsx`:**
- Change `items: z.array(z.object({ query: z.string() })).length(3)` to `items:
z.array(z.string()).length(3)`
- Update the `PartialRelated` and `Related` type exports to reflect the new
shape
- Add a comment noting this simplification reduces schema complexity for better
model compatibility

**Update `lib/agents/query-suggestor.tsx`:**
- Verify the `streamObject` call handles the updated schema
- Update any post-processing or result mapping that previously accessed `.query`
on each item — items are now plain strings (`string[]` instead of `{ query:
string }[]`)

**Update `components/search-related.tsx` and
`components/search-related-optimized.tsx`:**
- Update iteration over `related.items` to treat each item as a plain string
instead of an object with a `.query` property
- Verify click handlers and display logic correctly work with the simplified
string items

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- @coderabbitai You can skip phase 3. Add a simple unit test case for phase 2.
- @coderabbitai For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[ngoiyaeric]

@coderabbitai plan what are the user experience improvements from these proposed changes?

[coderabbitai]

I've updated the plan above based on your feedback. Ready to proceed when you are!