feat: Add JSON Schema support for State outputs #11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit implements comprehensive JSON Schema support for Terraform/OpenTofu state outputs, enabling clients to declare and retrieve expected output types. - Clients can publish JSON Schema definitions before outputs exist in state - Schemas are preserved when Terraform state is uploaded - TypeScript and other clients can generate type-safe code from schemas - Full RBAC integration with new state-output:schema-write/read actions - Added SetOutputSchema and GetOutputSchema RPC methods - Extended OutputKey message with optional schema_json field - Both RPCs support state reference by logic_id or guid - Added schema_json TEXT column to state_outputs table - Migration 20251123000001_add_output_schemas.go - Schemas stored alongside output metadata (no separate table needed) - New actions: state-output:schema-write, state-output:schema-read - Added to product-engineer role seed data - Middleware interceptor checks for both new RPCs - Repository: SetOutputSchema/GetOutputSchema with upsert logic - Service: Business logic with state existence validation - Connect handlers: State reference resolution and RPC routing - SDK: Client methods supporting StateReference - CLI: gridctl state set-output-schema / get-output-schema commands ```bash gridctl state set-output-schema --output-key vpc_id --schema-file schema.json my-state gridctl state get-output-schema --output-key vpc_id my-state ``` - Schemas can be set before outputs exist (pre-declaration) - UpsertOutputs preserves existing schemas when state is uploaded - Empty schema returns are not errors (indicates no schema set) - Supports .grid context for state resolution in CLI **IMPORTANT**: Run `buf generate` to generate protobuf code: - Go stubs in api/state/v1/ - TypeScript stubs in js/sdk/gen/ Code will not compile until buf generate is run in development environment. See OUTPUT_SCHEMA_IMPLEMENTATION.md for detailed documentation.
Implements complete test coverage for JSON Schema output functionality across authentication modes and use cases. ## Test Coverage ### Basic Tests (No Auth) - output_schema_test.go - ✅ UC1: Basic schema operations (set/get/list with schemas) - ✅ UC2: Schema pre-declaration (before output exists in state) - ✅ UC3: Schema update (modify existing schemas) - ✅ UC4: Schema preservation during state uploads - ✅ UC5: Schema with dependencies (producer/consumer) - ✅ UC7: Complex schemas (nested objects, arrays) - ✅ UC8: Schema via gridctl CLI commands - ✅ State reference resolution (logic_id vs guid) ### Mode 1 Auth Tests - auth_mode1_state_output_test.go - ✅ Product engineer can set/get schemas on env=dev states - ✅ Product engineer denied access to env=prod state schemas - ✅ RBAC properly enforces state-output:schema-write/read actions ## Test Fixtures Created ### Terraform State Files (testdata/) - vpc_output_with_schema.json - VPC outputs (vpc_id, vpc_cidr, azs) - subnet_output_with_schema.json - Subnet outputs (subnet_ids, private_subnet_ids) - complex_output.json - Complex nested objects (network_config, tags) ### JSON Schema Files (testdata/) - schema_vpc_id.json - VPC ID pattern schema (^vpc-[a-z0-9]+$) - schema_subnet_ids.json - Array of subnet IDs schema - schema_config_object.json - Complex object schema with required fields - schema_tags.json - Tags map schema with pattern properties ## Test Strategy ### No Auth Tests - SDK-based tests using direct API calls - Verify schema CRUD operations - Test schema preservation during state updates - Validate CLI commands work correctly - Test both logic_id and guid state references ### Mode 1 Auth Tests - CLI-based tests using gridctl with OAuth tokens - Verify RBAC enforcement for schema operations - Test cross-scope denial (env=dev vs env=prod) - Validate proper authorization errors ## Validation Approach 1. **Pre-declaration**: Schemas can be set before outputs exist 2. **Preservation**: Schemas persist through state uploads (serial changes) 3. **Update**: Schemas can be modified/updated 4. **Dependencies**: Schemas work alongside dependency tracking 5. **Complex Types**: Nested objects and arrays supported 6. **CLI Integration**: gridctl commands functional 7. **Authorization**: RBAC properly enforced ## Running Tests ```bash # All integration tests (no auth) make test-integration # Mode 1 tests (with Keycloak) make test-integration-mode1 # Specific test cd tests/integration go test -v -run TestBasicSchemaOperations go test -v -run TestMode1_OutputSchemaAuthorization ``` ## Documentation See OUTPUT_SCHEMA_TEST_PLAN.md for complete test plan with all use cases, expected behaviors, and validation strategy.
This document details the implementation plan for validating Terraform state output values against their associated JSON Schemas during tfstate POST operations. Key components: - Recommended library: santhosh-tekuri/jsonschema/v6 (Draft 7 compliant) - Architecture: Validation in service layer with schema caching - Database schema: validation_status, validation_error, validated_at columns - Edge integration: New "schema-invalid" edge status - Error handling: Soft validation (non-blocking) with future strict mode - Performance: LRU cache with 5min TTL, ~0.1-1ms overhead per output The plan includes: - 8-day phased implementation (foundation → validation → integration → testing) - Database migration strategy for existing deployments - Integration with edge update jobs - Comprehensive test coverage - Future enhancements (strict mode, webhooks, versioning) Addresses user request for JSON Schema validation during state uploads.
Comprehensive summary of output schema integration tests including: - 8 test functions covering all use cases (UC1-UC8) - 9 fixture files (3 TF states + 6 JSON schemas) - Basic tests without authentication (output_schema_test.go) - Mode 1 OIDC auth tests with RBAC enforcement - Test coverage metrics and validation strategy - Instructions for running tests locally and in CI Test coverage: - 460 lines of test code in output_schema_test.go - ~140 lines added to auth_mode1_state_output_test.go - All critical paths tested: CRUD, auth, CLI, dependencies Ready for CI/CD integration and pull request review.
Comprehensive UI/UX design document for displaying output schemas and validation status in the Grid webapp. Key features: - New "Outputs" tab in DetailView modal with enhanced OutputCard components - Schema preview with human-readable format extraction - Validation status indicators (valid/invalid/error) with icons and colors - Expandable JSON Schema viewer with syntax highlighting - New edge status "schema-invalid" across all views (Graph, List, Detail) - Color-coded borders and badges for validation states Components: - OutputCard (NEW) - Enhanced output display with schema metadata - DetailView - Add Outputs tab, move outputs from Overview - ListView - Add schema-invalid badge styling - GraphView - Add schema-invalid edge color (red #ef4444) TypeScript model updates: - Extend OutputKey interface with schema_json, validation_status, validation_error, validated_at fields - Add "schema-invalid" to EdgeStatus type Design principles: - Read-only UI (no schema editing at this stage) - Accessibility: icons + colors + text for color-blind users - Performance: lazy parsing, useMemo for schema objects - Error handling: graceful degradation for invalid schemas Includes: - Visual mockups and component code - User flows for investigating validation failures - Testing strategy (unit, integration, visual regression) - Migration & rollout plan - Future enhancement ideas Estimated effort: 15 hours (P0-P2 tasks) Dependencies: Backend API changes, protobuf regeneration
so0k
force-pushed
the
claude/add-output-schema-support-01BKuzdyJiCw1HazmCpKNRdA
branch
from
44e2118 to
310ddbd
Compare
- Replace default e2e to run without Keycloak - Add basic first no-auth flow
- chore: Adopt Claude Code Web branch into spec kit - chore: Collect output-schema user stories, edge cases and functional requirements - fix: failing output schema integration tests
Generate implementation plans with Opus 4.5
Add inferrer background job to automatically generate json-schema from /tstate POST data. Implemented with Sonnet 4.5 and reviewed with Opus 4.5 for llm trials
- Fixed outputs schema purge logic post schema inference feature (grid-1e1f) - Added State serial check to async inference (grid-f430) - Implemented validation service with caching (grid-1908, grid-fd88, grid-bef1, grid-0ad0) - Implemented synchronous validation job (grid-1c39) - Completed integration tests (grid-c833) Implementation Summary: - Validation Service: Uses jsonschema/v6 with 1000-entry LRU cache, distinguishes data errors (invalid) from system errors (error) - Validation Job: Runs SYNCHRONOUSLY (blocks response ~10-50ms) to prevent race conditions with EdgeUpdateJob - Repository Methods: Added GetSchemasForState() and UpdateValidationStatus() - Integration: Fully wired through serve.go → router.go → tfstate_handlers.go Updated integration tests 13 Test Functions: 1. ✅ TestValidationPassPattern - Valid pattern validation (FR-029, FR-031) 2. ✅ TestValidationFailPattern - Invalid pattern validation (FR-029, FR-030) 3. ✅ TestValidationSkipWhenNoSchema - Skip when no schema (FR-033) 4. ✅ TestValidationComplexSchema - Complex nested objects (FR-029) 5. ✅ TestValidationStatusInResponse - Status in ListStateOutputs (FR-034) 6. ✅ TestValidationErrorMessage - Structured error format (FR-035) 7. ✅ TestValidationNonBlocking - Non-blocking state upload (FR-032) 8. ✅ TestValidationMetadataInResponses - Validation fields in RPC responses (FR-034) 9. ✅ TestValidationTransitionFromInvalidToValid - Status transitions 10. ✅ TestValidationWithManualSchemaSource - Manual schema validation 11. ✅ TestValidationWithInferredSchema - Inferred schema validation 12. ✅ TestValidationErrorWithArrayItemViolation - Array validation errors Test Fixtures Created: - testdata/schema_pattern_strict.json - VPC ID pattern schema (string) - testdata/schema_subnet_array_pattern.json - Subnet array pattern schema - testdata/tfstate_valid_pattern.json - Valid state matching patterns - testdata/tfstate_invalid_pattern.json - Invalid state violating patterns
Closed Epics - ✅ grid-093b - Phase 2B: Schema Validation - ✅ grid-e70b - Phase 2C: Edge Status Updates Original EdgeStatus design had a simple `schema-invalid` status that would override "drift" (clean/dirty) status. We implemented a better composite model where validation is a separate dimension: - Old: missing-output > schema-invalid > dirty > clean - New: missing-output > (dirty × valid|invalid) > clean × valid|invalid Added full integration tests and updated js/sdk as well as gridctl display.
…rid-a966, grid-522d) ## Summary Fixed three critical issues with JSON schema handling and validation error reporting: 1. Schema validation on upload (grid-e903) 2. Async validation triggering after SetOutputSchema (grid-a966) 3. Error message formatting and file path leakage (grid-522d) ## Changes ### 1. Schema Validation Before Storage (grid-e903) - Added validateJSONSchema() call in SetOutputSchema RPC handler - Validates schema structure against JSON Schema Draft 7 metaschema before storing - Returns clear error message if schema is invalid - Prevents storing malformed schemas that would fail later during validation ### 2. Async Validation Job Triggering (grid-a966) - SetOutputSchema handler now triggers validation for existing outputs - Fetches current state content and enqueues validation job asynchronously - Validation runs without blocking response, visible to users immediately - Eliminated need to run "terraform refresh" to see validation results ### 3. Validation Error Message Cleanup (grid-522d) - Fixed formatValidationError() to use leaf.Error() for full formatted messages * Previously: Only extracted raw constraint info (e.g., "pattern: vpc-.*") * Now: Gets complete English sentences (e.g., "does not match pattern ^vpc-.*$") - Added cleanSchemaErrorMessage() to strip file:// paths from schema validation errors - Prevents leaking local filesystem paths in error messages - All integration tests passing with clean error formatting ## Files Modified - cmd/gridapi/internal/services/state/service.go * Added validateJSONSchema() with regex cleanup * SetOutputSchema now validates before storing - cmd/gridapi/internal/services/validation/validator.go * formatValidationError() uses leaf.Error() for proper formatting - cmd/gridapi/internal/server/connect_handlers.go * SetOutputSchema triggers validation after storing schema - cmd/gridapi/internal/server/connect_handlers_deps.go * Added validationJob dependency to handlers - cmd/gridapi/internal/server/router.go * Registered validationJob in handler setup - tests/integration/output_*_test.go * All validation tests passing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…f, grid-25e5, grid-7f81, grid-fb4e)
Implement complete webapp UI for output schema validation display:
- Create OutputCard component: Enhanced output display with schema preview, validation status, and expandable JSON viewer
- Add Outputs tab to DetailView: Dedicated tab for outputs with count badge, moved from Overview tab
- Update edge status colors: Support all 8 edge statuses including schema-invalid, clean-invalid, dirty-invalid
- Full test coverage: 30 unit tests for OutputCard + 39 total integration tests passing
Features:
- Schema preview with type, pattern, enum extraction
- Validation status indicators (valid/invalid/error) with color-coded cards
- Error messages displayed inline with validation details
- Expandable JSON schema viewer with syntax-like formatting
- Sensitive flag support (maintains existing UX pattern)
- Relative timestamp formatting ("2 minutes ago")
- Full accessibility support (ARIA labels, keyboard navigation)
Edge Status Colors:
- schema-invalid: #ef4444 (red-500) - Validation failure on synchronized output
- clean-invalid: #ef4444 - Drift=clean, Validation=invalid
- dirty-invalid: #ef4444 - Drift=dirty, Validation=invalid
- missing-output: #dc2626 (red-600) - Output removed from state
- mock: #6b7280 (gray-500) - Using mock value
Test Results: 39/39 passing (30 new OutputCard tests + 9 updated integration tests)
Note: Discovered backend issue grid-8918 - EdgeUpdateJob not triggered when validation status changes. Edge statuses computed correctly in backend but not updated for already-synchronized outputs with new validation failures. Documented separately for backend engineering.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
Author
|
Small bug remaining in the UpdateEdgeStatus job related to output schema validation see |
so0k
deleted the
claude/add-output-schema-support-01BKuzdyJiCw1HazmCpKNRdA
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit implements comprehensive JSON Schema support for Terraform/OpenTofu state outputs, enabling clients to declare and retrieve expected output types.
Key Features
Changes
Protocol & API
Database
Authorization
Implementation Layers
CLI Commands
Implementation Notes
Next Steps
IMPORTANT: Run
buf generateto generate protobuf code:Code will not compile until buf generate is run in development environment.
See OUTPUT_SCHEMA_IMPLEMENTATION.md for detailed documentation.