Skip to content

Improve YAML parsing robustness and handle null Avro types#202

Merged
jogrogan merged 7 commits intomainfrom
fix/yaml-log-level-and-null-safety
Apr 1, 2026
Merged

Improve YAML parsing robustness and handle null Avro types#202
jogrogan merged 7 commits intomainfrom
fix/yaml-log-level-and-null-safety

Conversation

@jogrogan
Copy link
Copy Markdown
Collaborator

Problem

Improve YAML parsing robustness and handle null Avro types

The operator processes Kubernetes resource YAML that can be arbitrarily malformed — truncated, invalid mappings, duplicate document markers, or Java type tags. Several code paths were either throwing these exceptions to callers when they should have been handling them gracefully, or logging caught exceptions at ERROR level when WARN is more appropriate (since the code recovers successfully).

Additionally, AvroConverter did not handle Avro schemas with null types, causing NullPointerException during SQL planning for schemas with optional fields.

Changes

K8sPipelineElementStatusEstimator

  • Downgraded YAML parse exception log level from ERROR to WARN. The exceptions are caught and handled correctly by returning "not ready" status — ERROR was a false signal.

Operator

  • Added null-safety guards in isReady() and isFailed() for resources with no namespace or no metadata. Both methods now return false rather than throwing IllegalArgumentException or NullPointerException.

SubscriptionReconciler

  • Added null metadata guard in fetchAttributes() to return an empty map rather than throwing.
  • Added isSqlValidationError() helper to downgrade SQL validation errors (caused by bad user-supplied SQL) from ERROR to WARN.

AvroConverter

  • Added null guard at the entry of rel() for null schema parameters.
  • Added explicit case NULL: handler returning SqlTypeName.NULL for null-typed Avro fields.
  • Nullable union fields (["null", "T"]) were already handled; the null schema case was the gap.

Tests
New tests covering each scenario:

  • K8sPipelineElementStatusEstimatorTest: malformed YAML (IndexOutOfBounds), ScannerException, ParserException, ConstructorException, null metadata
  • OperatorTest: ScannerException and ParserException in isReady()/isFailed(), null namespace, null metadata
  • SubscriptionReconcilerFetchAttributesTest: ClassCastException, ScannerException, null metadata in fetchAttributes()
  • AvroConverterTest: nullable union fields, null-typed fields, null schema parameter

jogrogan and others added 7 commits March 31, 2026 15:10
snakeyaml throws IndexOutOfBoundsException when parsing malformed YAML
via Dynamics.newFromYaml(). The call in estimateElementStatus() was
outside the existing try/catch, causing the exception to propagate up
through the JDBC layer and crash getPipelineStatus requests.

- Wrap Dynamics.newFromYaml() in estimateElementStatus() with try/catch;
  return a default unready status on parse failure instead of propagating
- Same fix in getElementConfiguration() which had the identical pattern
- Add tests for both: malformed YAML returns unready status / empty map

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SnakeYAML throws ScannerException, ParserException, and ConstructorException
when parsing malformed YAML in Kubernetes resource specs or annotations. Two
call sites were unprotected:

1. Operator.isReady(String yaml) and Operator.isFailed(String yaml): the
   Dynamics.newFromYaml() call was outside the existing try/catch, so any
   parse error would propagate up and crash the SubscriptionReconciler
   (stack trace through SubscriptionReconciler.reconcile line 171 ->
   Operator.isReady line 180). 

2. SubscriptionReconciler.fetchAttributes(String yaml): same unprotected
   Dynamics.newFromYaml() call, returns empty map on parse failure instead.

The K8sPipelineElementStatusEstimator call site was already fixed in a prior commit; this commit adds additional tests covering ScannerException,
ParserException, and ConstructorException for completeness.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…timator

When Dynamics.newFromYaml() parses YAML without a metadata field, the
resulting DynamicKubernetesObject has null metadata. Calling
obj.getMetadata().getName() on it caused NPE in estimateElementStatus().

Guard against null metadata by returning an unready status early,
consistent with how we handle other parse/retrieval failures.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Add null guard at entry to AvroConverter.rel() and explicit NULL type
handling. When a null schema or NULL-typed schema is encountered,
returns a proper SQL NULL type instead of throwing NPE.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- K8sPipelineElementStatusEstimator: downgrade YAML parse errors from
  ERROR to WARN — exceptions are caught and handled correctly, ERROR
  level was causing spurious automated tickets
- Operator.isReady/isFailed: guard against null metadata and null
  namespace in K8s resource
- SubscriptionReconciler.fetchAttributes: guard against null metadata;
  downgrade SQL validation errors to WARN level

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jogrogan jogrogan merged commit f6d1dff into main Apr 1, 2026
1 check passed
@jogrogan jogrogan deleted the fix/yaml-log-level-and-null-safety branch April 1, 2026 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants