Skip to content

Cross-SDK DagSerialization compatibility CI for the Go SDK #66942

@jason810496

Description

@jason810496

Background

ADR 0003 explicitly promises:

The Go SDK gains a serde package that performs this encoding from
bundlev1.Bundle / bundlev1.Task, validated against
validation/serialization/test_dags.yaml (the same fixture set the
Java SDK uses), so the Go and Java outputs are byte-identical for
shared inputs.

The Go serde package exists
(go-sdk/pkg/execution/serde.go, serde_test.go) but the cross-SDK
parity hook does not - there is no prek/CI step that runs the
Go serializer against the shared test_dags.yaml and diffs the output
against Python's SerializedDAG.serialize_dag like the Java SDK does
via compare.py.

What needs to happen

  1. Add a Go-side serializer driver that consumes
    validation/serialization/test_dags.yaml and emits a JSON file
    per Dag fixture (parallel to the Java
    SerializationCompatibilityTest).
  2. Wire it into scripts/ci/prek/ so the existing compare.py runs
    against serialized_go.json alongside serialized_java.json.
  3. Fail the check on any field-level divergence.

Acceptance criteria

  • A regression in SerializedDAG.serialize_dag shape that Go doesn't
    follow fails CI before merge.
  • The Go and Java SDKs share the fixture file (no Go-only copy).

Context


Drafted-by: Claude Code (Opus 4.7) (no human review before posting)

Metadata

Metadata

Assignees

Labels

area:CIAirflow's tests and continious integrationarea:dev-toolsgo-sdkLabel to track work items for golang task sdkkind:featureFeature Requestskind:taskA task that needs to be completed as part of a larger issuepriority:mediumBug that should be fixed before next release but would not block a release

Type

No type
No fields configured for issues without a type.

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions