Fix: handle proto type instead of schema on Lineage API - #57
Merged
Conversation
mabdh
approved these changes
May 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The lineage API previously mapped
{schema_name}in the URL directly to both a schema container (stored artifact in the DB) and the proto message name to trace. This worked only in the early one-schema-one-message model but breaks completely for real usage where one schema artifact (e.g.esb-log-entities) bundles hundreds of proto messages.This MR redesigns the lineage API endpoint so that:
schema_id) and the proto message root (type_name) explicitly.type_nameis a required path parameter — missing it returns400.Problem
Before
schema_nameserved dual purpose: DB lookup key and proto message root.GET .../schemas/esb-log-entities/lineagewould try to find a proto message namedesb-log-entitiesin the descriptor — and always return empty results for real schema containers.Item,Order— ambiguous without package context.After
schema_id= DB artifact key (esb-log-entities)type_name= fully-qualified proto message (gojek.esb.types.Location)type_name→400 Bad Requestgotocompany.events.Order.ItemnotItemChanges
API path
/schemas/{schema_name}/lineage/schemas/{schema_id}/types/{type_name}/lineageschema_namedual-useschema_idtype_name?type_name=Response contract
root_schema.schema_name"User"root_schema.schema_id"esb-log-entities"root_schema.type_name"gojek.esb.types.Location"schema_name"Item"schema_id"esb-log-entities"type_name"Item""gotocompany.events.Order.Item"path[i]["User","Item"]["gotocompany.events.User","gotocompany.events.Order.Item"]Files changed
internal/api/api.go/schemas/{schema_id}/types/{type_name}/lineageinternal/api/schema.goschema_id+type_namefrom path params; 400 whentype_nameemptyinternal/api/mocks/schema_service.goGetLineage(ctx, namespaceID, schemaID, rootType, level, direction)internal/api/lineage_test.gotype_name400 testcore/schema/lineage.goconvertLineageNodesnow emits full FQN inTypeNameandPath;RootSchemaRefandLineageSchemauseschema_id+type_namecore/schema/service.goGetLineagereceivesschemaID+rootTypeseparately; descriptor bytes fetched byschemaID; lineage computed fromrootTypecore/schema/lineage_test.goproto/gotocompany/apidocs.swagger.jsontype_namemarkedrequired: true(in path);schema_id/type_nameadded to response definitionstest_helper/seed_lineage_data.goesb-log-entitiescontainer, asserts FQN-based lineage per test caseExample
{ "root_schema": { "namespace_id": "gojek", "schema_id": "esb-log-entities", "type_name": "gojek.esb.types.Location" }, "direction": "downstream", "downstream": [ { "namespace_id": "gojek", "schema_id": "esb-log-entities", "type_name": "gojek.esb.booking.GoKilatBookingLogMessage", "level": 1, "path": [ "gojek.esb.types.Location", "gojek.esb.booking.GoKilatBookingLogMessage" ] } ], "summary": { "downstream_count": 1, "upstream_count": 0, "total_count": 1 } }Testing
Unit tests
go test ./internal/api ./core/schemaEnd-to-end
E2E harness covers:
type_nameFQN downstream traversal (top-level message)type_nameFQN downstream traversal (nested message, e.g.Payment.Receipt)type_nameFQN upstream traversal400on invaliddirectionBreaking changes
/schemas/{schema_name}/lineage→/schemas/{schema_id}/types/{type_name}/lineagetype_nameis now required400 Bad Requestschema_namefield removedschema_id+type_namein both root and lineage nodespathvalues changed