Strip backticks from table name and schema#30240
Conversation
Iceberg quotes namespace/table parts that contain special characters (e.g. the hyphen in `silver-stagingdb`) when it builds the TableIdentifier string that becomes the Glue symlink facet's name. _parse_glue_table_name never accounted for that quoting, so any Glue database name with a hyphen fails to match a catalogued table, even when the table exists in OpenMetadata under the correct name.
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
…, not raw datasets event_has_no_outputs/event_has_no_inputs checked the raw inputs/outputs lists from the OpenLineage event, not whether those datasets actually resolved to OpenMetadata entities. When a job has an unresolvable input (e.g. a raw S3 path with no Glue/Iceberg symlink) alongside a resolvable output, the raw list is non-empty so the pipeline-as-node fallback never fires, and product(input_edges, output_edges) is also empty since input_edges has nothing in it. Result: no lineage edge at all is created for the output, even though it resolved successfully. Check output_edges/input_edges (the resolved lineage nodes) instead, so an unresolvable counterpart no longer blocks the fallback edge.
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
🔴 Playwright Results — 4 pipeline/setup failure(s)✅ 0 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 0 skipped Pipeline and setup failures
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
Code Review ✅ ApprovedStrips backticks from Glue schema and table names to allow proper identification of Iceberg tables with special characters. The PR also corrects OpenLineage fallback logic to verify resolved edges rather than raw datasets. OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Gitar |
Iceberg quotes namespace/table parts that contain special characters (e.g. the hyphen in
silver-stagingdb) when it builds the TableIdentifier string that becomes the Glue symlink facet's name. _parse_glue_table_name never accounted for that quoting, so any Glue database name with a hyphen fails to match a catalogued table, even when the table exists in OpenMetadata under the correct name.Describe your changes:
Fixes #
I worked on ... because ...
Type of change:
High-level design:
N/A — small change.
Tests:
Use cases covered
Unit tests
Backend integration tests
Ingestion integration tests
Playwright (UI) tests
Manual testing performed
UI screen recording / screenshots:
Not applicable.
Checklist:
Fixes <issue-number>: <short explanation>Fixes #<issue-number>above.Summary by Gitar
schemaandnamecomponents in_parse_glue_table_nameto ensure proper matching.input_edgesandoutput_edgesinstead of raw event inputs and outputs.This will update automatically on new commits.
Greptile Summary
This PR improves OpenLineage table matching and lineage fallback behavior. The main changes are:
Confidence Score: 5/5
This looks safe to merge.
Important Files Changed
Reviews (3): Last reviewed commit: "Merge branch 'main' into patch-1" | Re-trigger Greptile
Context used: