Skip to content

Strip backticks from table name and schema#30240

Open
romsharon98 wants to merge 3 commits into
open-metadata:mainfrom
romsharon98:patch-1
Open

Strip backticks from table name and schema#30240
romsharon98 wants to merge 3 commits into
open-metadata:mainfrom
romsharon98:patch-1

Conversation

@romsharon98

@romsharon98 romsharon98 commented Jul 20, 2026

Copy link
Copy Markdown

Iceberg quotes namespace/table parts that contain special characters (e.g. the hyphen in silver-stagingdb) when it builds the TableIdentifier string that becomes the Glue symlink facet's name. _parse_glue_table_name never accounted for that quoting, so any Glue database name with a hyphen fails to match a catalogued table, even when the table exists in OpenMetadata under the correct name.

Describe your changes:

Fixes #

I worked on ... because ...

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

High-level design:

N/A — small change.

Tests:

Use cases covered

Unit tests

Backend integration tests

Ingestion integration tests

Playwright (UI) tests

Manual testing performed

UI screen recording / screenshots:

Not applicable.

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • My PR is linked to a GitHub issue via Fixes #<issue-number> above.
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.
  • For UI changes: I attached a screen recording and/or screenshots above.
  • I have added tests (unit / integration / Playwright as applicable) and listed them above.

Summary by Gitar

  • Bug fixes:
    • Strip backticks from Glue schema and name components in _parse_glue_table_name to ensure proper matching.
    • Correct OpenLineage fallback logic to check input_edges and output_edges instead of raw event inputs and outputs.

This will update automatically on new commits.

Greptile Summary

This PR improves OpenLineage table matching and lineage fallback behavior. The main changes are:

  • Removes surrounding backticks from Glue schema and table names.
  • Uses resolved edges to detect single-sided lineage events.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
ingestion/src/metadata/ingestion/source/pipeline/openlineage/metadata.py Normalizes quoted Glue identifiers and bases single-sided lineage fallback on resolved input and output edges.

Reviews (3): Last reviewed commit: "Merge branch 'main' into patch-1" | Re-trigger Greptile

Context used:

  • Context used - CLAUDE.md (source)
  • Context used - AGENTS.md (source)

Iceberg quotes namespace/table parts that contain special characters (e.g. the hyphen in `silver-stagingdb`) when it builds the TableIdentifier string that becomes the Glue symlink facet's name. _parse_glue_table_name never accounted for that quoting, so any Glue database name with a hyphen fails to match a catalogued table, even when the table exists in OpenMetadata under the correct name.
@romsharon98
romsharon98 requested a review from a team as a code owner July 20, 2026 13:12
@github-actions

Copy link
Copy Markdown
Contributor

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

  • No GitHub issue is linked. Link an issue in the Development section of the PR (or add Fixes #12345 to the description). For a same-org cross-repo issue, add Fixes open-metadata/<repo>#123 to the description.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

…, not raw datasets

event_has_no_outputs/event_has_no_inputs checked the raw inputs/outputs lists from the OpenLineage event, not whether those datasets actually resolved to OpenMetadata entities. When a job has an unresolvable input (e.g. a raw S3 path with no Glue/Iceberg symlink) alongside a resolvable output, the raw list is non-empty so the pipeline-as-node fallback never fires, and product(input_edges, output_edges) is also empty since input_edges has nothing in it. Result: no lineage edge at all is created for the output, even though it resolved successfully. Check output_edges/input_edges (the resolved lineage nodes) instead, so an unresolvable counterpart no longer blocks the fallback edge.
@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Playwright Results — 4 pipeline/setup failure(s)

✅ 0 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 0 skipped

Pipeline and setup failures

  • The build job finished with status failure.
  • Shard detection finished with status failure.
  • The Playwright shard matrix was unexpectedly skipped.
  • No expected Playwright shards were declared.
Shard Passed Failed Flaky Skipped

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@gitar-bot

gitar-bot Bot commented Jul 22, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Strips backticks from Glue schema and table names to allow proper identification of Iceberg tables with special characters. The PR also corrects OpenLineage fallback logic to verify resolved edges rather than raw datasets.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants