Skip to content

resolve entity schema path from the classpath scan instead of a hardcoded map - #30618

Queued
Vishnuujain wants to merge 2 commits into
mainfrom
fix/schema-path-lookup
Queued

resolve entity schema path from the classpath scan instead of a hardcoded map#30618
Vishnuujain wants to merge 2 commits into
mainfrom
fix/schema-path-lookup

Conversation

@Vishnuujain

@Vishnuujain Vishnuujain commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Describe your changes:

Fixes #30533

Entity schemas are discovered by scanning json/schema/entity/, but were loaded by looking the entity name up in a hardcoded folder map that defaulted to data. The two drifted, so six entity types resolved to a folder they do not live in.

  • Removed the hardcoded entity-to-folder map.
  • The classpath scan already reads each schema's real path and threw it away; it now keeps it, and the loader reads from that.
  • Discovery and loading share one source of truth, so they cannot drift again.
  • Unknown entity types (reachable via /fields/{entityType}) keep the previous data fallback, unchanged.

Fixed on main: mcpServer, mcpExecution, aiGovernanceFramework, aiFrameworkControl, auditReport, contextMemory.

Also fixes 9 entity types shipped by downstream modules on the same classpath, which a map in this repo could never cover.

Type of change:

  • Bug fix

Backend integration tests

  • Not applicable (no API changes).

Ingestion integration tests

  • Not applicable.

Playwright (UI) tests

  • Not applicable.

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR is linked to a GitHub issue via Fixes # above.
  • For JSON Schema changes: no schema changes, only how their location is resolved at load time.
  • I have added a test that covers the exact scenario we are fixing.

Reported by @jun-su-lee, who also proposed a fix in #30534. This takes the structural approach they suggested there.

Greptile Summary

The PR replaces hardcoded entity-schema directory mappings with paths captured during classpath discovery.

  • Records each discovered entity type’s schema path and uses it during field extraction.
  • Preserves the existing data directory fallback for undiscovered entity types.
  • Adds deterministic handling and warnings for duplicate entity-type declarations.
  • Adds test-only schema probes and coverage for schemas in previously unsupported directories.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
openmetadata-service/src/main/java/org/openmetadata/service/util/SchemaFieldExtractor.java Replaces the hardcoded schema-directory map with classpath-discovered paths while retaining the legacy fallback.
openmetadata-service/src/test/java/org/openmetadata/service/util/SchemaFieldExtractorTest.java Adds coverage for nonstandard schema directories, duplicate declarations, fallback behavior, and affected entity schemas.
openmetadata-service/pom.xml Adds the isolated schema-probe directory to the unit-test classpath without publishing it in the test JAR.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Scan json/schema/entity] --> B{Entity annotation present?}
  B -- Yes --> C[Record entity type to schema path]
  B -- No --> D[Ignore resource]
  E[Field extraction request] --> F{Discovered path exists?}
  F -- Yes --> G[Load recorded schema path]
  F -- No --> H[Load data/entityType.json fallback]
  C --> F
Loading

Reviews (2): Last reviewed commit: "gitarbot feedback: make duplicate entity..." | Re-trigger Greptile

@Vishnuujain
Vishnuujain requested a review from a team as a code owner July 29, 2026 06:54
@github-actions github-actions Bot added the safe to test Add this label to run secure Github workflows on PRs label Jul 29, 2026
@Vishnuujain Vishnuujain changed the title Fixes 30533: resolve entity schema path from the classpath scan instead of a hardcoded map resolve entity schema path from the classpath scan instead of a hardcoded map Jul 29, 2026
@Vishnuujain Vishnuujain self-assigned this Jul 29, 2026
@gitar-bot

gitar-bot Bot commented Jul 29, 2026

Copy link
Copy Markdown
Code Review ✅ Approved 1 resolved / 1 findings

Derives entity schema paths dynamically from classpath scans instead of using a hardcoded map, addressing the duplicate entity-type resolution finding. No issues found.

✅ 1 resolved
Edge Case: Duplicate entity-type resolution depends on classpath scan order

📄 openmetadata-service/src/main/java/org/openmetadata/service/util/SchemaFieldExtractor.java:179-189
recordSchemaPath uses put(), so when two schemas on the classpath declare the same entity type name (the PR notes downstream modules ship schemas on the same classpath), the last one scanned wins. ClassGraph scan order across jars is not guaranteed stable, so the resolved schema path — and thus which schema's fields load — can vary between runs/deployments; only a warn log flags it. Consider making resolution deterministic (e.g., keep the first-seen path via putIfAbsent, or a defined precedence such as this module over downstream) so behavior is reproducible.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar | Powered by Gitar — free for open source

@github-actions

Copy link
Copy Markdown
Contributor

✅ Playwright Results — workflow succeeded

Validated commit e6fdcf6828cf70af6dc72eaec32568e846a41b80 in Playwright run 30431340920, attempt 1.

✅ 537 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 5 skipped · 🧰 0 lifecycle flaky

Performance

Blocking targets: ✅ met · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 53m 14s

⏱️ Max setup 2m 50s · max shard execution 14m 20s · max shard-job elapsed before upload 17m 59s · reporting 4s

🌐 202.85 requests/attempt · 2.86 app boots/UI scenario · 3.95% common-shard skew

Optimization targets still in progress:

  • Browser traffic was 202.85 requests per attempt (convergence target: fewer than 200).
  • Application boot ratio was 2.86 per UI scenario (1596 boots / 558 scenarios; convergence target: at most 1).
Shard Passed Failed Flaky Skipped Lifecycle failed Lifecycle flaky
✅ Shard chromium-01 102 0 0 3 0 0
✅ Shard chromium-02 88 0 0 0 0 0
✅ Shard chromium-03 100 0 0 0 0 0
✅ Shard chromium-04 109 0 0 0 0 0
✅ Shard data-asset-rules-01 61 0 0 0 0 0
✅ Shard domain-isolation-01 14 0 0 0 0 0
✅ Shard global-state-01 23 0 0 0 0 0
✅ Shard ingestion-01 1 0 0 0 0 0
✅ Shard reindex-01 2 0 0 0 0 0
✅ Shard search-01 10 0 0 0 0 0
✅ Shard search-rbac-01 27 0 0 2 0 0

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@Vishnuujain
Vishnuujain added this pull request to the merge queue Jul 29, 2026
Any commits made after this event will not be merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mcpServer / mcpExecution missing from SchemaFieldExtractor subdirectory map — schema lookup resolves to entity/data/ and fails

2 participants