sap-success-factors#27664
Merged
ulixius9 merged 4 commits intoopen-metadata:mainfrom Apr 27, 2026
Merged
Conversation
Contributor
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
varun-lakhyani
added
the
safe to test
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
Contributor
Jest test CoverageUI tests summary
|
Contributor
🟡 Playwright Results — all passed (14 flaky)✅ 3959 passed · ❌ 0 failed · 🟡 14 flaky · ⏭️ 86 skipped
🟡 14 flaky test(s) (passed on retry)
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
Contributor
|
Contributor
|
Contributor
|
Code Review ✅ Approved 2 resolved / 2 findingsConfiguration updates for SAP SuccessFactors standardize authentication requirements and ensure proper JSON formatting. Username dependencies for BasicAuth and OAuth2 are now enforced, and trailing newline issues have been resolved. ✅ 2 resolved✅ Edge Case: Username not required — BasicAuth/OAuth2 both need it
✅ Quality: Missing newline at end of JSON files
OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
|
|
chirag-madlani
approved these changes
Apr 27, 2026
jatinmasaram
pushed a commit
to jatinmasaram/OpenMetadata
that referenced
this pull request
May 2, 2026
ShubhamOulkar
added a commit
to ShubhamOulkar/OpenMetadata
that referenced
this pull request
May 3, 2026
* Fix PR playwright failures (#27701)
* Fix PR playwright failures
* fix lint check
* Ontology filter persist (#27702)
* Added filter persist overall mode
* Fix name fallback
* Fixed data Assest lineage spec (#27671)
* Removed table version page spec (#27684)
* Removed table version page spec
* lint fix
* Added API polling before adding the badge (#27714)
* hide task form and workflows from global settings (#27690)
* hide task form and workflows from global settings
* fix lint issue
* Fixes #24208: team ownership broken for special-character names and partial name matches (#27453)
* fix: URL-encode ES query_filter and prefer exact name lookup for team ownership
* Fix copilot suggestions
* add dbt fix
* add test for dbt key change
* fix: use historical column snapshot in table version view
* Fix copilot suggestions
* style: apply organize-imports and prettier formatting
* style: apply organize-imports and prettier formatting
* Fix search query issue
* fix failing checks
* Fix styling issue
* revert unnecessary changes
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* fix(description-sanitizer): allow BlockEditor file attachment and callout attributes on div elements (#27658)
* fix(description-sanitizer): allow BlockEditor file attachment and callout attributes on div elements
* addressed gitar comments
* unit test fix
* removed attribute
* fix java check style
* nit
* chore(ui-security): bump uuid and dompurify (#27696)
* chore(ingestion): migrate to ruff for format + isort + unused-import (#27739)
* chore(ingestion): replace black/isort/pycln with ruff
- Swap formatter + import-sorter + unused-import tooling for ruff
(line-length 120, target py3.10) in ingestion + openmetadata-airflow-apis
- Drop dead [tool.mypy] config; basedpyright is the active type checker
- Bump requires-python to >=3.10 to match noxfile and CLAUDE.md (3.9 is
documented as broken on Mac in noxfile.py)
- Bump pre-commit-hooks v2.3 -> v5.0; the new check-json catches four
pre-existing JSON issues now excluded with an inline TODO
- Update Makefile py_format / py_format_check targets to call ruff
* chore(ingestion): grandfather ruff lint violations and apply ruff format
- 253 noqa markers added via 'ruff check --add-noqa' across 128 files,
freezing existing violations so this PR is a tooling-only swap. Per-rule
cleanup tracked in the TODO comment in ingestion/pyproject.toml.
- Bulk reformat from black 22.3 -> ruff format @ line-length 120.
Cosmetic only: imports balanced (-32/+32), structural keywords balanced
(-2221/+2221), no logic changes.
- Star-import rules (F403/F405) globally ignored; refactoring wildcard
imports across connectors is a separate effort.
* chore(ingestion): fix pylint findings surfaced after ruff format
- filters.py: drop redundant parens around re.match(...) in `if`
(C0325 superfluous-parens) — exposed when ruff format unwrapped them
- nosql_adaptor.py: move `# pylint: disable=unused-argument` from the
`column:` line to the `def` line so it covers `table` too (W0613) —
scope was line-based, lost when ruff split params onto multiple lines
- action1xx.py: replace `arguments-differ` with `signature-differs` in
the disable directive (was always wrong code) and drop the now-useless
`unused-argument` suppression (I0021)
* fix(ingestion): make ruff extend-exclude robust to multi-root invocations
CI's `make py_format_check` runs from the repo root and passes both
`ingestion/` and `./openmetadata-airflow-apis/` to ruff in a single
invocation. With multiple root paths, ruff's parallel file discovery
races on extend-exclude matching against the project root, so files
under `ingestion/src/metadata/generated/` were intermittently scanned
and produced ~830 I001 violations.
20-run repro: 10/20 fail without the fix, 20/20 pass with the fix.
Each excluded directory now appears twice in extend-exclude:
- the project-root-relative pattern (cwd = ingestion/)
- the prefixed pattern (cwd = repo root, multi-root invocation)
* chore(ingestion): address gitar-bot findings + cross-version pylint disable
- openmetadata-airflow-apis/pyproject.toml: switch coverage to module-name
source + [tool.coverage.paths] glob remap (matches the ingestion pattern).
Drops the hardcoded `env/lib/python3.9/site-packages/...` source path,
which broke after the requires-python bump to 3.10. (Finding 1)
- ingestion/setup.py: remove dead python_version<'3.9' / >='3.9' guards on
mysql-connector-python and testcontainers; promote locust to a regular
test dep (was conditionally added under sys.version_info >= (3, 9)). Also
remove the now-unused `import sys`. (Finding 3)
- ingestion/src/metadata/great_expectations/action1xx.py: cover both
arguments-differ (great_expectations 0.18.x parent) and signature-differs
(great_expectations 1.x parent) in the pylint disable comment, since
CI installs 0.18.x and local often has 1.x. unused-argument covers the
unused action_context. The opposite rule fires as I0021 useless-suppression
on each environment, which is informational and does not affect pylint's
exit code.
* feat(ui): Incident test case status & severity migrate from mui to core components (#27595)
* feat(ui): Refactor incident test case status and severity with core components
Extract InlineIncidentStatus subcomponents (chip, popover shell, header, scroll list,
failure reason chips, icons). Align status dropdown and incident UX with design tokens.
Expand unit tests for InlineTestCaseIncidentStatus and InlineSeverity.
Made-with: Cursor
* fix failing checks and tests
* address comments
* address comments
* minor fixes
* fix failing check
* fix failing test
* fix(ui): match Tier/Certification tag FQNs by prefix, not substring (#27689) (#27700)
Closes #27689
`getTagAssetsQueryFilter` used `fqn.includes('Tier.')` / `.includes('Certification.')`
to pick which index field to query, so any tag whose classification name *ended*
with "Tier" or "Certification" (e.g. `DataTier.Bronze`) was routed to the
`tier.tagFQN` / `certification.tagLabel.tagFQN` fields and returned no assets.
Anchor the check to the start of the FQN using `FQN_SEPARATOR_CHAR` so only the
built-in Tier and Certification classifications hit the special fields.
Co-authored-by: Siddhant <siddhant@MacBook-Pro-621.local>
* feat(connections): add ConnectionsRouterClassBase for pluggable connection routing (#27662)
* feat(connections): add ConnectionsRouterClassBase for pluggable connection routing
* address gitar
* fix checkstyle
* add edit ingestion path
* fix checkstyle
* Fix: recoverable 'I/O reactor has been shut down' on OpenSearch HC5 transport (#27698)
* Fix I/O reactor has been shutdown
* Add Safe Consumer Class
* Fix review comments
* fix(playwright): fix incorrect selectors in UserCreationWithPersona.spec.ts (#27679)
* sap-success-factors (#27664)
* fix(advanced-search): move entityStatus to common config with hard-coded list values (#27642)
* fix(advanced-search): move entityStatus to common config with hard-coded list values
* lint and unit test fix
* addressed PR comment
* fallback for datacontract status check (#27665)
* fallback for datacontract status check
* refactor: extract TestCaseEntry type to eliminate duplicated inline annotations
Co-authored-by: shrabantipaul-collate <253027805+shrabantipaul-collate@users.noreply.github.com>
* address review comments
---------
Co-authored-by: Shrabanti Paul <shrabantipaul@Shrabantis-MacBook-Pro.local>
Co-authored-by: Gitar <noreply@gitar.ai>
Co-authored-by: shrabantipaul-collate <253027805+shrabantipaul-collate@users.noreply.github.com>
* fix flaky contract semantic rules version (#27704)
Co-authored-by: Shrabanti Paul <shrabantipaul@Shrabantis-MacBook-Pro.local>
* Fix server-side role search behavior across role selectors (#27737)
* Fix server-side role search behavior across role selectors
* address gitar
* fix failing spec
* Add relevent terms relation in term details page (#27751)
* Add relevent terms relation in term details page
* fix lint issue
* Add warning message on search of terms (#27759)
* Add warning message on search of terms
* nit
* feat: add FormBuilderV1 component to common components (#27075)
* feat: add FormBuilderV1 component to common components
Move Collate's CoreFormBuilder (RJSF-based MUI form builder) into the
OpenMetadata submodule as FormBuilderV1, including all fields, templates,
and widgets. Updated external imports to use submodule-relative paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* add tests
* fix lint
* fix feedbacks
* fix: add aria-labels to icon-only buttons in FormBuilderV1 templates
Agent-Logs-Url: https://github.com/open-metadata/OpenMetadata/sessions/814ea72a-5a8b-4181-bce3-9ab07ac8571d
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
* fix
* fix
* lint
* fix
* lint
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
* fix(migration): introduce ServiceLoader SPI for extension migration providers (#27760)
* fix(migration): align PG v200 migration with MySQL #27687 (#27757)
Comment out migrateThreadTasksToTaskEntity in the PostgreSQL v200
migration to match the MySQL fix from PR #27687. The original fix was
only applied to MySQL, leaving PostgreSQL deployments exposed to the
same Collate CI failure.
* chore(ingestion): enable basedpyright across the codebase via baseline (#27755)
* chore(ingestion): enable basedpyright across the codebase via baseline
Removes the ~25 paths from `[tool.basedpyright] ignore` (which excluded
roughly 90% of the codebase from type checking) and grandfathers the
existing violations into a baseline file. New violations in any
previously-ignored file now fail CI.
Changes:
- ingestion/pyproject.toml: drop the entire `ignore = [...]` block
- ingestion/setup.py: bump `basedpyright~=1.14` to `~=1.39.0`
- ingestion/.basedpyright/baseline.json (new, ~13MB): captures the
starting violation set (~18.8K errors + ~37.4K warnings) so the
migration is behavior-preserving. Regenerate with
`cd ingestion && basedpyright -p pyproject.toml --baselinefile
.basedpyright/baseline.json --writebaseline`. basedpyright analysis
has minor non-determinism (similar to ruff's), so re-running
--writebaseline a few times converges the baseline.
- ingestion/noxfile.py: pass `--baselinefile .basedpyright/baseline.json`
to the basedpyright invocation in the `static-checks` session so CI
honors the grandfathering. CI already runs the session via
`cd ingestion && nox --no-venv -s static-checks` (py-tests.yml).
- ingestion/Makefile: `make static-checks` now delegates to
`nox -s static-checks` so local invocations match CI exactly. Also
drops the dead Python 3.9 / OM_SKIP_SDK_PY39 branch (we require
Python >=3.10 since the previous modernization PR).
- .gitignore: add `.serena/` (local language-server cache)
* chore(ingestion): add nox to the dev dependency set
The static-checks Makefile target and the py-tests CI job both delegate
to `nox -s static-checks`, but nox was being installed as a separate
side step (`pip install nox` in `install_dev_env`, `uv pip install nox`
in the test-environment composite action). Listing it in dev extras
means a plain `pip install ingestion[dev]` brings it in.
* chore(ingestion): pin basedpyright analysis to py3.10; CI runs once
Following the basedpyright + multi-Python-version research:
- ingestion/pyproject.toml: add `pythonVersion = "3.10"` to
[tool.basedpyright] so type-checking always analyzes for the lowest
supported Python version. Forward-incompatible code (tomllib usage,
PEP 695 generics, etc.) is caught at type-check time regardless of
which Python interpreter runs the checker.
- .github/workflows/py-tests.yml: gate the "Run Static Checks" step on
`matrix.py-version == '3.10'`. With pythonVersion pinned, results are
identical across the matrix; running once avoids redundant work and
keeps the baseline file deterministic. Unit tests still run on the
full 3.10/3.11/3.12 matrix to verify runtime compatibility.
- ingestion/.basedpyright/baseline.json: regenerated cleanly with the
new pythonVersion config (~18.8K errors / ~37.3K warnings, similar
scale to the previous baseline). Aligns with the canonical
type-check-on-floor / test-on-matrix pattern used by Pydantic, CPython,
and other major Python projects.
* chore(ingestion): pin basedpyright pythonPlatform to Linux + regen baseline
CI's previous run still surfaced ~9 issues (2 errors + 7 warnings) that
weren't in the baseline. Root cause: my local environment differs from
CI's in three ways that affect type inference — Python interpreter
(3.11 vs 3.10), platform (Darwin vs Linux), and pip-resolved package
versions (couchbase, avro, trino, sqlalchemy stubs all differ slightly).
This commit closes the platform gap and regenerates the baseline from a
fresh CI-equivalent environment:
- ingestion/pyproject.toml: add `pythonPlatform = "Linux"` to
[tool.basedpyright] so type-checking uses the Linux subset of stdlib /
third-party stubs regardless of where the analyzer runs.
- ingestion/.basedpyright/baseline.json: regenerated against a fresh
Python 3.10 venv installed via `uv pip install ingestion[test]` (the
same install path CI's setup-openmetadata-test-environment composite
action uses). New scale: ~18.7K errors / ~37.5K warnings — same
ballpark as the previous baseline, with column positions now matching
CI's environment.
Local-developer note: when running `make static-checks` from a venv
that doesn't mirror CI exactly (e.g. macOS, Python 3.11, different
package versions), you may see drift errors. The supported workflow for
regenerating the baseline is to mirror CI:
python3.10 -m venv /tmp/ci-mirror
source /tmp/ci-mirror/bin/activate
uv pip install --upgrade pip "setuptools<81"
uv pip install --no-build-isolation "cx_Oracle>=8.3.0,<9"
uv pip install -e "ingestion[test]"
uv pip install "basedpyright~=1.39.0" nox
cd ingestion && basedpyright -p pyproject.toml \
--baselinefile .basedpyright/baseline.json --writebaseline
* chore(ingestion): drop pythonPlatform pin and regen baseline from CI-mirror
The previous attempt added `pythonPlatform = "Linux"` thinking it would
make the local-generated baseline match CI. It did the opposite — Linux
platform stubs activate additional conditional code paths that weren't
analyzed before, so CI saw 101 errors instead of the prior 2 errors.
Reverting:
- Drop `pythonPlatform = "Linux"` from [tool.basedpyright]. Without it,
basedpyright analyzes for the host platform; on CI's ubuntu-latest
runner that's Linux automatically, but type-stub coverage stays the
same as before (matching the d9196dff6b baseline).
- Regenerate ingestion/.basedpyright/baseline.json against a fresh
Python 3.10 venv installed via `uv pip install ingestion[test]`
(mirroring CI's setup-openmetadata-test-environment composite action).
~18.8K errors / 37.7K warnings captured — same scale as the working
d9196dff6b version.
Local-developer note: any baseline regeneration done on macOS will drift
from CI's Linux env (different transitive package versions, different
stubs). The supported local mirror procedure:
python3.10 -m venv /tmp/ci-mirror
source /tmp/ci-mirror/bin/activate
uv pip install --upgrade pip "setuptools<81"
uv pip install --no-build-isolation "cx_Oracle>=8.3.0,<9"
uv pip install -e "ingestion[test]"
uv pip install "basedpyright~=1.39.0" nox
cd ingestion && basedpyright -p pyproject.toml \\
--baselinefile .basedpyright/baseline.json --writebaseline
* chore(ingestion): regen baseline from full CI install (mac arm64 mirror)
Prior CI-mirror only installed [test], skipping [all] and the four
--no-deps SA pins (sqlalchemy-redshift/databricks/ibmi, pydoris-custom).
That left ~75 connector packages out of the analysis env, so basedpyright
couldn't resolve types from databricks.sqlalchemy, GE 0.18 Batch,
sklearn BaseEstimator, airflow SQLAlchemy models, pandas/numpy stubs,
etc. CI saw 129 errors absent from the baseline.
Regenerated against a fresh py3.10 venv that mirrors
.github/actions/setup-openmetadata-test-environment exactly:
uv pip install ./ingestion[dev]
make generate
uv pip install "setuptools<81"
uv pip install --no-build-isolation "cx_Oracle>=8.3.0,<9"
uv pip install --no-deps sqlalchemy-redshift==0.8.14 \
sqlalchemy-databricks==0.2.0 \
sqlalchemy-ibmi==0.9.3 \
pydoris-custom==1.1.0
uv pip install ./ingestion[all]
uv pip install ./ingestion[test]
uv pip install nox
First run: 128 errors, 272 warnings — within 1 error of CI's 129/272.
Wrote baseline with 56,100 entries across 1,035 files. Verify run with
the new baseline reports 0/0/0.
macOS arm64 vs Linux x86_64 wheel resolution may leave a small residual
(~3-7 errors per the d9196dff6b precedent). Re-run --writebaseline 2-3x
if any show up in CI.
* chore(ingestion): silence avro.py:95 basedpyright residual
CI's Linux fastavro stub returns Schema as `str | List[Any]`, while
the macOS arm64 wheel narrows to `str` — the only error not absorbed
by the regenerated baseline. Add a targeted pyright: ignore on the
parse_avro_schema call instead of broadening behavior.
* chore(ingestion): tolerate cross-platform pyright ignore drift
CI's `--baselinemode=lock` (default) requires the baseline to match
exactly — neither up nor down. Two related issues:
1. The avro.py noqa silenced not just the surfaced error but 10
cascading entries at line 95 (sub-errors propagating from the
unresolved `schema` arg type). Baseline went `down by 10` → lock
violated → exit 3 even with `0 errors` reported. Regenerate baseline
so the 10 stale entries are dropped.
2. The macOS arm64 fastavro stub doesn't surface that error in the
first place, so basedpyright treats the noqa as
`reportUnnecessaryTypeIgnoreComment` locally — causing the opposite
lock mismatch on CI (a warning entry that doesn't exist there).
Disable the rule so platform-specific residuals can land without
flapping between local and CI.
* chore(ingestion): use --baselinemode=discard for cross-platform tolerance
CI's implicit default is `lock`, which fails on any baseline change in
either direction (errors going up *or* down) via console.error → exit 3.
That cannot accommodate macOS arm64 vs Linux x86_64 stub drift: a
baseline regenerated locally always carries some entries that don't fire
on CI (and vice versa).
`auto` would tolerate the drift but silently overwrites the baseline
file — unacceptable in CI, where unreviewed changes never get committed
back.
`discard` is the right balance:
- New errors not in the baseline still fail the run (early-return path
in BaselineHandler.write before the lock/discard branch).
- Stale baseline entries (errors that no longer fire on the current
platform) print an info message and exit 0.
- The baseline file is never modified.
* fix(it): stop flaky integration tests (#27649) (#27651)
* fix(it): stop two flaky integration tests (#27649)
GlossaryOntologyExportIT: mark @Isolated. @BeforeAll flips RdfUpdater (a
JVM-wide singleton) on, which makes every concurrent test class start
doing synchronous Fuseki writes on entity create, saturating the
Dropwizard thread pool and causing 60s request timeouts. @Execution
(SAME_THREAD) alone only serialises within this class.
WorkflowDefinitionResourceIT#triggerWorkflow_SDK: drop the redundant
waitForWorkflowDeployment call — the create path already waits. Add
descriptive aliases to the two await() polls so the next flake tells
us which FQN or workflow name actually timed out instead of an
anonymous lambda.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): never skip live indexing during reindex (#27649)
Live search indexing was silently skipped whenever a reindex job was in
RUNNING/READY/STOPPING state. SearchRepository.createEntityIndex() and
six sibling methods consulted SearchIndexRetryQueue.isEntityTypeSuspended()
and returned early with nothing written, nothing enqueued — entities
vanished from search until a future reindex happened to cover them.
The retry worker doubled down: when the scope refresh observed an active
job, it purged the retry queue; and processRecord() deleted records
whose type was suspended. So even manually enqueued retries were wiped.
This is how the #27649 flake surfaced: AppsResourceIT triggers
SearchIndexingApplication runs and its best-effort 30s wait silently
swallows timeouts. If a run was still RUNNING when AppsResourceIT
finished, the next class in the sequential fork
(WorkflowDefinitionResourceIT) inherited the suspension and its
freshly-created tables were never indexed — waitForEntityIndexedInSearch
then timed out at 120s. Same mechanism bites real users mid-reindex in
production.
Remove the suspension mechanism entirely:
* SearchRepository — drop the 8 isEntityTypeSuspended() early-returns;
the client-availability path already enqueues for retry on its own.
* SearchIndexRetryWorker — drop refreshReindexSuspensionScopeIfNeeded()
and the suspension branches in processRecord(); remove the retry-queue
purge on suspendAll.
* SearchIndexRetryQueue — delete the updateSuspension / clearSuspension
/ isEntityTypeSuspended / isStreamingSuspended / isSuspendAllStreaming
/ getSuspendedEntityTypes API and the static AtomicBoolean /
AtomicReference they backed.
* Drop the two IT cases that asserted the removed behaviour.
Live writes now always reach the search client; reindex and live
writes both target the same indices as before. Version conflicts
between the two paths (stale reindex batch overwriting a newer live
write) remain possible as they did before suspension was introduced —
that is the race suspension was meant to dodge, but dropping writes
altogether was worse than the race.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): route live writes to staged index during reindex (#27649)
The distributed reindex has a TOCTOU: partitions read from a DB
snapshot at T0 and write to a staged index, then at T1 (seconds
later) the alias is atomically swapped from the old index to the
staged one and the old index is deleted. Any entity that live-writers
create between T0 and T1 goes via the alias → old index, and is
destroyed when that old index is deleted post-swap.
The CI log for #27649 shows this directly:
10:13:35 staged table_search_index_rebuild_…_215646 built from snapshot
10:13:40 POST /v1/tables table1_gold → written to alias target
(old index _179670)
10:13:40 table2_silver, table3_bronze, table4_brass all written to
old index _179670
10:13:42 Atomically swapped aliases from [_179670] to _215646
10:13:42 Successfully deleted index _179670
10:13:43+ waitForEntityIndexedInSearch polls, finds nothing, times
out at 2 min
Removing the silent-skip suspension mechanism in the previous commit
exposed this race (it had been hidden by dropping the writes outright,
which was strictly worse).
Route live writes to the staged index during the reindex window:
* SearchRepository gains an activeStagedIndices map (entityType →
stagedIndex) plus register/unregister/resolveWriteIndex. Writes
resolve to the staged index when one is registered for the type,
otherwise to the canonical alias — the existing behaviour.
* DefaultRecreateHandler.recreateIndexFromMapping registers the
staged index as soon as it is created; finalizeReindex and
promoteEntityIndex unregister it on every exit path (successful
swap, swap failure, failed-reindex delete, exception).
* Every live-write path in SearchRepository — createEntityIndex,
createEntitiesIndex, indexTableColumns, indexColumnsForTables,
updateEntityIndex, createTimeSeriesEntity, updateTimeSeriesEntity,
deleteEntityIndex, deleteEntityByFQNPrefix, deleteTimeSeriesEntityById
— goes through resolveWriteIndex instead of reading the canonical
alias directly.
During a reindex, live writes land in the index that the alias will
promote to; after the swap the alias points to that same index and
subsequent writes continue to reach the same place. Old-index deletion
no longer discards fresh data.
Note: searches through the alias during the brief reindex window (<
seconds in the CI log) can miss a write until the swap lands — an
acceptable trade compared to silently dropping the write or losing it
on deletion. The #27649 test tolerates this because its 120s poll
spans many swap cycles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): re-register SearchIndexHandler on every SearchRepository init (#27649)
The previous commit routed live writes through resolveWriteIndex so they
land in the staged index during reindex. The CI log for the next run
showed the register/unregister fire correctly, but the live writes to
tables still went to the canonical alias — as if activeStagedIndices
was empty for the entity type.
Root cause: stale handler pointing at a stale SearchRepository.
TestSuiteBootstrap creates SearchRepository three times (migration,
createIndices, and finally the embedded OpenMetadataApplication). Each
constructor calls registerSearchIndexHandler → new SearchIndexHandler(this)
→ dispatcher.registerHandler(…). EntityLifecycleEventDispatcher.
registerHandler silently SKIPS if a handler with the same name already
exists (see EntityLifecycleEventDispatcher.java:80-86), so the dispatcher
keeps the FIRST SearchIndexHandler forever — bound to the migration-time
SearchRepository.
Meanwhile DefaultRecreateHandler.registerStagedIndex writes into
Entity.getSearchRepository(), which by then is the third (current)
instance. Live writes flowing through the stale handler never see that
entry; resolveWriteIndex falls through to the canonical alias; the alias
swap at the end of the reindex drops the writes, same as before.
Fix: unregister any existing SearchIndexHandler by name before
registering the new one. The latest-constructed SearchRepository always
owns the handler delivered through the dispatcher, so its
activeStagedIndices is the one consulted on every live write.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): centralize staged-index routing on canonical name (#27649)
Re-key activeStagedIndices by canonical index name (e.g.
openmetadata_table_search_index) instead of entity type, and route
every live-write site through a single getWriteIndexName(IndexMapping)
helper.
Why
- Previous routing went through resolveWriteIndex(entityType, mapping)
but only at hand-picked call sites. Several write paths still
resolved indexMapping.getIndexName(clusterAlias) directly and
bypassed routing — bulkIndexPipelineExecutions, deleteByScript,
softDeleteOrRestoreEntity, propagateToDomainChildren,
updateEntityCertificationInSearch, propagateToRelatedEntities (PAGE),
deleteTableColumns, updateTableColumnsInheritedFields. Any reindex
in flight could lose those writes on the alias swap.
- Keying by canonical index name lets any write site resolve correctly
even without entity type in scope (FQN-prefix deletes, child
propagation, script updates).
What
- activeStagedIndices: Map<canonicalIndexName, stagedIndexName>.
- registerStagedIndex(entityType, stagedIndex) now resolves the
canonical name from the IndexMapping before storing.
- New getWriteIndexName(IndexMapping) is the single point of
resolution; routeToStagedIfActive(String) handles raw alias names
(e.g. pipeline_status_search_index resolved via
getIndexOrAliasName).
- Replaced every direct indexMapping.getIndexName(clusterAlias) for
writes with getWriteIndexName(indexMapping). Admin/setup paths
(createIndex/updateIndex/deleteIndex/createOrUpdateIndexTemplate)
intentionally keep canonical names — they manage the alias itself.
- Cascade ops on shared aliases (GLOBAL_SEARCH_ALIAS,
DATA_ASSET_SEARCH_ALIAS, child aliases) are not entity-scoped and
cannot route to a single staged index; left untouched.
- resolveWriteIndex(entityType, mapping) preserved as a thin wrapper
for binary compatibility.
Also runs spotless:apply on the file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(it): bump DB sort memory + search-wait ceilings (#27649)
Two CI failures observed under the parallel-tests fork load on the
post-centralization run:
1. TagResourceIT line 161 (listEntities) — the server returned 500
wrapping "java.sql.SQLException: Out of sort memory, consider
increasing server sort buffer size" from TagDAO.listAfter. The
query joins tag → entity_relationship → classification and orders
by tag.name,tag.id; with the tag table accumulating across many
parallel test classes (reuseForks=true), MySQL's default 256KB
sort_buffer_size overflows. Bump it to 8MB. Add a parallel
work_mem=32MB bump to the postgres command for the same query.
2. TagResourceIT line 1 — Awaitility timeout at 1m30s waiting for a
freshly created tag to appear in search index. Five inherited
waits in BaseEntityIT had a 90s ceiling while the sibling
checkCreatedEntity already used 180s. Standardise on 180s — under
tag-scale data the alias swap that the staged-index routing
depends on can take longer than 90s in slow CI workers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): address Copilot/Gitar review on staged-index routing (#27649)
* DefaultRecreateHandler.finalizeReindex / promoteEntityIndex — wrap
the entire promote block in try/finally so unregisterStagedIndex
always runs, including on swap failure, empty aliasesToAttach, and
exceptions. Without this the routing map could be left pointing at
a staged index nobody reads from, silently diverging live writes
from search results until the next reindex (Copilot, multiple
comments).
* SearchRepository.resolveWriteIndex — deprecate. The entityType
argument is unused; getWriteIndexName(IndexMapping) is the single
resolution point now (Copilot + Gitar).
* SearchRepository.routeToStagedIfActive — tighten the Javadoc to
state explicitly that it expects a canonical index name and that
short/parent aliases are passed through unchanged (Copilot).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): fan out cross-alias update-by-query to staged indices (#27649)
The four bulk update-by-query operations rooted on shared aliases —
updateAssetDomainsForDataProduct, updateAssetDomainsByIds,
updateDomainFqnByPrefix, updateAssetDomainFqnByPrefix — hardcoded their
target to GLOBAL_SEARCH_ALIAS / Entity.DOMAIN. During an in-flight
reindex those updates landed on the about-to-be-discarded active
index only; on alias swap, the new staged index (built from a DB
snapshot taken before the script ran) replaced it and the script's
effect was lost. Copilot called this out four times.
Add SearchRepository.getWriteFanoutTargets(aliasOrIndex) — returns the
caller's alias plus every currently-staged index. Pass that list to
req.index(...) on all four methods in both OpenSearchEntityManager and
ElasticSearchEntityManager. The OS/ES update-by-query API natively
takes a list, so the fan-out is one request per call.
The scripts these methods run are idempotent (UPDATE_ASSET_DOMAIN_SCRIPT
checks `exists` before adding a domain; UPDATE_DOMAIN_FQN_BY_PREFIX_SCRIPT
walks the array and rewrites in place), so applying them again to the
staged index — even if the staged copy of the document already reflects
the latest DB state — converges to the same result.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): scope fan-out to canonical input vs multi-entity alias (#27649)
Previous getWriteFanoutTargets always appended every staged index,
which made entity-scoped update-by-query calls (e.g.
updateDomainFqnByPrefix targeting only the domain canonical index)
fan out onto unrelated staged indices. Adds avoidable load on every
currently-reindexing entity type for an update that should touch one
index.
Branch the implementation on whether the input is a known canonical
entity index name. If yes, only the matching staged index is added.
If no — i.e. the caller is hitting a multi-entity alias such as
GLOBAL_SEARCH_ALIAS — every staged index is added because the
update's match query can hit documents from any reindexing type.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Chore(UI): Fix playwright test failures in AUTs (#27706)
* Fix the failures in ExplorePageRightPanel, EntityPermissions, CustomProperties specs
* Fix checkstyle
* Fix customizeDetailsPage spec
* Fix the test flakiness and failures
* fix checkstyle
* Added column icon in explore page (#27672)
* Added column icon in explore page
* updated the column icon in all the pages
* address gitar comment
* added column with bg icon
* fix the column icon color in explore page
* addressed gitar comment
* rename the column icon
* removed unused column icon
* lint fixes
* chore: move warning logs to error logs (#27728)
* chore: move warning logs to error logs
* chore: fix python linting
* ISSUE #27718 - make data type required for OpenMetadata test types (#27721)
* fix: make data type required for OpenMetadata test types
* Update openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/DataQuality/TestLibrary.spec.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* chore: fix failing playwright tests
* chore: ran typescript linting
* chore: fix test library playwright failure
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Chore(deps-dev): Bump postcss in /openmetadata-ui/src/main/resources/ui (#27729)
Bumps [postcss](https://github.com/postcss/postcss) from 8.4.31 to 8.5.10.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.4.31...8.5.10)
---
updated-dependencies:
- dependency-name: postcss
dependency-version: 8.5.10
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harsh Vador <harsh.vador@somaiya.edu>
* chore(ingestion): drop pylint, expand ruff (#27774)
* chore(ingestion): drop pylint, expand ruff to Stage 2c
Replace pylint with a coherent ruff-only stack (Stage 2c of the modernize
roadmap). Pylint is dropped from dev deps and CI workflows; ruff selected
ruleset expanded to ~22 families covering style, bug catchers, hygiene,
and the pylint port (PLE/PLC/PLW/PLR with the noisy "too-many-X"
complexity caps + magic-value disabled).
What's selected (with rationale in pyproject.toml):
E, W, F, I, N — style + correctness baseline + naming
UP — pyupgrade (py>=3.10 modernizations)
B, C4, C90, RET, SIM, TRY — bug catchers
PIE, ICN, T20, TC, TID, PTH, PERF — hygiene
PLE, PLC, PLW, PLR — pylint port (PLR complexity caps ignored)
RUF — ruff-native (incl. RUF100 unused-noqa)
What's removed:
- .pylintrc (root) — duplicate of the ingestion pylint config
- [tool.pylint.*] block in ingestion/pyproject.toml (~140 lines)
- ingestion/plugins/{print_checker,import_checker}.py + tests + README
(replaced by built-in T20 + TID251 banned-api respectively)
- pylint dep from ingestion/setup.py and openmetadata-airflow-apis/pyproject.toml
- `make lint` Makefile target + the pylint invocation in py_format_check
- dead pylint TODO comment + ignored test entry in noxfile.py
Cwd-stable config: ruff is invoked both from the repo root (pre-commit,
CI) and from ingestion/ (`make py_format_check`). The `src`,
`extend-exclude`, and per-file-ignores entries are listed twice — once
relative to ingestion/ and once with the `ingestion/` prefix — so
first-party isort detection and exclusions match in both invocations.
Grandfathering: ran `ruff check --add-noqa` once + format-stable
iteration. ~12,130 noqa directives across ~1,400 files. Cleanup is
deferred to follow-up PRs that drop noqas one rule at a time.
Documentation sweep: replaced `make lint` references in CLAUDE.md,
AGENTS.md, DEVELOPER.md, copilot-instructions, and 6 SKILL files with
the apply+verify shape `make py_format && make py_format_check`.
`make py_format` is NOT a strict superset of pylint — it only applies
auto-fixable violations; `make py_format_check` catches the rest.
Basedpyright baseline regenerated: ruff format reflowed multi-line
signatures in ~70 files, shifting type-error column positions. The
basedpyright baseline matches by (file path, error code, range), so
column shifts caused 19 entries to mis-align. Net diff is small
(154 lines in/out of the 13MB baseline.json) — purely positional.
Verified locally:
- make py_format_check → All checks passed
- nox --no-venv -s static-checks → 0 errors, 0 warnings, 0 notes
* chore(ingestion): finish ruff swap — nox lint session + skill docs
Three remaining stale-tooling references after Stage 2c:
- `ingestion/noxfile.py` `lint` session was still calling `black --check`,
`isort --check-only`, `pycln --diff`. Those tools aren't installed
anywhere (we dropped them from dev deps). Replace with the ruff
equivalents that mirror `make py_format_check`.
- `skills/standards/code_style.md`: stack listed as `black + isort +
pycln`; line length claimed 88 (black default). Both wrong: stack is
ruff, line length is 120.
- `skills/connector-building/SKILL.md`: `make py_format` comment said
`# black + isort + pycln`. Same swap.
* chore(ingestion): keep main's baseline + globally ignore TRY400
Per gitar-bot's review on PR #27774:
1. Main's PR #27728 promoted ~60 `logger.warning()` → `logger.error()`
inside `except` blocks. Those changes landed on main with their own
baseline updates. Our PR doesn't promote anything — the merge from
origin/main brought those `error` calls along with their baseline
entries.
The bot interpreted the `# noqa: TRY400` we added next to those lines
as us silencing the rule case-by-case. Cleaner: globally ignore
TRY400 in pyproject.toml, with a comment explaining why the codebase's
`logger.error(...)` + separate `logger.debug(traceback.format_exc())`
pattern is intentional. Strip ~430 per-line `# noqa: TRY400` markers
from source.
2. Document that `S101` in `per-file-ignores` is a forward-looking
entry — flake8-bandit (`S`) is not yet selected, so the rule is
no-op today; the entry stays so when `S` lands later, tests don't
immediately error.
Reverts the platform pin and Linux Docker–generated baseline. Keep
main's baseline intact and let CI surface the exact column-shifted
entries; the team will decide whether to fix in-place (revert format
on affected files) or add per-line `# pyright: ignore` markers.
* chore(ingestion): regen baseline for new connector type debt
Main's baseline was stale relative to recently-added connectors
(McpConnection, CustomDriveConnection) that lack common attributes
like `hostPort`, `database`, `catalog` etc. — all sites that access
those attributes via the union-typed `serviceConnection.root.config`
fire `reportAttributeAccessIssue` errors that aren't baselined.
71 errors + 58 warnings absorbed. Local macOS regen; pushing to see
CI's drift count. Per the basedpyright-baseline-and-ci PR experience,
macOS↔Linux column drift on this size of regen has historically been
1-7 residuals.
* fix(sas): drop logger.info call leaking user password (#27786)
`SASClient.get_token` logged the cleartext password (and username + token
response body) at INFO level on every auth call. The line appears to be
debug spam left in by mistake.
Verified leak path (static + runtime):
- `__init__` calls `get_token(serverHost, username, password.get_secret_value())`
- `password.get_secret_value()` returns the raw user password (Pydantic
SecretStr unwrap)
- `get_token` interpolates that into an f-string at INFO
- Triggered every time a SAS connector is instantiated
- `ingestion_logger()` has no redaction filter, so the cleartext reaches
whatever sink (stdout, file, airflow logs, CloudWatch, etc.) is configured
* Feat/UI/refactor add domain form (#26951)
* feat(ui): refactor AddDomainForm to use core components library
Migrate AddDomainForm from Ant Design to @openmetadata/ui-core-components
using HookForm, getField, and FieldTypes. Add onFocus prop support to
AutocompleteBase, add rules prop to FormField, and export form-field
components from the core library.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* feat(ui): enhance AddDomainForm Select, avatars, and glossary terms
- Switch FieldTypes.SELECT to use Select with Select.Item children
for proper icon/avatar/supportingText support
- Pass fontSize through SelectContext so items respect the fontSize prop
- Show colored-initial avatars for users and team icons via Avatar
- Show tag color dots using Dot component on tag options
- Replace custom glossary terms autocomplete with MUIGlossaryTagSuggestion
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(ui): resolve ESLint, TypeScript, and review issues in AddDomainForm PR
Fix jsx-sort-props errors in form-field.tsx, replace `any` with `Domain`
type, use MutableRefObject to fix readonly ref assignment, add missing
fontSize to SelectContext providers, and restrict experts field to
user-only options matching original behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(ui): restructure form-field into typed, single-responsibility modules
Split the monolithic form-field.tsx (~1290 lines) into focused files:
- fields/color-picker-field.tsx and fields/icon-picker-field.tsx for standalone field components
- render-field-element.tsx for the central field type dispatch
- form-field.tsx reduced to thin Field/FormFields wrapper (~95 lines)
Replaced Record<string, unknown> FieldPropsMap with a properly typed interface,
eliminated all `as` casts and `unknown` types, and removed the unnecessary
select normalization layer (consumers pass correctly shaped SelectItemType data).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ui): store full SelectItemType objects in form state and fix TS errors
The form field system now stores entire SelectItemType objects instead of
just IDs, matching the data format expected by form submit handlers.
Also adds filterOption, onFocus, size, fontSize to FieldPropsMap and
fixes type compatibility in AddDomainForm.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ui): add missing MUIGlossaryTagSuggestion mock in AddDomainForm tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ui): resolve icon rendering and i18n issues in domain form
- Fix getEntityAvatarProps to resolve icon names via ICON_MAP as
placeholderIcon, so icons like "Bank" render on domain/DP pages
instead of showing the default placeholder
- Extract transformFormData from handleFormSubmit and reuse in
validateFields so it returns API-ready payloads
- Remove hardcoded English strings in core-components form fields;
accept labels as props for i18n support
- Fix unstable onBlur ref in icon-picker causing effect listener churn
- Remove no-op useMemo in icon-picker
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* lint
* fix(ui): CI failures on add-domain-form refactor
- lint/prettier autofix on core-components files flagged by CI
- wrap Domain Type field with data-testid="domainType" so Playwright
can locate the new core-components Select trigger
- update fillDomainForm helper to click the Select trigger button and
use exact option match instead of the old Ant combobox locator
- update DomainUIInteractions special-character validation test to
assert the error text instead of the gone .ant-form-item-explain-error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(playwright): resolve all domain form test failures after refactor
Fix 4 root causes: add missing domain-select testid, scope domainType
selector to form to avoid strict mode violation with KnowledgePanel,
use role-based selectors for tag options in new Autocomplete, and fix
special characters test to use :: which the regex actually rejects.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(playwright): update data product domain selector
* fix
* lint
* chore(ui): apply prettier fix to render-field-element
* fix(playwright): use add-domain-form testid for drawer visibility checks
The AddDomainForm root element testid was renamed from add-domain to
add-domain-form during the refactor, but drawer visibility assertions
in DataMarketplace and DataProductAndSubdomains specs still referenced
the old id.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ui): align domain form coverImage type and drawer scroll-to-error
- Type coverImage as CoverImageFileValue | null to match the runtime
value written by FieldTypes.COVER_IMAGE_UPLOAD and consumed by
createEntityWithCoverImage.
- Switch useFormDrawerWithHook's scroll-to-error selector from the
Ant Design class to [aria-invalid="true"] so validation failures
land on the first invalid react-hook-form field.
- Move the Description wrapper's aria-invalid into the FormField render
callback so the rich-text block participates in the same scroll path.
* fix(ui): restore domain cover image validation and stabilize tag test helper
- Reintroduce the size (5 MB) and dimension (800x400) checks that
existed on the legacy MUI cover image upload by adding a react-hook-
form validate rule on the coverImage field.
- Use tagFqn with an exact match in selectTagInTagSuggestion so search
terms containing regex metacharacters cannot match unintended options.
* refactor(ui): extract submitAndClose helper for drawer submit flow
Consolidate the duplicated "await submit, close drawer, refresh list"
pattern into one helper in FormDrawerUtils so the six useFormDrawerWithHook
consumers no longer reference closeDrawer/refresh* from a useCallback that
runs before the drawer hook binds them. Error flows are preserved: if the
submit handler throws, closeDrawer/onSuccess are skipped and the drawer's
existing try/catch keeps the drawer open.
* fix(ui-core): forward onFocus through form-field renderer, drop unsafe cast
- Type FieldPropsMap.onFocus as FocusEventHandler so consumers can pass
event-carrying or plain handlers without coercion.
- Destructure onFocus (and the 'sm'|'md' style size) out of the spread
in renderFieldElement so they land only on Autocomplete, not on
NativeSelect whose native size attribute expects a number.
- Thread onFocus through to Autocomplete so focus-triggered prefetch
in AddDomainForm (tags, domain, owners, experts) actually fires.
- Explicitly type AutocompleteProps.onFocus and drop the cast in
AutocompleteBase that the type mismatch was papering over.
* Update openmetadata-ui/src/main/resources/ui/src/components/Domain/AddSubDomainModal/AddSubDomainModal.component.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: use hookForm.handleSubmit() in useFormDrawerWithHook to preserve RHF submit lifecycle
Agent-Logs-Url: https://github.com/open-metadata/OpenMetadata/sessions/6f7c35d8-9f8f-4da6-8d0e-e8b78e33a732
Co-authored-by: siddhant1 <30566406+siddhant1@users.noreply.github.com>
* fallback
* fix(test): use stable testId to select tag option in TagSuggestion
The new Autocomplete.Item in @openmetadata/ui-core-components computes
the option's accessible name as `label + ' ' + supportingText`, so
`getByRole('option', { name: tagFqn, exact: true })` never matches when
a tag has a displayName. Revert to `getByTestId('tag-option-<fqn>')`,
which is still emitted by TagSuggestion on every Autocomplete.Item and
is unaffected by the supporting-text concatenation.
Fixes three TagSuggestion tests that timed out on all retries in
playwright-ci-postgresql (4, 6):
- DataProducts.spec.ts:449 Create data product with tags using TagSuggestion
- Domains.spec.ts:1196 Create domain with tags using TagSuggestion
- Domains.spec.ts:1243 Create subdomain with tags using TagSuggestion
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Update openmetadata-ui/src/main/resources/ui/src/components/Domain/AddDomainForm/AddDomainForm.component.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix(domain-form): emit data-testid on tag option in AddDomainForm
The refactored AddDomainForm renders the tag input via the generic
Autocomplete renderer (getDefaultAutocompleteItems), which never passes
data-testid to Autocomplete.Item. As a result, the playwright helper
selectTagInTagSuggestion() could not find tag-option-<fqn> elements in
the dropdown, causing three TagSuggestion specs to time out:
- DataProducts.spec.ts:449 Create data product with tags using TagSuggestion
- Domains.spec.ts:1196 Create domain with tags using TagSuggestion
- Domains.spec.ts:1243 Create subdomain with tags using TagSuggestion
Provide a custom renderItem on the tags field that mirrors the default
item rendering and forwards data-testid="tag-option-${item.id}" so the
existing test selector works against the new component path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* tests: fix flaky SystemCertificationTags test where Gold tag missing from dropdown after classification re-enable (#27415)
* tests: fix flaky SystemCertificationTags test where Gold tag missing from dropdown after classification re-enable
* nit
* Fix restrore entity inherited fields specs (#27414)
* increase data contracts test timeout
* fix failure in restroreEntityInheritedFields
* fix restrore entity inheritance specs
* remove unnecessary multiple domains check
---------
Co-authored-by: Shrabanti Paul <shrabantipaul@Shrabantis-MacBook-Pro.local>
* Fix flaky customizedetail page spec (#27420)
* increase data contracts test timeout
* fix flaky customizeDetailPage specs
---------
Co-authored-by: Shrabanti Paul <shrabantipaul@Shrabantis-MacBook-Pro.local>
* fix(test): unflake GlossaryPermissions team-based test (#27422)
* fix(test): assign role to team in P-11 team-based permissions test
The "Team-based permissions work correctly" test created a team and added
testUser to it, but never attached the new role to the team — so the user
inherited no permissions from team membership. The test relied on default
Organization conditional rules, which the frontend treats as no-permission
(only Access.Allow becomes truthy in PermissionsUtils). The glossary page
then rendered the no-permission placeholder and never fired
/api/v1/glossaries?fields=*, hanging visitGlossaryPage's waitForResponse
for the full 30s toPass budget.
Patch the team's defaultRoles after initializePermissions so the user
actually inherits Allow via team membership, matching the test's intent.
Local repro (full file × 10 repeats × 4 workers, retries=0):
- Before: 10/10 P-11 failures
- After: 10/10 P-11 pass, 93/93 overall pass
* fix(test): isolate P-11 to a dedicated user/team
Avoid mutating the file-shared testUser. Previously the fix added testUser
to a temporary team and attached an Allow role; if cleanup failed, the
elevated permissions would leak into subsequent tests sharing the same
worker.
Now P-11 creates its own UserClass + page, runs the team-permission
verification in isolation, and tears down the user, team, role, and policy
at the end. Other tests in the file see no state change from this test
beyond what initializePermissions/cleanupPermissions already do.
Local verification (full file × 10 repeats × 4 workers, retries=0):
93/93 pass, P-11 10/10.
---------
Co-authored-by: Siddhant <siddhant@MacBook-Pro-621.local>
* Fixes #24636: use test_metadata.kwargs['model'] to identify primary table for dbt test entity links (#27366)
* fix: use test_metadata.kwargs['model'] to identify primary table for entity links (issue #24636)
For dbt relationship tests with multiple upstream dependencies, the order of
tables in depends_on.nodes varies by database engine (Snowflake vs Unity Catalog).
The primary table being tested is explicitly specified in test_metadata.kwargs['model']
for generic tests, making this a reliable order-independent way to identify the correct
table for entity link generation. This fixes validation failures when columns exist in
the primary table but not in the referenced table.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Run formatter
* Apply Gitar-bot comments
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Chore(UI): consolidated UI checkstyle fix commands and modify workflow comment (#27402)
* feat: add consolidated UI checkstyle commands for all and changed files
* update prt to pr
* test commit to fail ui-checkstyle
* update the comment
* Revert "test commit to fail ui-checkstyle"
This reverts commit ed056f06292bc9dbbdf0be1fbdcb762a625ac238.
* Revert "update prt to pr"
This reverts commit 0666fa51a36da3ce73ef9fab04480eeb27c0412d.
* Worked on comments
* pull request target remove
* Revert "pull request target remove"
This reverts commit b61e98c16bef6bad50633ba373df977624b900a0.
* Worked on comments
* fix: add missing SecurityConfig init in reIndex and reIndexDI CLI commands (#27424) (#27425)
* fix(migration): revert webhook authType back to secretKey in v1126 and remove broken v1125 migration (#27427)
* fix(migration): add v1126 reverse migration to revert webhook authType back to secretKey
* fix(migration): remove migrateWebhookSecretKeyToAuthType from v1125 migration
* fix(test): remove migrateWebhookSecretKeyToAuthType references from v1125 migration tests
* fix(migration): address copilot review comments on v1126 migration
* fix(migration): case-insensitive bearer check and verify JSON content in v1126 tests
* fix(migration): remove unused constants from v1125 and add postgres path + SQL verification to v1126 tests
* feat(explore): redesign search export scope to export full tab results with accurate counts (#27354)
* feat(explore): redesign search export scope to export full tab results with accurate counts
* fix UI checkstyle and explore support
* fix checkstyle
* fix failing spec & code refactor
* add backend support for handling totalVotes export in explore page
* fix specs
* add skeleton loader
* Fix omjob pod/label naming length constraints (#27143)
* Fix omjob pod/label naming length constraints
- Fix SHA-256 hash byte formatting with & 0xff mask for proper
2-hex-digit encoding per byte
- Enforce Kubernetes 63-character label limit via hash-based truncation
- Extract shared hash utility to HashUtils
- Add comprehensive tests for truncation, uniqueness, and edge cases
Fixes #27004
* Address review: fix hash bounds, add edge case tests, remove redundant substring
- Guard ensureValidLabelValue fallback against StringIndexOutOfBounds
- Add tests for separator-only inputs exercising the fallback path
- Remove redundant .substring() since HashUtils.hash() already returns 6 chars
- Use 253 (K8s DNS subdomain limit) instead of 260 in PodManagerTest
- Fix wrong assertion in LabelBuilderTest (podSelector has 2 entries)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: complete all code review issues for PR merge
- Add HTTP 409 idempotency handling in TableClass.create() for sharded Playwright tests
- Apply Java Spotless formatting to fix checkstyle violations
- Apply Playwright formatting: organize-imports, eslint --fix, prettier --write
- Resolve all 10 code review findings:
✅ StringIndexOutOfBoundsException in hash truncation (already fixed)
✅ Redundant substring operation (already fixed)
✅ Duplicate hash code extraction (already done)
✅ Playwright 409 conflict handling (now added)
✅ Java formatting compliance (now applied)
✅ TypeScript formatting compliance (now applied)
PR is now ready for merge with all CI checks expected to pass.
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test:Added missing test for ontology (#27423)
* test:Added missing test for ontology
* Added other missing part of test
* fix lint issue
* fix(workflows): make Flowable schema upgrades idempotent to survive partial migrations (#27234)
* fix(workflows): make Flowable schema upgrades idempotent to survive partial migrations
Fixes #26048.
When the server crashed mid-startup during a Flowable schema upgrade, the DB
was left in a partially-migrated state. On restart, Flowable re-ran the same
DDL and failed on already-existing objects (indexes, tables, columns), permanently
wedging both the server and migrate --force.
Changes:
1. WorkflowHandler: webserver now uses DB_SCHEMA_UPDATE_FALSE — it validates the
schema but never runs DDL. Only migrate CLI uses DB_SCHEMA_UPDATE_TRUE.
2. OpenMetadataOperations: explicit WorkflowHandler.initialize(config, true) inside
the migrate command so Flowable DDL always runs during migration.
3. WorkflowHandler: catches FlowableWrongDbException on webserver startup and
rethrows with an actionable message directing the operator to run migrate.
4. IdempotentDdlDataSource + IdempotentDdlStatement: JDBC DataSource wrapper used
exclusively in migration context. Intercepts execute(sql) for CREATE INDEX,
CREATE TABLE, and ALTER TABLE ADD COLUMN and pre-checks existence via standard
DatabaseMetaData (getIndexInfo, getTables, getColumns) before executing. If the
object already exists it logs a skip and returns — no SQL state codes, no string
matching, works on MySQL and PostgreSQL.
Unit tests cover schema-update mode selection in both contexts.
* fix(workflows): address review comments on idempotent DDL wrapper
- Extract shouldSkip() helper; apply idempotency checks to all execute()
and executeUpdate() overloads, not just execute(String)
- Tighten ALTER TABLE regex with negative lookahead to exclude SQL keywords
(CONSTRAINT, PRIMARY, UNIQUE, FOREIGN, CHECK, INDEX, KEY) from being
matched as column names
- IdempotentDdlDataSource now wraps a DataSource delegate instead of calling
DriverManager directly; uses migrationDataSource() helper in WorkflowHandler
to resolve from existing DataSource or JDBC params
- Fix InvocationTargetException wrapping in Connection proxy — unwrap cause
so callers receive the original SQLException
- Wrap all createStatement() variants in the proxy, not just the no-arg form
- Contextual error message in WorkflowHandler — distinguish between server
startup and migration context
- Add IdempotentDdlStatementTest: 11 tests covering skip/execute for
CREATE INDEX, CREATE UNIQUE INDEX, CREATE TABLE, ALTER TABLE ADD COLUMN,
keyword-guarded ALTER TABLE, executeUpdate overload, and pass-through
* fix(workflows): include DB/library versions in FlowableWrongDbException message
* test(workflows): add IdempotentDdlDataSourceTest for proxy wrapping and exception surfacing
* test(workflows): assert exception identity in proxy exception-surfacing tests
* fix(workflows): catalog-aware identifier normalization in IdempotentDdlStatement
On MySQL with lower_case_table_names=0 (default on Linux), table names are
stored as-is and catalog=null metadata lookups can miss existing objects.
- Use connection.getCatalog() for all getIndexInfo/getTables/getColumns calls
- Normalize identifiers via DatabaseMetaData.storesLowerCaseIdentifiers() /
storesUpperCaseIdentifiers() instead of unconditional toLowerCase()
- stripIdentifierQuotes() handles backtick, double-quote and bracket quoting
- extractObjectName() handles schema-qualified names (schema.table)
- columnExists now iterates and normalizes COLUMN_NAME from ResultSet
- Test: added MySQL uppercase storage case to IdempotentDdlStatementTest
* fix(workflows): null guard in shouldSkip, drop-create Flowable init, robust test indexing
- shouldSkip() returns false immediately for null SQL, preserving JDBC contract
(delegate handles null and throws the driver's own error)
- drop-create command now calls WorkflowHandler.initialize(config, true) after
native migrations so it produces a fully startable DB including Flowable tables
- WorkflowHandlerSchemaUpdateTest: replace brittle get(1) with getLast() so the
test is not sensitive to how many StandaloneProcessEngineConfiguration instances
are constructed before initializeNewProcessEngine runs
* Move ontology/glossary relation migration from 1.14.0 back to 1.13.0 (#27431)
* Move ontology/glossary relation migration from 1.14.0 back to 1.13.0
Ontology feature will ship in 1.13.0, not 1.14.0. Move the glossary term
relation migrations (relationType backfill, settings insert, stale
relatedTerms strip, conceptMappings backfill) back to the 1.13.0
postDataMigrationSQLScript for both MySQL and PostgreSQL.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Restore empty 1.14.0 SQL migration files for Java migration framework
The V114 MigrationUtil.java package requires the 1.14.0 migration
directory to exist with SQL files for the migration to be picked up.
Keep them as empty files (matching convention of other versions with
no post-data SQL).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add schemaChanges.sql and comment all 1.14.0 SQL migration files
Add both schemaChanges.sql and postDataMigrationSQLScript.sql for
mysql and postgres with a comment explaining the directory is required
for the V114 Java migrations to be picked up by the migration framework.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix missing trailing newline in postgres postDataMigrationSQLScript
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* address feedback
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* feat(migration): add webhook secretKey to authType migration in v1130 (#27438)
* feat(migration): add webhook secretKey to authType migration in v1130
* test(migration): add idempotency test for already-migrated webhook rows in v1130
* Add backticks to table name in mysql median function (#27406)
* Fixes open-metadata#26198: Filter constraints referencing non-existent columns for Redshift AUTO-distribution MVs (#27016)
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
* Chore(deps): Bump org.eclipse.jetty:jetty-http in /openmetadata-service (#27372)
Bumps org.eclipse.jetty:jetty-http from 12.1.6 to 12.1.7.
---
updated-dependencies:
- dependency-name: org.eclipse.jetty:jetty-http
dependency-version: 12.1.7
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* Fixes #21953, #23338, #27380: upgrade collate-sqllineage to >=2.1.1 with regression tests (#27413)
* deps(ingestion): upgrade collate-sqllineage to >=2.1.1 with expanded lineage test coverage
* address copilot comments
* style(ui): apply prettier fix in combobox.tsx
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ui): address Copilot review comments on domain form refactor
- Memoize fetchDataProducts with useCallback and add it to
onDataProductCreateSuccess deps (stale closure fix)
- Correct misleading JSDoc in FormDrawerUtils.submitAndClose
- Pass translated labels to IconPickerField in AddDomainForm
- Narrow Alert helperText guard to typeof string to prevent empty title
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ui-core): guard Alert against empty-string helperText
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style(ui-core): prettier fix on form-field.tsx
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Update openmetadata-ui/src/main/resources/ui/src/components/Domain/AddDomainForm/AddDomainForm.component.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix(ui-core): forward onBlur in Autocomplete fields and fix ReactNode helperText alert
- Pass onBlur handler to Autocomplete in renderFieldElement so RHF marks
fields as touched and b…
jaya6400
pushed a commit
to jaya6400/OpenMetadata
that referenced
this pull request
May 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes:
Fixes
I worked on ... because ...
Type of change:
Checklist:
Fixes <issue-number>: <short explanation>Summary by Gitar
SapSuccessFactorsdatabase service and connection schema to support OData API metadata extraction.sapSuccessFactorsConnection.jsondefiningBasicAuthandOAuth2Credentialsauthentication methods.sapsuccessfactors.jsonto verify API access and metadata discovery.ServiceType.constant.tsandServiceUtilClassBase.tsto includeSapSuccessFactorsin the UI service registry.databaseService.jsonto register the new service type and link its connection schema.This will update automatically on new commits.