ISSUE-25600: Add support for ingesting Oracle Global Temporary Tables

[kratipaliwal]

ISSUE-25600: Add support for ingesting Oracle Global Temporary Tables

Summary

Oracle Global Temporary Tables (GTTs) have permanent definitions in the data dictionary (*_TABLES.DURATION is SYS$SESSION or SYS$TRANSACTION); only their data is session-scoped. Despite being introspectable like any regular table, the Oracle connector filtered them out unconditionally via AND DURATION IS NULL in ORACLE_GET_TABLE_NAMES, so they never appeared in OpenMetadata.

This PR adds an opt-in includeTemporaryTables boolean to OracleConnection. When enabled, GTTs are fetched via a new ORACLE_GET_TEMPORARY_TABLE_NAMES query and surfaced with TableType.Local so users can distinguish them from regular tables in the UI.

This mirrors the spirit of Snowflake's enableTempTableLineage flag — opt-in, defaults to off, preserves existing behavior.

Why GTTs (and not, e.g., Snowflake temp tables)

Oracle GTTs are fundamentally different from Snowflake temporary tables:

• Oracle GTT — schema, columns, constraints, comments are permanent and queryable from ALL_TABLES / ALL_TAB_COLS for any session. Only the row data is session-scoped.
• Snowflake temp table — entirely ephemeral; only exists for the session that created it, so external ingestion can't see it.

So Oracle GTTs can be ingested cleanly using the existing reflection paths — no extra plumbing needed for columns, constraints, or comments.

Changes

Schema (openmetadata-spec/.../oracleConnection.json)

• New includeTemporaryTables: boolean property (default false).

Connector (ingestion/.../oracle/)

• queries.py — new ORACLE_GET_TEMPORARY_TABLE_NAMES constant (same shape as the regular query but DURATION IS NOT NULL). Original ORACLE_GET_TABLE_NAMES left untouched, so default behavior is unchanged.
• utils.py — new get_temporary_table_names(dialect, connection, schema) helper. Kept get_table_names's SQLAlchemy Inspector contract intact.
• metadata.py — query_table_names_and_types now appends GTTs as TableType.Local entries when the flag is set. Read defensively via getattr so it's safe even before model regeneration.

Tests

Unit (ingestion/tests/unit/topology/database/test_oracle.py)

• test_temporary_tables_query_filters_on_duration_not_null — sanity check on the SQL constants.
• test_default_excludes_temporary_tables — verifies no GTT fetch and engine.connect not called when flag off.
• test_include_temporary_tables_yields_local_type — verifies GTTs come back as TableType.Local and the executed query targets DURATION IS NOT NULL.

Integration (ingestion/tests/integration/oracle/) — new module

• Spins up gvenzl/oracle-free:23-slim-faststart testcontainer.
• Seeds one regular table + one session-level GTT + one transaction-level GTT.
• Runs the metadata ingestion workflow twice (flag off / flag on, against distinct service names) and asserts via the OpenMetadata REST API:
  • test_gtts_excluded_by_default — flag off ⇒ regular table present, GTTs absent.
  • test_gtts_ingested_when_flag_enabled — flag on ⇒ regular + both GTTs present.
  • test_gtts_carry_local_table_type — GTTs surface as TableType.Local, regular as TableType.Regular.
  • test_gtt_columns_are_ingested — column reflection works for GTTs.
• Service teardown via metadata.delete(... recursive=True, hard_delete=True) after the package finishes.

Backward compatibility

• Default value is false, so behavior for existing Oracle services is unchanged on upgrade.
• Switching the flag to true only adds entities to the service; it does not modify or remove anything previously ingested.
• Switching the flag back to false later will leave previously-ingested GTT entities in OpenMetadata as orphans (they simply stop being refreshed). Standard re-ingest / soft-delete handling applies.

Reproducing

Sample DDL the integration test uses (paste into sqlplus or any Oracle client):

CREATE TABLE regular_orders (
    order_id NUMBER PRIMARY KEY,
    sku VARCHAR2(40),
    qty NUMBER(10,2)
);

-- Session-level GTT: data persists for the duration of the session
CREATE GLOBAL TEMPORARY TABLE gtt_user_session (
    user_id      NUMBER       NOT NULL,
    action_code  VARCHAR2(50) NOT NULL,
    payload      CLOB
) ON COMMIT PRESERVE ROWS;

-- Transaction-level GTT: data is wiped on every COMMIT
CREATE GLOBAL TEMPORARY TABLE gtt_order_staging (
    order_id NUMBER NOT NULL,
    sku VARCHAR2(40) NOT NULL,
    qty NUMBER(10,2),
    CONSTRAINT pk_gtt_order_staging PRIMARY KEY (order_id, sku)
) ON COMMIT DELETE ROWS;

Verify they're flagged as temporary in the data dictionary:

SELECT table_name, temporary, duration
FROM all_tables
WHERE table_name IN ('REGULAR_ORDERS', 'GTT_USER_SESSION', 'GTT_ORDER_STAGING');

-- Expected:
-- TABLE_NAME           TEMPORARY  DURATION
-- REGULAR_ORDERS       N          (null)
-- GTT_USER_SESSION     Y          SYS$SESSION
-- GTT_ORDER_STAGING    Y          SYS$TRANSACTION

Test plan

• Unit tests pass (pytest ingestion/tests/unit/topology/database/test_oracle.py)
• Integration tests pass (pytest ingestion/tests/integration/oracle/ against a running OpenMetadata instance with Docker available)
• Manual: run ingestion against a real Oracle DB with includeTemporaryTables: true, verify GTTs appear in the UI tagged as Local
• Manual: run ingestion with the flag omitted/false, verify GTTs are NOT ingested (regression check)
• Verify make generate regenerates the Pydantic + Java models cleanly from the updated JSON schema

Closes

Closes #25600

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[gitar-bot]

Code Review ✅ Approved

Integrates Oracle Global Temporary Table ingestion support into the pipeline. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}

[github-actions]

⚠️ TypeScript Types Need Update

The generated TypeScript types are out of sync with the JSON schema changes.

Since this is a pull request from a forked repository, the types cannot be automatically committed.
Please generate and commit the types manually:

cd openmetadata-ui/src/main/resources/ui
./json2ts-generate-all.sh -l true
git add src/generated/
git commit -m "Update generated TypeScript types"
git push

After pushing the changes, this check will pass automatically.

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
87.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

🟡 Playwright Results — all passed (19 flaky)

✅ 3668 passed · ❌ 0 failed · 🟡 19 flaky · ⏭️ 89 skipped

Shard	Passed	Failed	Flaky	Skipped
🟡 Shard 1	477	0	4	4
🟡 Shard 2	651	0	2	7
🟡 Shard 3	654	0	5	1
🟡 Shard 4	632	0	2	27
🟡 Shard 5	610	0	1	42
🟡 Shard 6	644	0	5	8

🟡 19 flaky test(s) (passed on retry)

• Features/CustomizeDetailPage.spec.ts › Database Schema - customization should work (shard 1, 1 retry)
• Flow/Tour.spec.ts › Tour should work from help section (shard 1, 1 retry)
• Flow/Tour.spec.ts › Tour should work from URL directly (shard 1, 1 retry)
• Pages/UserCreationWithPersona.spec.ts › Create user with persona and verify on profile (shard 1, 1 retry)
• Features/BulkEditEntity.spec.ts › Glossary (shard 2, 1 retry)
• Features/ColumnBulkOperations.spec.ts › should search columns with server-side API call (shard 2, 1 retry)
• Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 1 retry)
• Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 1 retry)
• Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 2 retries)
• Features/RTL.spec.ts › Verify Following widget functionality (shard 3, 1 retry)
• Flow/SchemaTable.spec.ts › schema table test (shard 3, 1 retry)
• Pages/Customproperties-part2.spec.ts › entityReferenceList shows item count, scrollable list, no expand toggle (shard 4, 1 retry)
• Pages/Domains.spec.ts › Rename domain with tags and glossary terms preserves associations (shard 4, 1 retry)
• Pages/Glossary.spec.ts › Add and Remove Assets (shard 5, 2 retries)
• Pages/Lineage/LineageFilters.spec.ts › Verify lineage schema filter selection (shard 6, 1 retry)
• Pages/Lineage/LineageRightPanel.spec.ts › Verify custom properties tab IS visible for supported type: searchIndex (shard 6, 1 retry)
• Pages/ODCSImportExport.spec.ts › Multi-object ODCS contract - object selector shows all schema objects (shard 6, 1 retry)
• Pages/UserDetails.spec.ts › Create team with domain and verify visibility of inherited domain in user profile after team removal (shard 6, 1 retry)
• Pages/Users.spec.ts › Permissions for table details page for Data Consumer (shard 6, 1 retry)

📦 Download artifacts

How to debug locally

# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace