Skip to content

fix(db_engine_specs): skip malformed third-party dialect entry points - #43110

Merged
rusackas merged 7 commits into
apache:masterfrom
aminghadersohi:aminghadersohi/guard-third-party-dialect-enumeration
Aug 17, 2026
Merged

fix(db_engine_specs): skip malformed third-party dialect entry points#43110
rusackas merged 7 commits into
apache:masterfrom
aminghadersohi:aminghadersohi/guard-third-party-dialect-enumeration

Conversation

@aminghadersohi

Copy link
Copy Markdown
Contributor

SUMMARY

get_available_engine_specs() enumerates every installed sqlalchemy.dialects entry point and reads dialect.name unguarded:

for ep in entry_points(group="sqlalchemy.dialects"):
    try:
        dialect = ep.load()
    except Exception as ex:
        logger.debug(...)
    else:
        backend = dialect.name          # <-- unguarded
        driver = getattr(dialect, "driver", dialect.name)

If a third-party entry point loads successfully but does not resolve to a usable dialect — e.g. a malformed name = package:module entry point that yields a module (which has no .name) — the AttributeError aborts the whole enumeration. Because get_available_engine_specs() runs inside common_bootstrap_payload(), this 500s every page that builds the bootstrap (e.g. /welcome/), not just that one connector — the whole app becomes unusable.

Real-world trigger: sqlalchemy-exasol 7.1.1+ registers exa = sqlalchemy_exasol:base, which ep.load() resolves to the base module; module.nameAttributeError → every page 500s under SQLAlchemy 2.0.

The native-dialect loop directly above is already defensive (issubclass(dialect, DefaultDialect), hasattr(dialect, "driver"), per-dialect try/except). This makes the third-party loop equally defensive: if the loaded object has no usable .name, skip it with a logger.warning and continue. A malformed driver then degrades to "that one connector is unavailable" instead of "the app is down".

TESTING INSTRUCTIONS

pytest tests/unit_tests/db_engine_specs/test_init.py

Adds test_get_available_engine_specs_skips_malformed_dialect_entry_point: registers a bogus sqlalchemy.dialects entry point whose load() returns a module (no .name) and asserts get_available_engine_specs() skips it (with a warning) instead of raising. The test raises AttributeError on master and passes with this change.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
  • Introduces new feature or API
  • Removes existing feature or API

@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 40.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.65%. Comparing base (1991e3f) to head (93a818e).

Files with missing lines Patch % Lines
superset/db_engine_specs/__init__.py 40.00% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #43110      +/-   ##
==========================================
- Coverage   66.65%   66.65%   -0.01%     
==========================================
  Files        2874     2874              
  Lines      163784   163784              
  Branches    37798    37799       +1     
==========================================
- Hits       109177   109165      -12     
- Misses      52469    52479      +10     
- Partials     2138     2140       +2     
Flag Coverage Δ
hive 38.13% <40.00%> (-0.01%) ⬇️
mysql 57.83% <40.00%> (-0.01%) ⬇️
postgres 57.86% <40.00%> (-0.01%) ⬇️
presto 40.08% <40.00%> (-0.01%) ⬇️
python 59.25% <40.00%> (-0.01%) ⬇️
sqlite 57.50% <40.00%> (-0.01%) ⬇️
unit 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aminghadersohi
aminghadersohi marked this pull request as ready for review August 13, 2026 13:30
@dosubot dosubot Bot added the data:connect Namespace | Anything related to db connections / integrations label Aug 13, 2026
@bito-code-review

bito-code-review Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #8c27e3

Actionable Suggestions - 0
Review Details
  • Files reviewed - 2 · Commit Range: 3cc7661..3cc7661
    • superset/db_engine_specs/__init__.py
    • tests/unit_tests/db_engine_specs/test_init.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@bito-code-review

bito-code-review Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #2f1433

Actionable Suggestions - 0
Review Details
  • Files reviewed - 2 · Commit Range: 3cc7661..3cc7661
    • superset/db_engine_specs/__init__.py
    • tests/unit_tests/db_engine_specs/test_init.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

get_available_engine_specs() reads `dialect.name` unguarded for every
installed `sqlalchemy.dialects` entry point. A third-party entry point that
loads successfully but does not resolve to a usable dialect -- e.g. a malformed
`name = package:module` entry point that yields a module (which has no `.name`)
-- raises AttributeError and aborts the whole enumeration. Because this runs in
common_bootstrap_payload(), it 500s every page that builds the bootstrap (e.g.
/welcome/), not just that one connector.

Mirror the defensiveness of the native-dialect loop directly above: if the
loaded object has no usable `.name`, skip it with a warning and continue, so a
malformed driver degrades to "that connector is unavailable" instead of taking
down the app.

Signed-off-by: Amin Ghadersohi <amin.ghadersohi@gmail.com>
@aminghadersohi
aminghadersohi force-pushed the aminghadersohi/guard-third-party-dialect-enumeration branch from 3cc7661 to 9b2bb71 Compare August 14, 2026 07:15
Comment thread superset/db_engine_specs/__init__.py
@bito-code-review

Copy link
Copy Markdown
Contributor

The flagged issue is correct. The current implementation only checks for the existence of a name attribute, which is insufficient to guarantee that the loaded object is a valid SQLAlchemy dialect. To resolve this, you should verify that the loaded object is an instance of the expected SQLAlchemy dialect base class or implements the required dialect contract (e.g., having a dbapi() method or driver attribute) before registering it.

Here is a concise implementation to add this validation:

from sqlalchemy.engine import Dialect

# ... inside the loop ...
            backend = getattr(dialect, "name", None)
            if not isinstance(backend, (str, bytes)) or not isinstance(dialect, Dialect):
                logger.warning(
                    "Skipping SQLAlchemy dialect entry point %r: %r did not "
                    "resolve to a valid SQLAlchemy dialect",
                    ep.name,
                    ep.value,
                )
                continue

There are no other comments on this PR to address.

superset/db_engine_specs/init.py

backend = getattr(dialect, "name", None)
            if not isinstance(backend, (str, bytes)) or not isinstance(dialect, Dialect):
                logger.warning(
                    "Skipping SQLAlchemy dialect entry point %r: %r did not "
                    "resolve to a valid SQLAlchemy dialect",
                    ep.name,
                    ep.value,
                )
                continue

@bito-code-review

bito-code-review Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #41abd4

Actionable Suggestions - 0
Review Details
  • Files reviewed - 2 · Commit Range: 9b2bb71..9b2bb71
    • superset/db_engine_specs/__init__.py
    • tests/unit_tests/db_engine_specs/test_init.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@bito-code-review

bito-code-review Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #922717

Actionable Suggestions - 0
Additional Suggestions - 2
  • superset/db_engine_specs/__init__.py - 2
    • Missing test coverage for dbapi() exceptions · Line 188-195
      The `dialect.dbapi()` call (lines 190-195) adds exception handling for missing driver modules, but no test case covers this path. While not a blocker, consider adding a test that simulates `ModuleNotFoundError` on `dialect.dbapi()` to prevent regression.
    • Redundant cast at runtime · Line 188-188
      Line 188 adds `dialect = cast(type[DefaultDialect], dialect)`, but `cast` is a no-op at runtime — it only affects static type checkers. This has no functional impact but may confuse maintainers into thinking it performs a runtime check.
Review Details
  • Files reviewed - 2 · Commit Range: 9b2bb71..4d49315
    • superset/db_engine_specs/__init__.py
    • tests/unit_tests/db_engine_specs/test_init.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@rusackas

Copy link
Copy Markdown
Member

Heya @aminghadersohi! Good catch on the crash, and the guard makes sense.

One thing before I approve, though. dialect.dbapi() doesn't exist under SQLAlchemy 2.0 anymore (it's import_dbapi() now), so it throws AttributeError on any 2.0-native dialect and the broad except skips it. Our own postgres.psycopg2 entry point from setup.py is one of those, so postgresql ends up with no detected driver and drops out of the available databases list entirely. I nearly missed it.

I think we can drop the dbapi() call (the type/subclass/name guards already stop the crash), or use import_dbapi() with a fallback for older dialects. Either way, mind adding a case that asserts a valid dialect still comes through, so we'd catch a regression like this? Happy to lend a hand.

@pull-request-size pull-request-size Bot added size/L and removed size/M labels Aug 17, 2026
@aminghadersohi

Copy link
Copy Markdown
Contributor Author

Thanks for catching this — addressed in 93a818e1df. I dropped the third-party dbapi() probe, while retaining the type/subclass/name/driver guards. I also added a regression test using a valid SQLAlchemy 2.0-style DefaultDialect subclass (with import_dbapi() inherited and no dbapi()) and assert that its driver is preserved in the available engine specs.

@rusackas
rusackas merged commit aaf9eba into apache:master Aug 17, 2026
80 checks passed
@rusackas
rusackas deleted the aminghadersohi/guard-third-party-dialect-enumeration branch August 17, 2026 21:13
@bito-code-review

Copy link
Copy Markdown
Contributor

Bito Automatic Review Skipped – PR Already Merged

Bito scheduled an automatic review for this pull request, but the review was skipped because this PR was merged before the review could be run.
No action is needed if you didn't intend to review it. To get a review, you can type /review in a comment and save it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data:connect Namespace | Anything related to db connections / integrations size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants