Skip to content

[FIX] Propagate all *_cls/*_class plugin metadata fields in PluginManager#2201

Merged
athul-rs merged 3 commits into
mainfrom
fix/plugin-manager-headers-cache-class
Jul 24, 2026
Merged

[FIX] Propagate all *_cls/*_class plugin metadata fields in PluginManager#2201
athul-rs merged 3 commits into
mainfrom
fix/plugin-manager-headers-cache-class

Conversation

@jaseemjaskp

Copy link
Copy Markdown
Contributor

What

  • PluginManager.load_plugins() (in unstract/core) only copied a hardcoded 3-field allowlist (entrypoint_cls, exception_cls, service_class) from a plugin's metadata dict into the registered plugin dict.
  • Generalized this to copy any metadata key ending in _cls or _class, so new plugin fields don't require this shared loader to be updated every time.

Why

  • The verticals_usage plugin (enterprise/cloud-only, feeds API deployment usage into the API Hub subscription dashboard) defines a headers_cache_class metadata field that the loader's hardcoded allowlist didn't know about. It was silently dropped, so every plugin["headers_cache_class"] lookup in backend/plugins/workflow_manager/workflow_v2/api_hub_usage_utils.py raised KeyError — swallowed by broad except Exception blocks.
  • Net effect: API Hub request headers were never cached in Redis, store_usage() (the only path that writes to the verticals.subscription_usage table) was never reached, and the API Hub portal's subscription dashboard stopped showing usage — with no visible errors anywhere.
  • Traced the regression to PR refactor: Add dynamic plugin loading for enterprise components #1736 ("refactor: Add dynamic plugin loading for enterprise components"), which switched api_hub_usage_utils.py from a build-time file-replacement model to a runtime get_plugin() dict lookup, without the shared PluginManager (added in UN-2930 [MISC] Add plugin infrastructure to unstract-core #1618) being updated to support the headers_cache_class field.

How

  • Replaced the three explicit if "x" in metadata: plugin_data["x"] = metadata["x"] checks in unstract/core/src/unstract/core/plugins/plugin_manager.py with a loop that copies any metadata key matching *_cls/*_class.
  • Added unstract/core/tests/test_plugin_manager.py covering: unknown _class/_cls fields (e.g. headers_cache_class) now propagate, existing entrypoint_cls/exception_cls still work, non-class metadata isn't duplicated at the top level, and disabled plugins are still skipped.

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

  • No. Verified via grep across backend/ that no other plugin's metadata relies on a field outside the previous three-field allowlist (only service_class is used elsewhere — notifications, hubspot, subscription_usage, file_converter, payload_modifier, etc.), so existing plugin behavior is unchanged; this purely adds support for previously-dropped fields.

Database Migrations

  • None

Env Config

  • None

Relevant Docs

  • None

Related Issues or PRs

Dependencies Versions

  • None

Notes on Testing

  • Added unstract/core/tests/test_plugin_manager.py (4 new tests, all passing).
  • Ran full unstract/core test suite (uv run pytest unstract/core/tests/) — no new regressions; 2 pre-existing unrelated failures were already broken before this change.
  • Ran ruff check / ruff format --check on changed files — clean.

Screenshots

Checklist

I have read and understood the Contribution Guidelines.

…oded 3

PluginManager.load_plugins() only copied entrypoint_cls, exception_cls,
and service_class from a plugin's metadata into the registered plugin
dict. The verticals_usage plugin (which feeds API deployment usage into
the API Hub subscription dashboard) also defines headers_cache_class,
which was silently dropped, causing every plugin["headers_cache_class"]
lookup in api_hub_usage_utils.py to raise KeyError - swallowed by broad
except blocks - so API hub headers were never cached and usage never
reached the verticals.subscription_usage table.
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary by CodeRabbit

  • Improvements

    • Plugin loading now automatically surfaces additional class-related metadata fields ending with *_cls or *_class.
    • Legacy entrypoint_cls and exception_cls metadata is still supported.
    • Non-class metadata remains available without duplication at the top level.
    • Disabled and inactive plugins are excluded from loading as expected.
  • Tests

    • Expanded test coverage using a temporary, importable fake plugin package to validate promoted class fields, legacy behavior, non-class handling, and skip logic for disabled/inactive plugins.

Walkthrough

PluginManager.load_plugins now promotes all metadata keys ending in _cls or _class into plugin data. Tests validate new and legacy fields, non-class metadata, disabled or inactive plugins, temporary loading, and null-valued class fields.

Changes

Plugin metadata loading

Layer / File(s) Summary
Class-field metadata promotion
unstract/core/src/unstract/core/plugins/plugin_manager.py
PluginManager.load_plugins promotes metadata keys ending in _cls or _class while preserving existing plugin data.
Plugin loading validation
unstract/core/tests/test_plugin_manager.py
Tests cover class-field promotion, legacy fields, non-class metadata, disabled and inactive plugins, temporary plugin loading, and null values.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: chandrasekharan-zipstack, hari-kuriakose

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: propagating all _cls/_class plugin metadata in PluginManager.
Description check ✅ Passed The description mostly follows the template and fills the required sections with relevant implementation, impact, and testing details.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/plugin-manager-headers-cache-class

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR generalizes PluginManager metadata promotion and adds regression coverage.

  • Promotes string metadata keys ending in _cls or _class into the registered plugin dictionary.
  • Preserves existing class-handle fields while supporting fields such as headers_cache_class.
  • Tests propagation, non-class metadata handling, and disabled or inactive plugin behavior.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain.

Important Files Changed

Filename Overview
unstract/core/src/unstract/core/plugins/plugin_manager.py Replaces the fixed class-handle allowlist with suffix-based metadata promotion.
unstract/core/tests/test_plugin_manager.py Adds focused coverage for metadata propagation and plugin activation filtering.

Reviews (3): Last reviewed commit: "Merge branch 'main' into fix/plugin-mana..." | Re-trigger Greptile

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@unstract/core/src/unstract/core/plugins/plugin_manager.py`:
- Around line 218-224: Update the metadata iteration in the plugin loading logic
to validate that each key is a string before applying the "_cls"/"_class" suffix
check. Skip non-string keys while preserving the existing condition that only
adds missing plugin_data fields.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f3a67017-5865-495c-8dc9-587afd840ffe

📥 Commits

Reviewing files that changed from the base of the PR and between e2485a8 and 72862d5.

📒 Files selected for processing (2)
  • unstract/core/src/unstract/core/plugins/plugin_manager.py
  • unstract/core/tests/test_plugin_manager.py

Comment thread unstract/core/src/unstract/core/plugins/plugin_manager.py Outdated

@jaseemjaskp jaseemjaskp left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR review (PR Review Toolkit: code-reviewer, silent-failure-hunter, type-design-analyzer, pr-test-analyzer, comment-analyzer, code-simplifier). The core fix is correct and well-scoped; these are refinements. The non-string-key AttributeError case is already covered by CodeRabbit's comment and is not repeated here.

Comment thread unstract/core/src/unstract/core/plugins/plugin_manager.py
Comment thread unstract/core/src/unstract/core/plugins/plugin_manager.py Outdated
Comment thread unstract/core/src/unstract/core/plugins/plugin_manager.py Outdated
Comment thread unstract/core/tests/test_plugin_manager.py
Comment thread unstract/core/tests/test_plugin_manager.py
…rage

- key.endswith() on a non-string metadata key raised AttributeError,
  uncaught by the surrounding except KeyError, crashing load_plugins()
  for every plugin process-wide (CodeRabbit).
- Hoist the _cls/_class suffix tuple to a module constant and trim the
  comment so it doesn't reference one enterprise plugin's field name.
- Note that the `key not in plugin_data` guard is purely defensive today.
- Add tests locking in that a *_class field propagates by name even when
  its value is None, and that is_active=False also skips loading.
@jaseemjaskp
jaseemjaskp requested a review from athul-rs July 24, 2026 04:10

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
unstract/core/tests/test_plugin_manager.py (1)

42-58: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a regression test for non-string metadata keys.

The loader now skips non-string keys before checking _cls/_class, but this test does not pin that protection. Add an integer-key metadata entry and assert the plugin still loads successfully.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@unstract/core/tests/test_plugin_manager.py` around lines 42 - 58, Add an
integer-key metadata entry to the plugin fixture in
test_known_and_unknown_class_fields_are_both_registered, while retaining the
existing class-field assertions, and assert the plugin loads successfully
without error. Ensure the non-string key is ignored safely during metadata
processing and does not affect the existing plugin name or registered class
fields.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@unstract/core/tests/test_plugin_manager.py`:
- Around line 42-58: Add an integer-key metadata entry to the plugin fixture in
test_known_and_unknown_class_fields_are_both_registered, while retaining the
existing class-field assertions, and assert the plugin loads successfully
without error. Ensure the non-string key is ignored safely during metadata
processing and does not affect the existing plugin name or registered class
fields.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f90c5343-010e-4a47-99b5-9e5bf4a9b427

📥 Commits

Reviewing files that changed from the base of the PR and between 72862d5 and f0ccfc8.

📒 Files selected for processing (2)
  • unstract/core/src/unstract/core/plugins/plugin_manager.py
  • unstract/core/tests/test_plugin_manager.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • unstract/core/src/unstract/core/plugins/plugin_manager.py

@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

Unstract test results

Per-group results

Status Group Tier Passed Failed Errors Skipped Duration (s)
e2e-api-deployment e2e 3 0 0 0 20.7
e2e-coowners e2e 1 0 0 0 1.2
e2e-etl e2e 1 0 0 0 4.0
e2e-login e2e 2 0 0 0 1.2
e2e-prompt-studio e2e 1 0 0 0 4.1
e2e-smoke e2e 2 0 0 0 0.9
e2e-workflow e2e 1 0 0 0 21.5
integration-backend integration 162 0 0 26 40.1
integration-connectors integration 1 0 0 7 7.8
integration-workers integration 0 0 0 141 98.2
unit-backend unit 277 0 0 1 29.9
unit-connectors unit 63 0 0 0 9.9
unit-core unit 33 0 0 0 1.1
unit-platform-service unit 15 0 0 0 2.4
unit-rig unit 86 0 0 0 3.5
unit-sdk1 unit 480 0 0 0 20.5
unit-workers unit 1312 0 0 0 76.2
TOTAL 2440 0 0 175 343.1

Critical paths

⚠️ Critical paths not yet covered

  • workflow-execution-fan-out — Multi-file workflow execution fans out to file-processing workers and rejoins. (declared coverage: no groups declared)
✅ Covered critical paths
  • auth-login — covered by e2e-login
  • adapter-register-llm — covered by integration-backend
  • workflow-author — covered by integration-backend
  • co-owner-manage — covered by integration-backend, e2e-coowners
  • workflow-create-execute — covered by e2e-workflow
  • api-deployment-provision — covered by integration-backend
  • api-deployment-auth — covered by integration-backend
  • api-deployment-run — covered by e2e-api-deployment
  • prompt-studio-author — covered by integration-backend
  • prompt-studio-fetch-response — covered by e2e-prompt-studio
  • connector-register-test — covered by integration-backend
  • pipeline-etl-execute — covered by e2e-etl
  • usage-aggregate-read — covered by integration-backend
  • usage-token-tracking — covered by e2e-api-deployment
  • callback-result-delivery — covered by e2e-api-deployment

@athul-rs
athul-rs merged commit 023b140 into main Jul 24, 2026
11 checks passed
@athul-rs
athul-rs deleted the fix/plugin-manager-headers-cache-class branch July 24, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants