-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Protocol: Allowlisting mechanism for Kedro components
Source: kedro-org/kedro#5652 — "Design a unified allowlisting mechanism for Kedro components"
Allowlisting is opt-in extra surface for users to manage. It should be applied to as few places as possible, and only when there's a real, identified risk.
If you're unsure whether a new component needs this, raise it for discussion rather than implementing unilaterally — this is a security-surface decision, not just an API choice.
Precedence, highest to lowest:
-
Constructor arguments (where the component can be used standalone in code, e.g.
OmegaConfigLoader) -
settings.py— the default home. Rationale: it's where other project-level component config already lives (component classes,CONFIG_LOADER_ARGS, etc.); it's core/trusted (if an attacker can editsettings.pythey're already past the trust boundary); and it's loaded early and available to downstream processes. -
Environment variable — useful as a fallback for packaged/deployed projects where
settings.pycan't be edited post-build. Can be used in conjunction withsettings.py, not as a replacement.
Whatever is concretely provided at a given level takes precedence; unset levels don't implicitly widen or narrow it.
<COMPONENT>_ALLOWLIST (not WHITELIST). Existing RUNNER_MODULES_WHITELIST is to be renamed to RUNNER_MODULE_ALLOWLIST under this convention.
- Move/write validation as a shared utility function in
kedro.utils(private, so it can change without being a public API commitment) rather than bespoke per-component checks — avoids duplicating the same allow/deny logic across the codebase. - Applied as a pre-flight check at the point the component actually loads the external thing (e.g. when importing the runner module; for
OmegaConfigLoader, presumably at initialisation). Different components come alive at different points in the run cycle, so don't try to force one global validation moment — the utility function gets called wherever the loading happens. - The exact shape of what's validated differs per component (module path prefix vs. URL scheme/host vs. full URL prefix, etc.) — define that explicitly per component rather than assuming a single format fits all.
Default must always be "no restriction" — allowlisting is opt-in, so existing local/single-project workflows are unaffected if the user never sets it.
| Component | Status | Mechanism |
|---|---|---|
| Runner module (HTTP server) | Live on main
|
settings.RUNNER_MODULES_WHITELIST (pending rename to RUNNER_MODULE_ALLOWLIST) |
Runner module (CLI kedro run) |
Discussed, not implemented | Would reuse the same settings + utility |
OmegaConfigLoader remote conf_source
|
Proposed, #5636 |
allowed_schemes / allowed_hosts constructor args, falling back to settings.py/env |
| Dataset module loading (custom datasets) | Existing, not user-configurable | Checked against a fixed _DEFAULT_PACKAGES list (["kedro.io.", "kedro_datasets.", ""]) in kedro/io/core.py — no allowlist mechanism needed today |
Logging config source (KEDRO_LOGGING_CONFIG) |
Identified as a candidate | Not yet designed |
| Logging class restriction (#5630) | Identified as a candidate | Currently restricted to logging.Handler/Formatter/Filter by type check, not an allowlist |
| Starters (custom starter templates) | Identified as a candidate | Not yet designed |
| Catalog / dataset entries | Explicitly out of scope | Covered by config-source restriction instead; treated as platform/IAM concern where data access is involved |
- Confirm the risk is real and config/code-loading-shaped (not a data-access concern — that's out of scope).
- Define exactly what gets validated and at what granularity (module path prefix? URL scheme? host? full prefix?).
- Name the setting
<COMPONENT>_ALLOWLIST. - Add support in this precedence order, only where it makes sense for that component: constructor arg →
settings.py→ env var fallback. - Implement/reuse a shared validation utility in
kedro.utilsrather than writing bespoke inline checks. - Call the validation as a pre-flight check at the actual load site.
- Confirm default behaviour is unrestricted (no breaking change for existing projects).
- Document the new setting alongside the other
settings.pyentries.
- Contribute to Kedro
- Guidelines for contributing developers
- Contribute changes to Kedro that are tested on Databricks
- Backwards compatibility and breaking changes
- Contribute to the Kedro documentation
- Kedro documentation style guide
- Creating developer documentation
- Kedro new project creation - how it works
- The CI Setup: GitHub Actions
- The Performance Test Setup: Airspeed Velocity
- Protocol: Allowlisting mechanism
- Insights and opportunities related to helping Kedro impact more users
- Research summary of insights for improving Kedro's value
- Synthesis of research related to deployment of Kedro to modern MLOps platforms
- Market research on versioning tools
- Versioning research
- Research summary of insights for redesigning Kedro's data catalog API
- Research on Deployment