Skip to content

config: allow filesystem plugins to declare their own remote config schema via entry points #10993

@adamlabadorf

Description

@adamlabadorf

Problem

Third-party DVC filesystem plugins (installed via the dvc.fs entry point group) cannot be used as DVC remotes without modifying DVC core. The config validator in dvc/config_schema.py maintains a hardcoded list of URL schemes in REMOTE_SCHEMAS, so any plugin with a custom scheme gets rejected immediately:

dvc.config.ConfigError: config file error: Unsupported URL type osf:// for dictionary value @ data['remote']['myremote']

This was previously raised in #9711, which was closed due to lack of demand. I'm reopening as a concrete proposal with a working implementation.

Proposed Solution

Allow plugins to declare a REMOTE_CONFIG class attribute (a dict of config keys → validators) on their filesystem class. At import time, _discover_plugin_schemas() iterates over installed dvc.fs entry points and merges any declared schemas into REMOTE_SCHEMAS.

Plugin side (no DVC core changes needed by the plugin author beyond declaring the attribute):

class OSFFileSystem(ObjectFileSystem):
    protocol = "osf"
    REMOTE_CONFIG: ClassVar[dict] = {
        "token": str,
        "project_id": str,
        "provider": str,
        "endpoint_url": str,
    }

DVC core change (~25 lines in dvc/config_schema.py):

from importlib.metadata import entry_points

def _discover_plugin_schemas():
    for ep in entry_points(group="dvc.fs"):
        try:
            cls = ep.load()
        except Exception:  # noqa: BLE001,S112
            continue
        remote_config = getattr(cls, "REMOTE_CONFIG", None)
        if not remote_config:
            continue
        protocol = getattr(cls, "protocol", ep.name)
        schemes = (protocol,) if isinstance(protocol, str) else protocol
        for scheme in schemes:
            if scheme not in REMOTE_SCHEMAS:
                REMOTE_SCHEMAS[scheme] = {**remote_config, **REMOTE_COMMON}

_discover_plugin_schemas()

Properties

  • Backward compatible — existing hardcoded schemes are never overwritten
  • Zero overhead for users without third-party plugins (entry point group is empty)
  • Graceful degradation — plugins that fail to load are silently skipped
  • Minimal change — ~25 lines added to config_schema.py, no other files touched
  • Tested — 7 new unit tests covering schema registration, skipping, multi-protocol, load failures, and ByUrl validation

Working Implementation

I have a working implementation in adamlabadorf/dvc@feature/plugin-schema-discovery, developed alongside dvc-osf, a plugin for Open Science Framework storage.

The dvc-osf integration tests (30 passing) confirm that with this change, dvc remote add, dvc push, and dvc pull all work correctly with a third-party plugin.

Happy to submit a PR — would appreciate any guidance on approach or concerns before doing so.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions