Skip to content

Move cloud compatibility gating into a shared helper beside the object store configuration code #5658

Description

@dwsmith1983

What is the problem the feature request solves?

Follow-up from the native Delta Lake scan review (#5365). The Delta contrib carries its own cloud compatibility gating (S3 option divergence checks) next to Delta admission logic. Since native Parquet and native Delta share the same underlying object store client, that policy could live in a focused helper beside the shared object store configuration code so compatibility fixes apply consistently, and Delta admission can concentrate on protocol features, schemas, and Delta metadata.

The helper must preserve Hadoop's distinct rules for plain options, password-backed values, variable substitution, and bucket overrides, without introducing another provider abstraction.

Describe the potential solution

No response

Additional context

Raised in the #5365 review discussion.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions