diff --git a/site/content/in-dev/unreleased/managing-security/_index.md b/site/content/in-dev/unreleased/managing-security/_index.md index ec680572c1..5d59ea1384 100644 --- a/site/content/in-dev/unreleased/managing-security/_index.md +++ b/site/content/in-dev/unreleased/managing-security/_index.md @@ -22,7 +22,6 @@ linkTitle: Managing Security type: docs weight: 550 --- - -## [Access Control]({{< relref "access-control" >}}) - -## [Authentication and Identity Providers]({{< relref "external-idp" >}}) \ No newline at end of file +* [Access Control]({{< relref "access-control" >}}) +* [Authentication and Identity Providers]({{< relref "external-idp" >}}) +* [External Policy Decision Point]({{< relref "external-pdp" >}}) (PDP), including Open-Policy-Agent (OPA) diff --git a/site/content/in-dev/unreleased/managing-security/access-control.md b/site/content/in-dev/unreleased/managing-security/access-control.md index b8a1b697ca..b2243bf09d 100644 --- a/site/content/in-dev/unreleased/managing-security/access-control.md +++ b/site/content/in-dev/unreleased/managing-security/access-control.md @@ -28,6 +28,10 @@ This section provides information about how access control works for Apache Pola Polaris uses a role-based access control (RBAC) model in which the Polaris administrator assigns access privileges to catalog roles and then grants access to resources to principals by assigning catalog roles to principal roles. +{{% alert title="Note" color="primary" %}} +For advanced authorization scenarios requiring external policy management or integration with existing policy infrastructure, Polaris supports integration with external Policy Decision Points (PDPs) such as Open Policy Agent (OPA). See [External Policy Decision Point](../external-pdp/) for more information. +{{% /alert %}} + These are the key concepts to understanding access control in Polaris: - **Securable object** diff --git a/site/content/in-dev/unreleased/managing-security/external-pdp/_index.md b/site/content/in-dev/unreleased/managing-security/external-pdp/_index.md new file mode 100644 index 0000000000..c4e2c41584 --- /dev/null +++ b/site/content/in-dev/unreleased/managing-security/external-pdp/_index.md @@ -0,0 +1,76 @@ +--- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +title: External Policy Decision Point +linkTitle: External PDP +type: docs +weight: 300 +--- + +This section provides information about integrating Apache Polaris (Incubating) with external Policy Decision Points (PDPs) for authorization. + +## Overview + +By default, Apache Polaris uses an internal authorization system based on role-based access control (RBAC). For advanced use cases requiring external policy management, centralized authorization, or integration with existing policy infrastructure, Polaris supports integration with external Policy Decision Points (PDPs). + +## What is a Policy Decision Point? + +A Policy Decision Point (PDP) is a component that evaluates authorization requests against defined policies and returns authorization decisions (allow/deny). + +Organizations may choose to use an external PDP instead of Polaris's internal authorization in order to leverage a centralized policy store that manages authorization policies across multiple services and applications. + +## Architecture + +When using an external PDP, Polaris delegates authorization decisions as follows: + +1. **Client request**: A client makes a request to Polaris (e.g., read a table) +2. **Authorization check**: Polaris sends an authorization request to the external PDP +3. **Policy evaluation**: The PDP evaluates the request against configured policies +4. **Decision**: The PDP returns an allow/deny decision +5. **Enforcement**: Polaris enforces the decision and proceeds or rejects the request + +``` +┌─────────┐ ┌─────────────┐ ┌──────────────┐ +│ Client │─────────>│ Polaris │────────>│ External PDP │ +│ │ Request │ │ AuthZ │ │ +│ │ │ │ Request │ │ +│ │<─────────│ │<────────│ │ +│ │ Response │ │ Decision│ │ +└─────────┘ └─────────────┘ └──────────────┘ +``` + +## Available Implementations + +Apache Polaris currently supports the following external PDP integrations: + +- **[Open Policy Agent (OPA)]({{< relref "opa.md" >}})**: A general-purpose policy engine with a rich ecosystem and flexible policy language (Rego) + +## Configuration + +To enable external PDP integration, set the following configuration property: + +```properties +polaris.authorization.type= +``` + +Where `` is the identifier for the PDP implementation (e.g., `opa`). The default value is `internal`. + +See the specific PDP documentation for detailed configuration options: + +- [OPA Configuration]({{< relref "opa.md#configuration-reference" >}}) diff --git a/site/content/in-dev/unreleased/managing-security/external-pdp/opa.md b/site/content/in-dev/unreleased/managing-security/external-pdp/opa.md new file mode 100644 index 0000000000..6e2ffe673c --- /dev/null +++ b/site/content/in-dev/unreleased/managing-security/external-pdp/opa.md @@ -0,0 +1,462 @@ +--- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +title: Open Policy Agent (OPA) Integration +linkTitle: OPA +type: docs +weight: 100 +--- + +{{< alert warning "Preview Feature" >}} +**OPA integration is currently an preview feature** and may undergo breaking changes in future versions. Use with caution in production environments. +{{< /alert >}} + +This page describes how to integrate Apache Polaris (Incubating) with [Open Policy Agent (OPA)](https://www.openpolicyagent.org/) for external authorization. + +## Overview + +Open Policy Agent (OPA) is a general-purpose policy engine that enables unified, context-aware policy enforcement across your stack. OPA provides a high-level declarative language (Rego) for authoring policies and APIs to offload policy decision-making from your software. + +Key benefits of using OPA with Polaris: + +- **Flexible policy language**: Write authorization logic in Rego, a powerful declarative language +- **Centralized policy management**: Manage all policies in a single location +- **Policy testing**: Write unit tests for your authorization policies +- **Rich ecosystem**: Integrate with policy bundles, decision logs, and management tools +- **Attribute-based access control**: Make decisions based on user attributes, resource properties, and environmental context + +## Prerequisites + +Before configuring OPA integration: + +1. **OPA Server**: Deploy and configure an OPA server accessible from Polaris +2. **Policy Definition**: Write and deploy authorization policies to OPA +3. **Network Access**: Ensure Polaris can reach the OPA server + +## Quick Start + +### 1. Deploy OPA + +Deploy OPA server with your policies. For example, using Docker: + +```bash +docker run -d \ + --name opa \ + -p 8181:8181 \ + -v $(pwd)/policies:/policies \ + openpolicyagent/opa:latest \ + run --server --addr :8181 /policies +``` + +### 2. Create a Policy + +Create a policy file (e.g., `policies/polaris.rego`): + +```rego +package polaris.authz + +import future.keywords.if + +# Default deny +default allow := false + +# Allow admins to do everything +allow if { + "ADMIN" in input.actor.roles +} + +# Allow read operations on tables in analytics catalogs +allow if { + input.action == "LOAD_TABLE_WITH_READ_DELEGATION" + some target in input.resource.targets + some parent in target.parents + parent.type == "CATALOG" + startswith(parent.name, "analytics_") +} +``` + +### 3. Configure Polaris + +Add the following to your Polaris configuration: + +```properties +# Enable OPA authorization +polaris.authorization.type=opa + +# OPA server endpoint +polaris.authorization.opa.policy-uri=http://localhost:8181/v1/data/polaris/authz/allow +``` + +### 4. Restart Polaris + +Restart the Polaris service to apply the configuration. + +## Configuration Reference + +### Basic Configuration + +| Property | Required | Default | Description | +|----------|----------|---------|-------------| +| `polaris.authorization.type` | Yes | `internal` | Set to `opa` to enable OPA authorization | +| `polaris.authorization.opa.policy-uri` | Yes | - | Full URI to the OPA policy decision endpoint (must be http or https) | + +### HTTP Configuration + +| Property | Required | Default | Description | +|----------|----------|---------|-------------| +| `polaris.authorization.opa.http.timeout` | No | `PT2S` | HTTP request timeout (ISO-8601 duration format, e.g., `PT2S`, `PT10S`) | +| `polaris.authorization.opa.http.verify-ssl` | No | `true` | Whether to verify SSL certificates | +| `polaris.authorization.opa.http.trust-store-path` | No | - | Path to the trust store containing CA certificates | +| `polaris.authorization.opa.http.trust-store-password` | No | - | Password for the trust store | + +### Authentication Configuration + +OPA integration supports two authentication modes: + +#### No Authentication (default) + +No authentication configuration needed if OPA server doesn't require authentication: + +```properties +polaris.authorization.opa.auth.type=none +``` + +#### Bearer Token Authentication + +**Static Bearer Token:** + +Use a static bearer token: + +```properties +polaris.authorization.opa.auth.type=bearer +polaris.authorization.opa.auth.bearer.static-token.value=your-secret-token +``` + +| Property | Required | Default | Description | +|----------|----------|---------|-------------| +| `polaris.authorization.opa.auth.type` | No | `none` | Set to `bearer` to enable bearer token authentication | +| `polaris.authorization.opa.auth.bearer.static-token.value` | Yes* | - | The bearer token value (*when using static token) | + +**File-Based Bearer Token with Auto-Refresh:** + +Use a bearer token from a file with automatic refresh (ideal for JWT tokens): + +```properties +polaris.authorization.opa.auth.type=bearer +polaris.authorization.opa.auth.bearer.file-based.path=/var/secrets/token.txt +polaris.authorization.opa.auth.bearer.file-based.refresh-interval=PT5M +polaris.authorization.opa.auth.bearer.file-based.jwt-expiration-refresh=true +polaris.authorization.opa.auth.bearer.file-based.jwt-expiration-buffer=PT1M +``` + +| Property | Required | Default | Description | +|----------|----------|---------|-------------| +| `polaris.authorization.opa.auth.type` | No | `none` | Set to `bearer` to enable bearer token authentication | +| `polaris.authorization.opa.auth.bearer.file-based.path` | Yes* | - | Path to file containing the bearer token (*when using file-based token) | +| `polaris.authorization.opa.auth.bearer.file-based.refresh-interval` | No | - | How often to refresh the token (ISO-8601 duration, e.g., `PT5M` for 5 minutes). If not set and JWT refresh is disabled, token won't be refreshed | +| `polaris.authorization.opa.auth.bearer.file-based.jwt-expiration-refresh` | No | `true` | Automatically detect JWT tokens and refresh based on expiration time | +| `polaris.authorization.opa.auth.bearer.file-based.jwt-expiration-buffer` | No | `PT1M` | Buffer time before JWT expiration to trigger refresh (ISO-8601 duration) | + +**JWT Auto-Refresh**: When `jwt-expiration-refresh` is enabled (default), if the token file contains a valid JWT with an `exp` claim, Polaris will automatically refresh the token shortly before it expires based on the `jwt-expiration-buffer` setting. + +## Policy Development + +### Input Document Structure + +Polaris sends the following input structure to OPA: + +```json +{ + "actor": { + "principal": "user@example.com", + "roles": ["role1", "role2"] + }, + "action": "LOAD_TABLE_WITH_READ_DELEGATION", + "resource": { + "targets": [ + { + "type": "TABLE", + "name": "my_table", + "parents": [ + { + "type": "CATALOG", + "name": "my_catalog" + }, + { + "type": "NAMESPACE", + "name": "schema1" + } + ] + } + ], + "secondaries": [] + }, + "context": { + "request_id": "uuid" + } +} +``` + +#### Actor Object + +| Field | Type | Description | +|-------|------|-------------| +| `principal` | string | The principal identifier (e.g., username, service account) | +| `roles` | array | Array of role names assigned to the principal | + +#### Action Field + +The `action` field contains the operation being attempted as a string value from the `PolarisAuthorizableOperation` enum. + +For the complete list of available operations, see the [PolarisAuthorizableOperation enum](https://github.com/apache/polaris/blob/main/polaris-core/src/main/java/org/apache/polaris/core/auth/PolarisAuthorizableOperation.java) in the source code. + +Common examples include: +- Table operations: `LOAD_TABLE_WITH_READ_DELEGATION`, `LOAD_TABLE_WITH_WRITE_DELEGATION`, `CREATE_TABLE_DIRECT`, `UPDATE_TABLE`, `DROP_TABLE_WITHOUT_PURGE` +- Catalog operations: `CREATE_CATALOG`, `UPDATE_CATALOG`, `DELETE_CATALOG` +- Namespace operations: `CREATE_NAMESPACE`, `UPDATE_NAMESPACE_PROPERTIES`, `DROP_NAMESPACE` + +{{< alert warning "Important Policy Considerations" >}} +**Handle All Data Operations Explicitly**: When writing OPA policies, explicitly handle all `PolarisAuthorizableOperation` values that enable catalog, namespace and table operations. Operations not covered by your policy rules will fall through to your default decision. We recommend: +- Setting `default allow := false` to deny by default +- Explicitly allowing only operations your users need + +**Internal-Only Operations**: Some operations like `ADD_PRINCIPAL_GRANT_TO_PRINCIPAL_ROLE` and `CREATE_POLICY` manage Polaris's internal privilege system. **These should always be denied in OPA policies** since privilege management should be handled through Polaris's native authorization system, not external policies. +{{< /alert >}} + +#### Resource Object + +| Field | Type | Description | +|-------|------|-------------| +| `targets` | array | Array of target resource objects (primary resources being accessed) | +| `secondaries` | array | Array of secondary resource objects (related resources, if any) | + +Each resource object in `targets` or `secondaries` has: + +| Field | Type | Description | +|-------|------|-------------| +| `type` | string | Resource type (CATALOG, NAMESPACE, TABLE, VIEW, PRINCIPAL, CATALOG_ROLE, etc.) | +| `name` | string | Resource name | +| `parents` | array | Array of parent resource objects in the hierarchy (e.g., catalog and namespace for a table) | + +Each parent object has: + +| Field | Type | Description | +|-------|------|-------------| +| `type` | string | Parent resource type | +| `name` | string | Parent resource name | + +#### Context Object + +| Field | Type | Description | +|-------|------|-------------| +| `request_id` | string | UUID for correlating requests with logs | + +### Policy Example + +```rego +package polaris.authz + +import future.keywords.if +import future.keywords.in + +default allow := false + +# Admin role can perform all catalog, namespace, and table operations +allow if { + "ADMIN" in input.actor.roles + input.action in [ + # Catalog operations + "CREATE_CATALOG", + "UPDATE_CATALOG", + "DELETE_CATALOG", + "LIST_CATALOGS", + + # Namespace operations + "CREATE_NAMESPACE", + "UPDATE_NAMESPACE_PROPERTIES", + "DROP_NAMESPACE", + "LIST_NAMESPACES", + + # Table operations + "CREATE_TABLE_DIRECT", + "LOAD_TABLE_WITH_READ_DELEGATION", + "LOAD_TABLE_WITH_WRITE_DELEGATION", + "UPDATE_TABLE", + "DROP_TABLE_WITHOUT_PURGE", + # Add other data operations as needed + ] +} + +# Data engineers can create/read/update tables and namespaces +allow if { + "DATA_ENGINEER" in input.actor.roles + input.action in [ + # Table operations + "CREATE_TABLE_DIRECT", + "LOAD_TABLE_WITH_READ_DELEGATION", + "LOAD_TABLE_WITH_WRITE_DELEGATION", + "UPDATE_TABLE", + + # Namespace operations + "CREATE_NAMESPACE", + "UPDATE_NAMESPACE_PROPERTIES", + "LIST_NAMESPACES" + ] +} + +# Analysts can only read tables +allow if { + "ANALYST" in input.actor.roles + input.action == "LOAD_TABLE_WITH_READ_DELEGATION" + some target in input.resource.targets + target.type == "TABLE" +} +``` + +{{< alert info "Best Practice" >}} +**Explicit Allow Lists for Data Operations Only**: The example above uses an explicit allow-list approach where only catalog, namespace, and table operations are permitted. Policy and grant operations (like `ADD_PRINCIPAL_GRANT_TO_PRINCIPAL_ROLE`, `CREATE_POLICY`) are automatically denied by the `default allow := false` since they're not in any allow rule. This ensures privilege management remains within Polaris's native authorization system, and new operations added in future Polaris versions will be denied by default until you explicitly allow them. +{{< /alert >}} + +### Testing Policies + +OPA supports policy testing using `opa test`. Create a test file (e.g., `polaris_test.rego`): + +```rego +package polaris.authz + +import future.keywords.if + +test_admin_can_create_catalog if { + allow with input as { + "actor": {"principal": "admin", "roles": ["ADMIN"]}, + "action": "CREATE_CATALOG", + "resource": { + "targets": [{ + "type": "CATALOG", + "name": "new_catalog", + "parents": [] + }], + "secondaries": [] + }, + "context": {"request_id": "test"} + } +} + +test_admin_cannot_grant_privileges if { + not allow with input as { + "actor": {"principal": "admin", "roles": ["ADMIN"]}, + "action": "ADD_PRINCIPAL_GRANT_TO_PRINCIPAL_ROLE", + "resource": { + "targets": [{ + "type": "PRINCIPAL_ROLE", + "name": "some_role", + "parents": [] + }], + "secondaries": [] + }, + "context": {"request_id": "test"} + } +} + +test_data_engineer_can_create_table if { + allow with input as { + "actor": {"principal": "engineer", "roles": ["DATA_ENGINEER"]}, + "action": "CREATE_TABLE_DIRECT", + "resource": { + "targets": [{ + "type": "TABLE", + "name": "new_table", + "parents": [ + {"type": "CATALOG", "name": "prod"}, + {"type": "NAMESPACE", "name": "schema1"} + ] + }], + "secondaries": [] + }, + "context": {"request_id": "test"} + } +} + +test_data_engineer_cannot_delete_catalog if { + not allow with input as { + "actor": {"principal": "engineer", "roles": ["DATA_ENGINEER"]}, + "action": "DELETE_CATALOG", + "resource": { + "targets": [{ + "type": "CATALOG", + "name": "prod", + "parents": [] + }], + "secondaries": [] + }, + "context": {"request_id": "test"} + } +} + +test_analyst_can_read_table if { + allow with input as { + "actor": {"principal": "analyst", "roles": ["ANALYST"]}, + "action": "LOAD_TABLE_WITH_READ_DELEGATION", + "resource": { + "targets": [{ + "type": "TABLE", + "name": "data_table", + "parents": [ + {"type": "CATALOG", "name": "prod"}, + {"type": "NAMESPACE", "name": "schema1"} + ] + }], + "secondaries": [] + }, + "context": {"request_id": "test"} + } +} + +test_analyst_cannot_update_table if { + not allow with input as { + "actor": {"principal": "analyst", "roles": ["ANALYST"]}, + "action": "UPDATE_TABLE", + "resource": { + "targets": [{ + "type": "TABLE", + "name": "data_table", + "parents": [ + {"type": "CATALOG", "name": "prod"}, + {"type": "NAMESPACE", "name": "schema1"} + ] + }], + "secondaries": [] + }, + "context": {"request_id": "test"} + } +} +``` + +Run tests: + +```bash +opa test policies/ +``` + +## Additional Resources + +- [OPA Documentation](https://www.openpolicyagent.org/docs/latest/) +- [Rego Language Guide](https://www.openpolicyagent.org/docs/latest/policy-language/) +- [OPA Policy Testing](https://www.openpolicyagent.org/docs/latest/policy-testing/) +- [OPA Bundles](https://www.openpolicyagent.org/docs/latest/management-bundles/)