Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -208,4 +208,4 @@ For more details, see [CREATE USER](/sql/sql-commands/ddl/user/user-create-user)

### Connection Security

All connections to Databend Cloud warehouses use TLS encryption by default. For enterprise users requiring additional security, [AWS PrivateLink](/guides/sql-clients/privatelink) is available to establish private connections between your VPC and Databend Cloud.
All connections to Databend Cloud warehouses use TLS encryption by default. For enterprise users requiring additional security, [AWS PrivateLink](/guides/cloud/advanced/private-link) is available to establish private connections between your VPC and Databend Cloud.
Original file line number Diff line number Diff line change
@@ -1,18 +1,8 @@
---
title: "Creating External Stage with AWS IAM Role"
title: "Authenticate with AWS IAM Role"
sidebar_label: "AWS IAM Role"
---

# Why IAM Role

AWS IAM (Identity and Access Management) Role provides a secure and flexible way to manage access to your AWS resources. When working with Databend Cloud, using IAM Role offers several key benefits:

- **Enhanced Security**: Instead of storing AWS access keys and secrets, IAM Role enables temporary credential access, significantly reducing security risks.
- **Simplified Access Management**: You can manage permissions centrally through AWS IAM, making it easier to control who can access your S3 buckets and what operations they can perform.
- **Seamless Integration**: Databend Cloud can securely access your AWS S3 buckets without requiring you to manage or rotate credentials manually.
- **Compliance and Audit**: IAM Role provides detailed audit trails of access to your S3 buckets, helping you maintain compliance with security policies.

By using IAM Role, you can securely connect your Databend Cloud environment to your AWS S3 buckets while maintaining full control over access permissions and security policies.

# How to Use IAM Role

1. Raise a support ticket to get the IAM role ARN for your Databend Cloud organization:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"label": "IAM Role"
}
22 changes: 22 additions & 0 deletions docs/en/guides/20-cloud/30-advanced/01-iam-role/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
title: IAM Role
---

import IndexOverviewList from '@site/src/components/IndexOverviewList';

# Why IAM Role

Cloud-native identity delegation (AWS IAM Role, Azure Managed Identity, Google Service Account federation, etc.) lets Databend Cloud obtain short-lived credentials to your object storage without ever handling raw access keys. That keeps data plane access inside your cloud provider's control plane while you retain ownership of every permission.

## Benefits

- No static keys: temporary credentials eliminate long-lived secrets to rotate or leak.
- Least privilege: fine-grained policies restrict Databend Cloud to only the buckets and actions you approve.
- Central governance: continue auditing and revoking access through your existing IAM workflows.
- Automated rotation: the cloud provider refreshes tokens, so integrations keep working when teams change.

## How It Works

After Databend Cloud support shares the trusted principal information for your organization, you create an IAM role/identity in your cloud account, attach a policy that allows the object storage operations you need (for example reading a set of buckets), and configure the trust policy so only Databend Cloud can assume the role with a unique external ID. Databend Cloud then assumes that role on demand, uses the temporary credentials to access your storage, and automatically logs out when the session expires.

<IndexOverviewList />
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,11 @@ title: "Connecting to Databend Cloud with AWS PrivateLink"
sidebar_label: "AWS PrivateLink"
---

# Why PrivateLink

Connecting to Databend Cloud with AWS PrivateLink provides a secure and private way to access Databend Cloud services without traversing the public internet.

By leveraging AWS PrivateLink, you can establish a private connection between your Virtual Private Cloud (VPC) and Databend Cloud, ensuring that your data remains within your VPC boundaries and is not exposed to the internet. This setup enhances the security of your data and minimizes the risk of unauthorized access.

To connect to Databend Cloud using AWS PrivateLink, you need to configure a VPC endpoint in your VPC. This endpoint acts as a proxy that allows traffic to flow securely between your VPC and Databend Cloud. The VPC endpoint has a private IP address within your VPC and facilitates direct communication with Databend Cloud services.

Once the VPC endpoint is established, you can securely access Databend Cloud services, such as data storage, data processing, and analytics tools, using private IP addresses. This ensures that your data remains within the secure boundaries of your VPC, providing an additional layer of protection.

# How to Setup AWS PrivateLink

1. Provide the AWS account ID you are planning to connect to Databend Cloud:

For example: `952853449216`
For example: `123456789012`

2. Verify your VPC settings

Expand All @@ -27,8 +17,7 @@ Once the VPC endpoint is established, you can securely access Databend Cloud ser

3. Wait for cloud admin adding your account to whitelist, and get a service name for the cluster to connect to:

- `com.amazonaws.vpce.us-east-2.vpce-svc-0e494364a37d35445` (aws, us-east-2, white)
- `com.amazonaws.vpce.us-east-1.vpce-svc-0d9cd5a4a7c19796f` (aws, us-east-1, ivory)
For example: `com.amazonaws.vpce.us-east-2.vpce-svc-0123456789abcdef0`

4. Prepare a security group with tcp 443 port open:

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"label": "Private Link"
}
22 changes: 22 additions & 0 deletions docs/en/guides/20-cloud/30-advanced/02-private-link/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
title: Advanced Usage
---

import IndexOverviewList from '@site/src/components/IndexOverviewList';

# Why PrivateLink

PrivateLink-style private endpoints offered by major clouds (AWS PrivateLink, Azure Private Link, Google Private Service Connect, etc.) let you reach Databend Cloud through private IP addresses inside your own network boundary, so no traffic has to traverse the public internet. That keeps your datasets, credentials, and admin actions on the provider's backbone and aligned with the network policies you already operate.

## Benefits

- Network isolation: traffic never leaves your VPC/VPN boundary, removing exposure to public endpoints.
- Compliance ready: easier to satisfy internal audits and industry requirements that forbid internet egress.
- Stable performance: traffic follows the cloud provider backbone instead of unpredictable internet routes.
- Simplified controls: reuse your existing security groups, route tables, and monitoring to govern access.

## How It Works

After Databend Cloud approves the cloud account or project you plan to connect, you create a private endpoint that points to the Databend PrivateLink service for your region. The cloud provider automatically allocates private IP addresses and, once private DNS is enabled, your Databend Cloud domains resolve to those addresses so every session stays on the secure, private path.

<IndexOverviewList />
3 changes: 3 additions & 0 deletions docs/en/guides/20-cloud/30-advanced/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"label": "Advanced Usage"
}
7 changes: 7 additions & 0 deletions docs/en/guides/20-cloud/30-advanced/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
title: Advanced Usage
---

import IndexOverviewList from '@site/src/components/IndexOverviewList';

<IndexOverviewList />
7 changes: 0 additions & 7 deletions docs/en/guides/30-sql-clients/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,3 @@ Databend supports multiple connection methods to suit different use cases. All S
| **Node.js** | JavaScript Driver | Web Applications | [Node.js Guide](/guides/sql-clients/developers/nodejs) |
| **Java** | JDBC Driver | Enterprise Applications | [JDBC Guide](/guides/sql-clients/developers/jdbc) |
| **Rust** | Native Driver | System Programming | [Rust Guide](/guides/sql-clients/developers/rust) |

## Connection Methods

| Method | Security Level | Use Case | Setup Complexity |
|--------|----------------|----------|------------------|
| **Direct Connection** | Standard | Development, Testing | ⭐ Simple |
| **[AWS PrivateLink](/guides/sql-clients/privatelink)** | High | Production, Enterprise | ⭐⭐⭐ Advanced |
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ When working with external tables, AWS IAM roles provide significant security ad
- **Automatic rotation**: Handles credential rotation automatically
- **Fine-grained control**: Allows for more precise access control

To use IAM roles with Databend Cloud, see [Creating External Stage with AWS IAM Role](/guides/load-data/stage/aws-iam-role) for instructions.
To use IAM roles with Databend Cloud, see [Authenticate with AWS IAM Role](/guides/cloud/advanced/iam-role) for instructions.

## Examples

Expand All @@ -57,8 +57,8 @@ CREATE OR REPLACE TABLE external_table_test (
id INT,
name VARCHAR,
age INT
)
's3://testbucket/13_fuse_external_table/'
)
's3://testbucket/13_fuse_external_table/'
CONNECTION=(connection_name = 'external_table_conn');

-- Update the table to use the new connection with rotated credentials
Expand All @@ -80,8 +80,8 @@ CREATE TABLE sales_data (
order_id INT,
product_name VARCHAR,
quantity INT
)
's3://sales-bucket/data/'
)
's3://sales-bucket/data/'
CONNECTION=(connection_name = 's3_access_key_conn');

-- Later, create a new connection using IAM role authentication
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ IMPORTANT: When objects (stages, tables, etc.) use a connection, they copy and s
## Syntax

```sql
CREATE [ OR REPLACE ] CONNECTION [ IF NOT EXISTS ] <connection_name>
STORAGE_TYPE = '<type>'
CREATE [ OR REPLACE ] CONNECTION [ IF NOT EXISTS ] <connection_name>
STORAGE_TYPE = '<type>'
[ <storage_params> ]
```

Expand Down Expand Up @@ -51,9 +51,9 @@ Choose an authentication method for Amazon S3 and S3-compatible services:
<TabItem value="access-keys" label="Access Keys">

```sql
CREATE CONNECTION <connection_name>
STORAGE_TYPE = 's3'
ACCESS_KEY_ID = '<your-access-key-id>'
CREATE CONNECTION <connection_name>
STORAGE_TYPE = 's3'
ACCESS_KEY_ID = '<your-access-key-id>'
SECRET_ACCESS_KEY = '<your-secret-access-key>';
```

Expand All @@ -66,8 +66,8 @@ CREATE CONNECTION <connection_name>
<TabItem value="iam-role" label="IAM Role">

```sql
CREATE CONNECTION <connection_name>
STORAGE_TYPE = 's3'
CREATE CONNECTION <connection_name>
STORAGE_TYPE = 's3'
ROLE_ARN = '<your-role-arn>';
```

Expand Down Expand Up @@ -154,16 +154,16 @@ To switch an existing table to a new connection, use [`ALTER TABLE ... CONNECTIO

### Using Access Keys

This example creates a connection to Amazon S3 named 'toronto' and establishes an external stage named 'my_s3_stage' linked to the 's3://databend-toronto' URL, using the 'toronto' connection. For more practical examples about connection, see [Usage Examples](index.md#usage-examples).
This example creates a connection to Amazon S3 named 'toronto' and establishes an external stage named 'my_s3_stage' linked to the 's3://databend-toronto' URL, using the 'toronto' connection. For more practical examples about connection, see [Usage Examples](index.md#usage-examples).

```sql
CREATE CONNECTION toronto
STORAGE_TYPE = 's3'
CREATE CONNECTION toronto
STORAGE_TYPE = 's3'
ACCESS_KEY_ID = '<your-access-key-id>'
SECRET_ACCESS_KEY = '<your-secret-access-key>';

CREATE STAGE my_s3_stage
URL = 's3://databend-toronto'
CREATE STAGE my_s3_stage
URL = 's3://databend-toronto'
CONNECTION = (CONNECTION_NAME = 'toronto');
```

Expand All @@ -172,18 +172,18 @@ CREATE STAGE my_s3_stage
This example creates a connection to Amazon S3 using an IAM role and then creates a stage that uses this connection. This approach is more secure as it doesn't require storing access keys in Databend.

```sql
CREATE CONNECTION databend_test
STORAGE_TYPE = 's3'
CREATE CONNECTION databend_test
STORAGE_TYPE = 's3'
ROLE_ARN = 'arn:aws:iam::987654321987:role/databend-test';

CREATE STAGE databend_test
URL = 's3://test-bucket-123'
CREATE STAGE databend_test
URL = 's3://test-bucket-123'
CONNECTION = (CONNECTION_NAME = 'databend_test');

-- You can now query data from your S3 bucket
SELECT * FROM @databend_test/test.parquet LIMIT 1;
```

:::info
To use IAM roles with Databend Cloud, you need to set up a trust relationship between your AWS account and Databend Cloud. See [Creating External Stage with AWS IAM Role](/guides/load-data/stage/aws-iam-role) for detailed instructions.
To use IAM roles with Databend Cloud, you need to set up a trust relationship between your AWS account and Databend Cloud. See [Authenticate with AWS IAM Role](/guides/cloud/advanced/iam-role) for detailed instructions.
:::
8 changes: 8 additions & 0 deletions site-redirects.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1163,6 +1163,14 @@ const siteRedirects = [
{
from: '/sql/sql-functions/aggregate-functions/aggregate-any',
to: '/sql/sql-functions/aggregate-functions/aggregate-any-value'
},
{
from: '/guides/load-data/stage/aws-iam-role',
to: '/guides/cloud/advanced/iam-role'
},
{
from: '/guides/sql-clients/privatelink',
to: '/guides/cloud/advanced/private-link'
}
];
export default siteRedirects;
Loading