Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
222 changes: 117 additions & 105 deletions docs/doc/10-deploy/02-deploying-databend.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,33 +21,6 @@ import TabItem from '@theme/TabItem';

<Tabs groupId="operating-systems">

<TabItem value="MinIO" label="MinIO">

a. Follow the [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide.html) to download and install the MinIO package to your local machine.

b. Open a terminal window and navigate to the folder where MinIO is stored.

c. Run the command `vim server.sh` to create a file with the following content:

```shell
~/minio$ cat server.sh
export MINIO_ROOT_USER=minioadmin
export MINIO_ROOT_PASSWORD=minioadmin
./minio server --address :9900 ./data
```

d. Run the following commands to start the MinIO server:

```shell
chmod +x server.sh
./server.sh
```

e. In your browser, go to <http://127.0.0.1:9900> and enter the credentials (`minioadmin` / `minioadmin`) to log in to the MinIO Console.

f. In the MinIO Console, create a bucket named `databend`.

</TabItem>

<TabItem value="Amazon S3" label="Amazon S3">

Expand Down Expand Up @@ -78,7 +51,7 @@ For information about how to manage buckets and OAuth2 credentials in Google Clo

</TabItem>

<TabItem value="Tencent COS" label="Tencent COS">
<TabItem value="Azure Blob" label="Azure Blob">

Before deploying Databend, make sure you have successfully set up your object storage environment in the cloud, and the following tasks have been completed:

Expand All @@ -88,12 +61,12 @@ Before deploying Databend, make sure you have successfully set up your object st

For information about how to manage buckets and Access Keys for your cloud object storage, refer to the user manual from the solution provider. Here are some useful links you may need:

- <https://cloud.tencent.com/document/product/436/13309>
- <https://cloud.tencent.com/document/product/436/68282>
- <https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container>
- <https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?tabs=azure-portal#view-account-access-keys>

</TabItem>

<TabItem value="Alibaba OSS" label="Alibaba OSS">
<TabItem value="Tencent COS" label="Tencent COS">

Before deploying Databend, make sure you have successfully set up your object storage environment in the cloud, and the following tasks have been completed:

Expand All @@ -103,12 +76,12 @@ Before deploying Databend, make sure you have successfully set up your object st

For information about how to manage buckets and Access Keys for your cloud object storage, refer to the user manual from the solution provider. Here are some useful links you may need:

- <https://www.alibabacloud.com/help/zh/object-storage-service/latest/create-buckets-2>
- <https://help.aliyun.com/document_detail/53045.htm>
- <https://cloud.tencent.com/document/product/436/13309>
- <https://cloud.tencent.com/document/product/436/68282>

</TabItem>

<TabItem value="Wasabi" label="Wasabi">
<TabItem value="Alibaba OSS" label="Alibaba OSS">

Before deploying Databend, make sure you have successfully set up your object storage environment in the cloud, and the following tasks have been completed:

Expand All @@ -118,11 +91,12 @@ Before deploying Databend, make sure you have successfully set up your object st

For information about how to manage buckets and Access Keys for your cloud object storage, refer to the user manual from the solution provider. Here are some useful links you may need:

- <https://docs.wasabi.com/docs/creating-a-bucket>
- <https://docs.wasabi.com/docs/access-keys-1>
- <https://www.alibabacloud.com/help/zh/object-storage-service/latest/create-buckets-2>
- <https://help.aliyun.com/document_detail/53045.htm>

</TabItem>


<TabItem value="QingCloud QingStor" label="QingCloud QingStor">

Before deploying Databend, make sure you have successfully set up your object storage environment in the cloud, and the following tasks have been completed:
Expand All @@ -138,7 +112,8 @@ For information about how to manage buckets and Access Keys for your cloud objec

</TabItem>

<TabItem value="Azure Blob Storage" label="Azure Blob Storage">

<TabItem value="Huawei OBS" label="Huawei OBS">

Before deploying Databend, make sure you have successfully set up your object storage environment in the cloud, and the following tasks have been completed:

Expand All @@ -148,12 +123,12 @@ Before deploying Databend, make sure you have successfully set up your object st

For information about how to manage buckets and Access Keys for your cloud object storage, refer to the user manual from the solution provider. Here are some useful links you may need:

- <https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container>
- <https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?tabs=azure-portal#view-account-access-keys>
- <https://support.huaweicloud.com/intl/en-us/usermanual-obs/en-us_topic_0045853662.html>
- <https://support.huaweicloud.com/intl/en-us/api-obs/obs_04_0116.html>

</TabItem>

<TabItem value="Huawei Cloud OBS" label="Huawei Cloud OBS">
<TabItem value="Wasabi" label="Wasabi">

Before deploying Databend, make sure you have successfully set up your object storage environment in the cloud, and the following tasks have been completed:

Expand All @@ -163,10 +138,39 @@ Before deploying Databend, make sure you have successfully set up your object st

For information about how to manage buckets and Access Keys for your cloud object storage, refer to the user manual from the solution provider. Here are some useful links you may need:

- <https://support.huaweicloud.com/intl/en-us/usermanual-obs/en-us_topic_0045853662.html>
- <https://support.huaweicloud.com/intl/en-us/api-obs/obs_04_0116.html>
- <https://docs.wasabi.com/docs/creating-a-bucket>
- <https://docs.wasabi.com/docs/access-keys-1>

</TabItem>

<TabItem value="MinIO" label="MinIO">

a. Follow the [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide.html) to download and install the MinIO package to your local machine.

b. Open a terminal window and navigate to the folder where MinIO is stored.

c. Run the command `vim server.sh` to create a file with the following content:

```shell
~/minio$ cat server.sh
export MINIO_ROOT_USER=minioadmin
export MINIO_ROOT_PASSWORD=minioadmin
./minio server --address :9900 ./data
```

d. Run the following commands to start the MinIO server:

```shell
chmod +x server.sh
./server.sh
```

e. In your browser, go to <http://127.0.0.1:9900> and enter the credentials (`minioadmin` / `minioadmin`) to log in to the MinIO Console.

f. In the MinIO Console, create a bucket named `databend`.

</TabItem>

<TabItem value="WebHDFS" label="WebHDFS">

Before deploying Databend, make sure you have successfully set up your Hadoop environment, and the following tasks have been completed:
Expand Down Expand Up @@ -326,21 +330,7 @@ secret_access_key = "<your-account-key>"
d. Set your values in the `[storage.s3]`, `[storage.azblob]`, `[storage.gcs]`, `[storage.obs]` or `[storage.webhdfs]` block. Please note that the field `endpoint_url` refers to the service URL of your storage region and varies depending on the object storage solution you use:

<Tabs groupId="operating-systems">
<TabItem value="MinIO" label="MinIO">

```toml
[storage]
# s3
type = "s3"

[storage.s3]
bucket = "databend"
endpoint_url = "http://127.0.0.1:9900"
access_key_id = "minioadmin"
secret_access_key = "minioadmin"
```

</TabItem>

<TabItem value="Amazon S3" label="Amazon S3">

Expand Down Expand Up @@ -392,6 +382,27 @@ credential = "<your-credential>"

</TabItem>

<TabItem value="Azure Blob" label="Azure Blob">

```toml
[storage]
# azblob
type = "azblob"

[storage.azblob]
endpoint_url = "https://<your-storage-account-name>.blob.core.windows.net"

# https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container
container = "<your-azure-storage-container-name>"
account_name = "<your-storage-account-name>"

# https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?tabs=azure-portal#view-account-access-keys
account_key = "<your-account-key>"
```

</TabItem>


<TabItem value="Tencent COS" label="Tencent COS">

```toml
Expand Down Expand Up @@ -458,35 +469,6 @@ In this example OSS region id is `oss-cn-beijing-internal`.

</TabItem>

<TabItem value="Wasabi" label="Wasabi">

```toml
[storage]
# s3
type = "s3"

[storage.s3]
# How to create a bucket:
// highlight-next-line
bucket = "<your-bucket>"

# You can get the URL from:
# https://wasabi-support.zendesk.com/hc/en-us/articles/360015106031-What-are-the-service-URLs-for-Wasabi-s-different-regions-
// highlight-next-line
endpoint_url = "https://s3.us-east-2.wasabisys.com"

# How to get access_key_id and secret_access_key:
// highlight-next-line
access_key_id = "<your-key-id>"
// highlight-next-line
secret_access_key = "<your-access-key>"
```

:::tip
In this example Wasabi region is `us-east-2`.
:::

</TabItem>

<TabItem value="QingCloud QingStor" label="QingCloud QingStor">

Expand Down Expand Up @@ -514,27 +496,8 @@ In this example QingStor region is `pek3b`.

</TabItem>

<TabItem value="Azure Blob Storage" label="Azure Blob Storage">

```toml
[storage]
# azblob
type = "azblob"

[storage.azblob]
endpoint_url = "https://<your-storage-account-name>.blob.core.windows.net"

# https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container
container = "<your-azure-storage-container-name>"
account_name = "<your-storage-account-name>"

# https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?tabs=azure-portal#view-account-access-keys
account_key = "<your-account-key>"
```

</TabItem>

<TabItem value="Huawei Cloud OBS" label="Huawei Cloud OBS">
<TabItem value="Huawei OBS" label="Huawei OBS">

```toml
[storage]
Expand Down Expand Up @@ -564,6 +527,55 @@ In this example OBS region is `cn-north-4`.
:::

</TabItem>

<TabItem value="Wasabi" label="Wasabi">

```toml
[storage]
# s3
type = "s3"

[storage.s3]
# How to create a bucket:
// highlight-next-line
bucket = "<your-bucket>"

# You can get the URL from:
# https://wasabi-support.zendesk.com/hc/en-us/articles/360015106031-What-are-the-service-URLs-for-Wasabi-s-different-regions-
// highlight-next-line
endpoint_url = "https://s3.us-east-2.wasabisys.com"

# How to get access_key_id and secret_access_key:
// highlight-next-line
access_key_id = "<your-key-id>"
// highlight-next-line
secret_access_key = "<your-access-key>"
```

:::tip
In this example Wasabi region is `us-east-2`.
:::

</TabItem>


<TabItem value="MinIO" label="MinIO">

```toml
[storage]
# s3
type = "s3"

[storage.s3]
bucket = "databend"
endpoint_url = "http://127.0.0.1:9900"
access_key_id = "minioadmin"
secret_access_key = "minioadmin"
```

</TabItem>


<TabItem value="WebHDFS" label="WebHDFS">

```toml
Expand Down