You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Configure External Storage Access for {{{ .starter }}} or Essential
7
+
# Configure External Storage Access
8
+
9
+
<CustomContentplan="starter,essential">
7
10
8
11
If you want to import data from or export data to an external storage in a TiDB Cloud cluster, you need to configure cross-account access. This document describes how to configure access to an external storage for {{{ .starter }}} and {{{ .essential }}} clusters.
9
12
13
+
</CustomContent>
14
+
15
+
<CustomContentplan="premium">
16
+
17
+
If you want to import data from or export data to an external storage in a TiDB Cloud instance, you need to configure cross-account access. This document describes how to configure access to an external storage for {{{ .premium }}} instances.
18
+
19
+
</CustomContent>
20
+
10
21
If you need to configure these external storages for a TiDB Cloud Dedicated cluster, see [Configure External Storage Access for TiDB Cloud Dedicated](/tidb-cloud/dedicated-external-storage.md).
11
22
12
23
## Configure Amazon S3 access
13
24
14
-
To allow a TiDB Cloud clusterto access the source data in your Amazon S3 bucket, configure the bucket access for the cluster using either of the following methods:
25
+
To allow a TiDB Cloud <CustomContentplan="starter,essential">cluster</CustomContent><CustomContentplan="premium">instance</CustomContent> to access the source data in your Amazon S3 bucket, configure the bucket access for the <CustomContentplan="starter,essential">cluster</CustomContent><CustomContentplan="premium">instance</CustomContent> using either of the following methods:
15
26
16
27
-[Use a Role ARN](#configure-amazon-s3-access-using-a-role-arn): use a Role ARN to access your Amazon S3 bucket.
17
28
-[Use an AWS access key](#configure-amazon-s3-access-using-an-aws-access-key): use the access key of an IAM user to access your Amazon S3 bucket.
@@ -22,13 +33,13 @@ It is recommended that you use [AWS CloudFormation](https://docs.aws.amazon.com/
22
33
23
34
> **Note:**
24
35
>
25
-
> Role ARN access to Amazon S3 is only supported for clusters with AWS as the cloud provider. If you use a different cloud provider, use an AWS access key instead. For more information, see [Configure Amazon S3 access using an AWS access key](#configure-amazon-s3-access-using-an-aws-access-key).
36
+
> Role ARN access to Amazon S3 is only supported for <CustomContentplan="starter,essential">clusters</CustomContent><CustomContentplan="premium">instances</CustomContent> with AWS as the cloud provider. If you use a different cloud provider, use an AWS access key instead. For more information, see [Configure Amazon S3 access using an AWS access key](#configure-amazon-s3-access-using-an-aws-access-key).
26
37
27
-
1. Open the **Import** page for your target cluster.
38
+
1. Open the **Import** page for your target <CustomContentplan="starter,essential">cluster</CustomContent><CustomContentplan="premium">instance</CustomContent>.
28
39
29
-
1. Log in to the [TiDB Cloud console](https://tidbcloud.com/) and navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project.
40
+
1. Log in to the [TiDB Cloud console](https://tidbcloud.com/) and <CustomContentplan="starter,essential">navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project.</CustomContent><CustomContentplan="premium">navigate to the [**TiDB Instances**](https://tidbcloud.com/tidbs) page.</CustomContent>
30
41
31
-
2. Click the name of your target cluster to go to its overview page, and then click **Data** > **Import** in the left navigation pane.
42
+
2. Click the name of your target <CustomContentplan="starter,essential">cluster</CustomContent><CustomContentplan="premium">instance</CustomContent> to go to its overview page, and then click **Data** > **Import** in the left navigation pane.
32
43
33
44
2. Open the **Add New ARN** dialog.
34
45
@@ -40,7 +51,7 @@ It is recommended that you use [AWS CloudFormation](https://docs.aws.amazon.com/
40
51
41
52
- If you want to export data to Amazon S3, open the **Add New ARN** dialog as follows:
42
53
43
-
1. Click **Export data to...** > **Amazon S3**. If your cluster has neither imported nor exported any data before, click **Click here to export data to...** > **Amazon S3** at the bottom of the page.
54
+
1. Click **Export data to...** > **Amazon S3**. If your <CustomContentplan="starter,essential">cluster</CustomContent><CustomContentplan="premium">instance</CustomContent> has neither imported nor exported any data before, click **Click here to export data to...** > **Amazon S3** at the bottom of the page.
44
55
2. Fill in the **Folder URI** field.
45
56
3. Choose **AWS Role ARN** and click **Click here to create new one with AWS CloudFormation**.
46
57
@@ -56,7 +67,7 @@ It is recommended that you use [AWS CloudFormation](https://docs.aws.amazon.com/
56
67
57
68
5. After the CloudFormation stack is executed, you can click the **Outputs** tab and find the Role ARN value in the **Value** column.
If you have any trouble creating a role ARN with AWS CloudFormation, you can take the following steps to create one manually:
62
73
@@ -79,10 +90,10 @@ If you have any trouble creating a role ARN with AWS CloudFormation, you can tak
79
90
80
91
4. On the **Create policy** page, click the **JSON** tab.
81
92
82
-
5. Configure the policy in the policy text field according to your needs. The following is an example that you can use to export data from and import data into a TiDB Cloud cluster.
93
+
5. Configure the policy in the policy text field according to your needs. The following is an example that you can use to export data from and import data into a TiDB Cloud <CustomContentplan="starter,essential">cluster</CustomContent><CustomContentplan="premium">instance</CustomContent>.
83
94
84
-
- Exporting data from a TiDB Cloud cluster needs the **s3:PutObject** and **s3:ListBucket** permissions.
85
-
- Importing data into a TiDB Cloud cluster needs the **s3:GetObject**, **s3:GetObjectVersion**, and **s3:ListBucket** permissions.
95
+
- Exporting data from a TiDB Cloud <CustomContentplan="starter,essential">cluster</CustomContent><CustomContentplan="premium">instance</CustomContent> needs the **s3:PutObject** and **s3:ListBucket** permissions.
96
+
- Importing data into a TiDB Cloud <CustomContentplan="starter,essential">cluster</CustomContent><CustomContentplan="premium">instance</CustomContent> needs the **s3:GetObject**, **s3:GetObjectVersion**, and **s3:ListBucket** permissions.
86
97
87
98
```json
88
99
{
@@ -150,7 +161,7 @@ If you have any trouble creating a role ARN with AWS CloudFormation, you can tak
150
161
151
162
- In **Trusted entity type**, select **AWS account**.
152
163
- In **An AWS account**, select **Another AWS account**, and then paste the TiDB Cloud account ID to the **Account ID** field.
153
-
- In **Options**, click **Require external ID (Best practice when a third party will assume this role)**, and then paste the TiDB Cloud External ID to the **External ID** field.If the role is created without a Require external ID, once the configuration is done for one TiDB cluster in a project, all TiDB clusters in that project can use the same Role ARN to access your Amazon S3 bucket. If the role is created with the account ID and external ID, only the corresponding TiDB cluster can access the bucket.
164
+
- In **Options**, click **Require external ID (Best practice when a third party will assume this role)**, and then paste the TiDB Cloud External ID to the **External ID** field.<CustomContent plan="starter,essential"> If the role is created without a Require external ID, once the configuration is done for one TiDB cluster in a project, all TiDB clusters in that project can use the same Role ARN to access your Amazon S3 bucket. If the role is created with the account ID and external ID, only the corresponding TiDB cluster can access the bucket.</CustomContent>
154
165
155
166
3. Click **Next** to open the policy list, choose the policy you just created, and then click **Next**.
156
167
@@ -178,6 +189,8 @@ Take the following steps to configure an access key:
178
189
>
179
190
> TiDB Cloud does not store your access keys. It is recommended that you [delete the access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey) after the import or export is complete.
180
191
192
+
<CustomContent plan="starter,essential">
193
+
181
194
## Configure GCS access
182
195
183
196
To allow a TiDB Cloud cluster to access your GCS bucket, you need to configure the GCS access for the bucket. You can use a service account key to configure the bucket access:
@@ -206,11 +219,15 @@ Take the following steps to configure a service account key:
206
219
207
220
3. Choose the default `JSON` key type, and then click **CREATE** to download the Google Cloud credentials file. The file contains the service account key that you need to use when configuring the GCS access for the TiDB Cloud cluster.
208
221
222
+
</CustomContent>
223
+
224
+
<CustomContent plan="starter,essential">
225
+
209
226
## Configure Azure Blob Storage access
210
227
211
228
To allow TiDB Cloud to access your Azure Blob container, you need to create a service SAS token for the container.
212
229
213
-
You can create a SAS token either using an [Azure ARM template](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/overview) (recommended) or manual configuration.
230
+
You can create a SAS token either using an [Azure ARM template](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/overview) (recommended) or manual configuration.
214
231
215
232
To create a SAS token using an Azure ARM template, take the following steps:
216
233
@@ -221,23 +238,23 @@ To create a SAS token using an Azure ARM template, take the following steps:
221
238
2. Click the name of your target cluster to go to its overview page, and then click **Data** > **Import** in the left navigation pane.
222
239
223
240
2. Open the **Generate New SAS Token via ARM Template Deployment** dialog.
224
-
241
+
225
242
1. Click **Export data to...** > **Azure Blob Storage**. If your cluster has neither imported nor exported any data before, click **Click here to export data to...** > **Azure Blob Storage** at the bottom of the page.
226
-
227
-
2. Scroll down to the **Azure Blob Storage Settings** area, and then click **Click here to create a new one with Azure ARM template** under the SAS Token field.
228
-
243
+
244
+
2. Scroll down to the **Azure Blob Storage Settings** area, and then click **Click here to create a new one with Azure ARM template** under the SAS Token field.
245
+
229
246
3. Create a SAS token with the Azure ARM template.
230
247
231
248
1. In the **Generate New SAS Token via ARM Template Deployment** dialog, click **Click to open the Azure Portal with the pre-configured ARM template**.
232
-
249
+
233
250
2. After logging in to Azure, you will be redirected to the Azure **Custom deployment** page.
234
251
235
252
3. Fill in the **Resource group** and **Storage Account Name** in the **Custom deployment** page. You can get all the information from the storage account overview page where the container is located.
4. Click **Review + create** or **Next** to review the deployment. Click **Create** to start the deployment.
240
-
257
+
241
258
5. After it completes, you will be redirected to the deployment overview page. Navigate to the **Outputs** section to get the SAS token.
242
259
243
260
If you have any trouble creating a SAS token with the Azure ARM template, take the following steps to create one manually:
@@ -246,7 +263,7 @@ If you have any trouble creating a SAS token with the Azure ARM template, take t
246
263
<summary>Click here to see details</summary>
247
264
248
265
1. On the [Azure Storage account](https://portal.azure.com/#browse/Microsoft.Storage%2FStorageAccounts) page, click your storage account to which the container belongs.
249
-
266
+
250
267
2. On your **Storage account** page, click the **Security+network**, and then click **Shared access signature**.
@@ -269,26 +286,28 @@ If you have any trouble creating a SAS token with the Azure ARM template, take t
269
286
270
287
</details>
271
288
289
+
</CustomContent>
290
+
272
291
## Configure Alibaba Cloud Object Storage Service (OSS) access
273
292
274
293
To allow TiDB Cloud to access your Alibaba Cloud OSS bucket, you need to create an AccessKey pair for the bucket.
275
294
276
295
Take the following steps to configure an AccessKey pair:
277
296
278
297
1. Create a RAM user and get the AccessKey pair. For more information, see [Create a RAM user](https://www.alibabacloud.com/help/en/ram/user-guide/create-a-ram-user).
279
-
298
+
280
299
In the **Access Mode** section, select **Using permanent AccessKey to access**.
281
300
282
301
2. Create a custom policy with the required permissions. For more information, see [Create custom policies](https://www.alibabacloud.com/help/en/ram/user-guide/create-a-custom-policy).
283
-
302
+
284
303
- In the **Effect** section, select **Allow**.
285
304
- In the **Service** section, select **Object Storage Service**.
286
305
- In the **Action** section, select the permissions as needed.
287
-
288
-
To import data into a TiDB Cloud cluster, grant **oss:GetObject**, **oss:GetBucketInfo**, and **oss:ListObjects** permissions.
289
306
290
-
To export data from a TiDB Cloud cluster, grant **oss:PutObject**, **oss:GetBucketInfo**, and **oss:ListBuckets** permissions.
291
-
307
+
To import data into a TiDB Cloud <CustomContent plan="starter,essential">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>, grant **oss:GetObject**, **oss:GetBucketInfo**, and **oss:ListObjects** permissions.
308
+
309
+
To export data from a TiDB Cloud <CustomContent plan="starter,essential">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>, grant **oss:PutObject**, **oss:GetBucketInfo**, and **oss:ListBuckets** permissions.
310
+
292
311
- In the **Resource** section, select the bucket and the objects in the bucket.
293
312
294
313
3. Attach the custom policies to the RAM user. For more information, see [Grant permissions to a RAM user](https://www.alibabacloud.com/help/en/ram/user-guide/grant-permissions-to-the-ram-user).
If your source data is stored in Amazon S3 buckets, Azure Blob Storage containers, or Google Cloud Storage (GCS) buckets, before importing or migrating the data to TiDB Cloud, you need to configure cross-account access to the buckets. This document describes how to do this for TiDB Cloud Dedicated clusters.
10
10
11
-
If you need to configure these external storages for {{{ .starter }}} or {{{ .essential }}} clusters, see [Configure External Storage Access for {{{ .starter }}} or Essential](/tidb-cloud/serverless-external-storage.md).
11
+
If you need to configure these external storages for {{{ .starter }}} or {{{ .essential }}} clusters, see [Configure External Storage Access for {{{ .starter }}} or Essential](/tidb-cloud/configure-external-storage-access.md).
0 commit comments