Skip to content

fix object_storage_endpoint #884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 12, 2025
Merged

fix object_storage_endpoint #884

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions src/UserGuide/Master/Table/User-Manual/Tiered-Storage_timecho.md
View file
Open in desktop

Large diffs are not rendered by default.

16 changes: 8 additions & 8 deletions src/UserGuide/Master/Tree/Reference/DataNode-Config-Manual_timecho.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -560,14 +560,14 @@ There will be at most 10 gc.log.* files and each one can reach to 10MB.
|Default| iotdb_data |
|Effective| After restarting system |

* object\_storage\_endpoiont

|Name| object\_storage\_endpoiont |
|:---:|:--------------|
|Description| endpoint of cloud storage |
|Type| string |
|Default| None |
|Effective| After restarting system |
* object\_storage\_endpoint

|Name| object\_storage\_endpoint |
|:---:|:--------------------------------|
|Description| endpoint of cloud storage |
|Type| string |
|Default| None |
|Effective| After restarting system |

* object\_storage\_access\_key

Expand Down
24 changes: 12 additions & 12 deletions src/UserGuide/Master/Tree/User-Manual/Tiered-Storage_timecho.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,20 +34,20 @@ To enable tiered storage in IoTDB, you need to configure the following aspects:

The specific parameter definitions and their descriptions are as follows.

| Configuration | Default | Description | Constraint |
| ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| dn_data_dirs | data/datanode/data | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE |
| Configuration | Default | Description | Constraint |
| --------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| dn_data_dirs | data/datanode/data | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE |
| tier_ttl_in_ms | -1 | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs；"-1" means "unlimited". |
| dn_default_space_usage_thresholds | 0.85 | Define the maximum storage usage threshold ratio for each tier of data directories. When the used space exceeds this ratio, the data will be automatically migrated to the next tier. If the storage usage of the last tier surpasses this threshold, the system will be set to ​​READ_ONLY​​ mode. | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs |
| object_storage_type | `AWS_S3` | Cloud storage type. | all `AWS_S3` is supported. |
| object_storage_bucket | iotdb_data | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used |
| object_storage_endpoiont | | endpoint of cloud storage | endpoint of AWS S3；If remote storage is not used, no configuration required |
| dn_default_space_usage_thresholds | 0.85 | Define the maximum storage usage threshold ratio for each tier of data directories. When the used space exceeds this ratio, the data will be automatically migrated to the next tier. If the storage usage of the last tier surpasses this threshold, the system will be set to ​​READ_ONLY​​ mode. | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs |
| object_storage_type | `AWS_S3` | Cloud storage type. | all `AWS_S3` is supported. |
| object_storage_bucket | iotdb_data | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used |
| object_storage_endpoint | | endpoint of cloud storage | endpoint of AWS S3；If remote storage is not used, no configuration required |
| object_storage_region | (Empty) | Cloud storage Region. | Required only if cloud storage is used. |
| object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key；If remote storage is not used, no configuration required |
| object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret；If remote storage is not used, no configuration required |
| object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key；If remote storage is not used, no configuration required |
| object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret；If remote storage is not used, no configuration required |
| enable_path_style_access | false | Whether to enable path style access for object storage service. | Required only if cloud storage is used. |
| remote_tsfile_cache_dirs | data/datanode/data/cache | Cache directory stored locally in the cloud | If remote storage is not used, no configuration required |
| remote_tsfile_cache_page_size_in_kb | 20480 |Block size of locally cached files stored in the cloud | If remote storage is not used, no configuration required |
| remote_tsfile_cache_dirs | data/datanode/data/cache | Cache directory stored locally in the cloud | If remote storage is not used, no configuration required |
| remote_tsfile_cache_page_size_in_kb | 20480 |Block size of locally cached files stored in the cloud | If remote storage is not used, no configuration required |
| remote_tsfile_cache_max_disk_usage_in_mb | 51200 | Maximum Disk Occupancy Size for Cloud Storage Local Cache | If remote storage is not used, no configuration required |

## 3. local tiered storag configuration example
Expand Down Expand Up @@ -80,7 +80,7 @@ dn_default_space_usage_thresholds=0.2;0.15;0.1
object_storage_type=AWS_S3
object_storage_bucket=iotdb
object_storage_region=<your_region>
object_storage_endpoiont=<your_endpoint>
object_storage_endpoint=<your_endpoint>
object_storage_access_key=<your_access_key>
object_storage_access_secret=<your_access_secret>

Expand Down
16 changes: 8 additions & 8 deletions src/UserGuide/V1.2.x/Reference/DataNode-Config-Manual.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -417,14 +417,14 @@ There will be at most 10 gc.log.* files and each one can reach to 10MB.
|Default| iotdb_data |
|Effective| After restarting system |

* object\_storage\_endpoiont

|Name| object\_storage\_endpoiont |
|:---:|:--------------|
|Description| endpoint of cloud storage |
|Type| string |
|Default| None |
|Effective| After restarting system |
* object\_storage\_endpoint

|Name| object\_storage\_endpoint |
|:---:|:--------------------------------|
|Description| endpoint of cloud storage |
|Type| string |
|Default| None |
|Effective| After restarting system |

* object\_storage\_access\_key

Expand Down
26 changes: 13 additions & 13 deletions src/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,18 @@ To enable tiered storage in IoTDB, you need to configure the following aspects:

The specific parameter definitions and their descriptions are as follows.

| Configuration | Default | Description | Constraint |
| ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| dn_data_dirs | data/datanode/data | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE |
| default_ttl_in_ms | -1 | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs |
| dn_default_space_move_thresholds | 0.15 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs |
| object_storage_type | AWS_S3 | Cloud Storage Type | IoTDB currently only supports AWS S3 as a remote storage type, and this parameter can't be modified |
| object_storage_bucket | iotdb_data | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used |
| object_storage_endpoiont | | endpoint of cloud storage | endpoint of AWS S3；If remote storage is not used, no configuration required |
| object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key；If remote storage is not used, no configuration required |
| object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret；If remote storage is not used, no configuration required |
| remote_tsfile_cache_dirs | data/datanode/data/cache | Cache directory stored locally in the cloud | If remote storage is not used, no configuration required |
| remote_tsfile_cache_page_size_in_kb | 20480 |Block size of locally cached files stored in the cloud | If remote storage is not used, no configuration required |
| Configuration | Default | Description | Constraint |
| --------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| dn_data_dirs | data/datanode/data | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE |
| default_ttl_in_ms | -1 | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs |
| dn_default_space_move_thresholds | 0.15 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs |
| object_storage_type | AWS_S3 | Cloud Storage Type | IoTDB currently only supports AWS S3 as a remote storage type, and this parameter can't be modified |
| object_storage_bucket | iotdb_data | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used |
| object_storage_endpoint | | endpoint of cloud storage | endpoint of AWS S3；If remote storage is not used, no configuration required |
| object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key；If remote storage is not used, no configuration required |
| object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret；If remote storage is not used, no configuration required |
| remote_tsfile_cache_dirs | data/datanode/data/cache | Cache directory stored locally in the cloud | If remote storage is not used, no configuration required |
| remote_tsfile_cache_page_size_in_kb | 20480 |Block size of locally cached files stored in the cloud | If remote storage is not used, no configuration required |
| remote_tsfile_cache_max_disk_usage_in_mb | 51200 | Maximum Disk Occupancy Size for Cloud Storage Local Cache | If remote storage is not used, no configuration required |

## local tiered storag configuration example
Expand Down Expand Up @@ -77,7 +77,7 @@ default_ttl_in_ms=86400000;864000000;-1
dn_default_space_move_thresholds=0.2;0.15;0.1
object_storage_name=AWS_S3
object_storage_bucket=iotdb
object_storage_endpoiont=<your_endpoint>
object_storage_endpoint=<your_endpoint>
object_storage_access_key=<your_access_key>
object_storage_access_secret=<your_access_secret>

Expand Down
16 changes: 8 additions & 8 deletions src/UserGuide/V1.3.x/Reference/DataNode-Config-Manual-old_timecho.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -564,14 +564,14 @@ There will be at most 10 gc.log.* files and each one can reach to 10MB.
|Default| iotdb_data |
|Effective| After restarting system |

* object\_storage\_endpoiont

|Name| object\_storage\_endpoiont |
|:---:|:--------------|
|Description| endpoint of cloud storage |
|Type| string |
|Default| None |
|Effective| After restarting system |
* object\_storage\_endpoint

|Name| object\_storage\_endpoint |
|:---:|:--------------------------------|
|Description| endpoint of cloud storage |
|Type| string |
|Default| None |
|Effective| After restarting system |

* object\_storage\_access\_key

Expand Down
16 changes: 8 additions & 8 deletions src/UserGuide/V1.3.x/Reference/DataNode-Config-Manual_timecho.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -556,14 +556,14 @@ There will be at most 10 gc.log.* files and each one can reach to 10MB.
|Default| iotdb_data |
|Effective| After restarting system |

* object\_storage\_endpoiont

|Name| object\_storage\_endpoiont |
|:---:|:--------------|
|Description| endpoint of cloud storage |
|Type| string |
|Default| None |
|Effective| After restarting system |
* object\_storage\_endpoint

|Name| object\_storage\_endpoint |
|:---:|:--------------------------------|
|Description| endpoint of cloud storage |
|Type| string |
|Default| None |
|Effective| After restarting system |

* object\_storage\_access\_key

Expand Down
Loading