Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 15 additions & 7 deletions docs/en/docs/sql-manual/sql-reference/Data-Types/VARIANT.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,11 @@ Below are test results based on clickbench data:

| | Storage Space |
|--------------|------------|
| Predefined Static Columns | 24.329 GB |
| VARIANT Type | 24.296 GB |
| JSON Type | 46.730 GB |
| Predefined Static Columns | 12.618 GB |
| VARIANT Type | 12.718 GB |
| JSON Type | 35.711 GB |

**Saves approximately 50% storage capacity**
**Saves approximately 65% storage capacity**

| Query Counts | Predefined Static Columns | VARIANT Type | JSON Type |
|---------------------|---------------------------|--------------|-----------------|
Expand Down Expand Up @@ -84,12 +84,20 @@ CREATE TABLE IF NOT EXISTS ${table_name} (
)
table_properties;

-- Create an bloom filter on v column, to enhance query seed on sub columns
CREATE TABLE IF NOT EXISTS ${table_name} (
k BIGINT,
v VARIANT
)
...
properties("replication_num" = "1", "bloom_filter_columns" = "v");

```

**Query Syntax**

``` sql
-- use v['a']['b'] format for example
-- use v['a']['b'] format for example, v['properties']['title'] type is Variant
SELECT v['properties']['title'] from ${table_name}

```
Expand Down Expand Up @@ -351,8 +359,8 @@ When the above types cannot be compatible, they will be transformed into JSON ty
**Other limitations include:**

- Aggregate models are currently not supported.
- VARIANT columns can only create inverted indexes.
- Using the **RANDOM** mode is recommended for higher write performance.
- VARIANT columns can only create inverted indexes or bloom filter to speed up query.
- Using the **RANDOM** mode or [group commit](https://doris.apache.org/docs/dev/data-operate/import/import-way/group-commit-manual/) mode is recommended for higher write performance.
- Non-standard JSON types such as date and decimal should ideally use static types for better performance, since these types are infered to text type.
- Arrays with dimensions of 2 or higher will be stored as JSONB encoding, which might perform less efficiently than native arrays.
- Not supported as primary or sort keys.
Expand Down
22 changes: 15 additions & 7 deletions docs/zh-CN/docs/sql-manual/sql-reference/Data-Types/VARIANT.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,13 @@ VARIANT类型

| | 存储空间 |
|--------------|------------|
| 预定义静态列 | 24.329 GB |
| variant 类型 | 24.296 GB |
| json 类型 | 46.730 GB |
| 预定义静态列 | 12.618 GB |
| variant 类型 | 12.718 GB |
| json 类型 | 35.711 GB |



**节省约 50%存储容量**
**节省约 65%存储容量**

| 查询次数 | 预定义静态列 | variant 类型 | json 类型 |
|----------------|--------------|--------------|-----------------|
Expand Down Expand Up @@ -88,12 +88,20 @@ CREATE TABLE IF NOT EXISTS ${table_name} (
INDEX idx_var(v) USING INVERTED [PROPERTIES("parser" = "english|unicode|chinese")] [COMMENT 'your comment']
)
table_properties;

-- 在v列创建bloom filter
CREATE TABLE IF NOT EXISTS ${table_name} (
k BIGINT,
v VARIANT
)
...
properties("replication_num" = "1", "bloom_filter_columns" = "v");
```

**查询语法**

``` sql
-- 使用 v['a']['b'] 形式例如
-- 使用 v['a']['b'] 形式如下,v['properties']['title']类型是Variant
SELECT v['properties']['title'] from ${table_name}
```

Expand Down Expand Up @@ -359,8 +367,8 @@ VARIANT 动态列与预定义静态列几乎一样高效。处理诸如日志之
其它限制如下:

- 目前不支持 Aggregate 模型
- VARIANT 列只能创建倒排索引
- **推荐使用 RANDOM 模式, 写入性能更高效**
- VARIANT 列只能创建倒排索引或者bloom filter来加速过滤
- **推荐使用 RANDOM 模式和[Group Commit](https://doris.apache.org/zh-CN/docs/dev/data-operate/import/import-way/group-commit-manual/)模式, 写入性能更高效**
- 日期、decimal 等非标准 JSON 类型会被默认推断成字符串类型,所以尽可能从 VARIANT 中提取出来,用静态类型,性能更好
- 2 维及其以上的数组列存化会被存成 JSONB 编码,性能不如原生数组
- 不支持作为主键或者排序键
Expand Down