Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion community/source-install/compilation-with-ldb-toolchain.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
LDB Toolchain stands for **Linux Distribution Based Toolchain Generator**. It provides a standalone, portable, modern C++ compilation toolchain that can build modern C++ projects on almost any Linux distribution.
:::

Thanks to [Amos Bird](https://github.com/amosbird) for the contribution.

Check notice on line 42 in community/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//github.com/amosbird. Owner%3A @apache/doris-website-maintainers

## Version Mapping

Expand Down Expand Up @@ -67,7 +67,7 @@

### 1.1 Download `ldb_toolchain_gen.sh`

Download the matching version from [ldb_toolchain_gen Releases](https://github.com/amosbird/ldb_toolchain_gen/releases):

Check notice on line 70 in community/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//github.com/amosbird/ldb_toolchain_gen/releases. Owner%3A @apache/doris-website-maintainers

- x86_64 architecture: download `ldb_toolchain_gen.sh`
- ARM architecture: download `ldb_toolchain_gen.aarch64.sh`
Expand Down Expand Up @@ -99,10 +99,10 @@

| Component | Recommended version | Download URL |
| ---- | -------- | -------- |
| JDK 8 | 1.8.0_391 | [jdk-8u391-linux-x64.tar.gz](https://doris-thirdparty-1308700295.cos.ap-beijing.myqcloud.com/tools/jdk-8u391-linux-x64.tar.gz) |

Check notice on line 102 in community/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//doris-thirdparty-1308700295.cos.ap-beijing.myqcloud.com/tools/jdk-8u391-linux-x64.tar.gz. Owner%3A @apache/doris-website-maintainers
| JDK 17 | 17.0.10 | [jdk-17.0.10_linux-x64](https://download.oracle.com/java/17/archive/jdk-17.0.10_linux-x64_bin.tar.gz) |

Check notice on line 103 in community/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//download.oracle.com/java/17/archive/jdk-17.0.10_linux-x64_bin.tar.gz. Owner%3A @apache/doris-website-maintainers
| Maven | 3.9.9 | [apache-maven-3.9.9-bin.tar.gz](https://dlcdn.apache.org/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz) |
| Maven | 3.9.9 | [apache-maven-3.9.9-bin.tar.gz](https://archive.apache.org/dist/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz) |

Check notice on line 104 in community/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//archive.apache.org/dist/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz. Owner%3A @apache/doris-website-maintainers
| Node | v12.13.0 | [node-v12.13.0-linux-x64.tar.gz](https://doris-thirdparty-repo.bj.bcebos.com/thirdparty/node-v12.13.0-linux-x64.tar.gz) |

Check notice on line 105 in community/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//doris-thirdparty-repo.bj.bcebos.com/thirdparty/node-v12.13.0-linux-x64.tar.gz. Owner%3A @apache/doris-website-maintainers

JDK version selection rules:

Expand Down
4 changes: 2 additions & 2 deletions docs/table-design/data-model/unique.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ The Doris Unique Key Model provides two implementations, compared as follows:

| Implementation | Default since version | Merge timing | Query performance | Predicate pushdown | Applicable scenarios |
| --- | --- | --- | --- | --- | --- |
| Merge-on-write | Default since 1.2 | Merged immediately at write time | High | Supported | Most scenarios, balancing query and write performance |
| Merge-on-read | Default before 1.2 | Merged at query or compaction time | Lower | Not supported | Write-heavy, read-light scenarios |
| Merge-on-write | Default since 2.1 (introduced in 1.2) | Merged immediately at write time | High | Supported | Most scenarios, balancing query and write performance |
| Merge-on-read | Default before 2.1 | Merged at query or compaction time | Lower | Not supported | Write-heavy, read-light scenarios |

- **Merge-on-write**: Records with the same Key are merged immediately at write time, ensuring that the storage always holds the latest data. This mode balances query and write performance, avoids merging data across multiple versions, and supports predicate pushdown to the storage layer. **This mode is recommended for most scenarios.**
- **Merge-on-read**: Data is not merged at write time but is appended incrementally, with multiple versions retained inside Doris. At query or compaction time, versions with the same Key are merged. This mode suits write-heavy, read-light scenarios, but queries must merge multiple versions and predicates cannot be pushed down, which may affect query speed.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
LDB Toolchain 全称 **Linux Distribution Based Toolchain Generator**,提供独立、可移植的现代 C++ 编译工具链,可在几乎所有 Linux 发行版上编译现代 C++ 项目。
:::

感谢 [Amos Bird](https://github.com/amosbird) 的贡献。

Check notice on line 42 in i18n/zh-CN/docusaurus-plugin-content-docs-community/current/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//github.com/amosbird. Owner%3A @apache/doris-website-maintainers

## 版本对照

Expand Down Expand Up @@ -67,7 +67,7 @@

### 1.1 下载 `ldb_toolchain_gen.sh`

从 [ldb_toolchain_gen Releases](https://github.com/amosbird/ldb_toolchain_gen/releases) 下载对应版本:

Check notice on line 70 in i18n/zh-CN/docusaurus-plugin-content-docs-community/current/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//github.com/amosbird/ldb_toolchain_gen/releases. Owner%3A @apache/doris-website-maintainers

- x86_64 架构:下载 `ldb_toolchain_gen.sh`
- ARM 架构:下载 `ldb_toolchain_gen.aarch64.sh`
Expand Down Expand Up @@ -99,9 +99,9 @@

| 组件 | 推荐版本 | 下载地址 |
| ---- | -------- | -------- |
| JDK 8 | 1.8.0_391 | [jdk-8u391-linux-x64.tar.gz](https://doris-thirdparty-1308700295.cos.ap-beijing.myqcloud.com/tools/jdk-8u391-linux-x64.tar.gz) |

Check notice on line 102 in i18n/zh-CN/docusaurus-plugin-content-docs-community/current/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//doris-thirdparty-1308700295.cos.ap-beijing.myqcloud.com/tools/jdk-8u391-linux-x64.tar.gz. Owner%3A @apache/doris-website-maintainers
| JDK 17 | 17.0.10 | [jdk-17.0.10_linux-x64](https://download.oracle.com/java/17/archive/jdk-17.0.10_linux-x64_bin.tar.gz) |

Check notice on line 103 in i18n/zh-CN/docusaurus-plugin-content-docs-community/current/source-install/compilation-with-ldb-toolchain.md

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//download.oracle.com/java/17/archive/jdk-17.0.10_linux-x64_bin.tar.gz. Owner%3A @apache/doris-website-maintainers
| Maven | 3.9.9 | [apache-maven-3.9.9-bin.tar.gz](https://dlcdn.apache.org/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz) |
| Maven | 3.9.9 | [apache-maven-3.9.9-bin.tar.gz](https://archive.apache.org/dist/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz) |
| Node | v12.13.0 | [node-v12.13.0-linux-x64.tar.gz](https://doris-thirdparty-repo.bj.bcebos.com/thirdparty/node-v12.13.0-linux-x64.tar.gz) |

JDK 版本选择规则:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@

在 Doris 中主键模型有两种实现方式:

* **写时合并**(merge-on-write):自 1.2 版本起,Doris 默认使用写时合并模式,数据在写入时立即合并相同 Key 的记录,确保存储的始终是最新数据。写时合并兼顾查询和写入性能,避免多个版本的数据合并,并支持谓词下推到存储层。大多数场景推荐使用此模式;
* **写时合并**(merge-on-write):自 1.2 版本引入,并自 2.1 版本起默认启用写时合并模式,数据在写入时立即合并相同 Key 的记录,确保存储的始终是最新数据。写时合并兼顾查询和写入性能,避免多个版本的数据合并,并支持谓词下推到存储层。大多数场景推荐使用此模式;

* **读时合并**(merge-on-read):在 1.2 版本前,Doris 中的主键模型默认使用读时合并模式,数据在写入时并不进行合并,以增量的方式被追加存储,在 Doris 内保留多个版本。查询或 Compaction 时,会对数据进行相同 Key 的版本合并。读时合并适合写多读少的场景,在查询时需要进行多个版本合并,谓词无法下推,可能会影响到查询速度。

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@

- 从官网下载

```shell

Check warning on line 30 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers
# 包含依赖的安装包
wget https://apache-doris-releases.oss-cn-beijing.aliyuncs.com/extension/logstash-output-doris-1.2.0-java.gem
```
Expand All @@ -38,7 +38,7 @@
cd extension/logstash/

gem build logstash-output-doris.gemspec
```

Check warning on line 41 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers

### 安装插件

Expand Down Expand Up @@ -88,7 +88,7 @@
## 使用示例

### TEXT 日志采集示例

Check warning on line 91 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers
该示例以 Doris FE 的日志为例展示 TEXT 日志采集。

**1. 数据**
Expand All @@ -100,7 +100,7 @@
2024-07-08 21:18:53,710 WARN (STATS_FETCH-0|208) [StmtExecutor.executeInternalQuery():3332] Failed to run internal SQL: OriginStatement{originStmt='SELECT * FROM __internal_schema.column_statistics WHERE part_id is NULL ORDER BY update_time DESC LIMIT 500000', idx=0}
org.apache.doris.common.UserException: errCode = 2, detailMessage = tablet 10031 has no queryable replicas. err: replica 10032's backend 10008 does not exist or not alive
at org.apache.doris.planner.OlapScanNode.addScanRangeLocations(OlapScanNode.java:931) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.planner.OlapScanNode.computeTabletInfo(OlapScanNode.java:1197) ~[doris-fe.jar:1.2-SNAPSHOT]

Check warning on line 103 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers
```

**2. 建表**
Expand Down Expand Up @@ -141,7 +141,7 @@
"dynamic_partition.create_history_partition" = "true",
"compaction_policy" = "time_series"
);
```

Check warning on line 144 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers

**3. Logstash 配置**

Expand All @@ -152,7 +152,7 @@
pipeline.batch.size: 1000000
pipeline.batch.delay: 10000
```

Check warning on line 155 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers

某个日志采集的配置文件如 logstash_doris_log.conf 主要由 3 部分组成,分别对应 ETL 的各个部分:
1. input 负责读取原始数据
Expand Down Expand Up @@ -192,7 +192,7 @@
# doris output 将数据输出到 Doris,使用的是 Stream Load HTTP 接口。通过 headers 参数指定了 Stream Load 的数据格式为 JSON,通过 mapping 参数指定 Logstash 字段到 JSON 字段的映射。由于 headers 指定了 "format" => "json",Stream Load 会自动解析 JSON 字段写入对应的 Doris 表的字段。
output {
doris {
http_hosts => ["http://localhost:8630"]
http_hosts => ["http://localhost:8030"]
user => "root"
password => ""
db => "log_db"
Expand All @@ -215,7 +215,7 @@
}
log_request => true
}
}

Check warning on line 218 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers

```

Expand Down Expand Up @@ -255,14 +255,14 @@


### JSON 日志采集示例

Check warning on line 258 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers
该样例以 github events archive 的数据为例展示 JSON 日志采集。

**1. 数据**

github events archive 是 github 用户操作事件的归档数据,格式是 JSON,可以从 https://www.gharchive.org/ 下载,比如下载 2024 年 1 月 1 日 15 点的数据。

```

Check warning on line 265 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers
wget https://data.gharchive.org/2024-01-01-15.json.gz

```
Expand Down Expand Up @@ -296,7 +296,7 @@
"before": "85e6544ede4ae3f132fe2f5f1ce0ce35a3169d21"
},
"public": true,
"created_at": "2024-04-01T23:00:00Z"

Check warning on line 299 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/ecosystem/observability/logstash.md

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers
}
```

Expand Down Expand Up @@ -372,7 +372,7 @@

output {
doris {
http_hosts => ["http://fe1:8630", "http://fe2:8630", "http://fe3:8630"]
http_hosts => ["http://fe1:8030", "http://fe2:8030", "http://fe3:8030"]
user => "root"
password => ""
db => "log_db"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ SECONDS_SUB(<datetime>, <seconds>)
| 参数 | 说明 |
|--------------|---------------------------------------------|
| `<datetime>` | 必填,输入的日期时间值,支持 DATETIME 或 DATE 类型 |
| `<seconds>` | 必填,要减少或增加的秒数,支持整数类型(INT)。正数表示增加秒数,负数表示减少秒数。 |
| `<seconds>` | 必填,要减少或增加的秒数,支持整数类型(INT)。正数表示减少秒数,负数表示增加秒数。 |

## 返回值
- 返回一个日期时间值,类型与输入的 `<datetime>` 类型一致。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@

在 Doris 中主键表有两种实现方式:

* **写时合并**(merge-on-write):自 1.2 版本起,Doris 默认使用写时合并模式,数据在写入时立即合并相同 Key 的记录,确保存储的始终是最新数据。写时合并兼顾查询和写入性能,避免多个版本的数据合并,并支持谓词下推到存储层。大多数场景推荐使用此模式;
* **写时合并**(merge-on-write):自 1.2 版本引入,并自 2.1 版本起默认启用写时合并模式,数据在写入时立即合并相同 Key 的记录,确保存储的始终是最新数据。写时合并兼顾查询和写入性能,避免多个版本的数据合并,并支持谓词下推到存储层。大多数场景推荐使用此模式;

* **读时合并**(merge-on-read):在 1.2 版本前,Doris 中的主键表默认使用读时合并模式,数据在写入时并不进行合并,以增量的方式被追加存储,在 Doris 内保留多个版本。查询或 Compaction 时,会对数据进行相同 Key 的版本合并。读时合并适合写多读少的场景,在查询时需要进行多个版本合并,谓词无法下推,可能会影响到查询速度。
* **读时合并**(merge-on-read):在 2.1 版本前,Doris 中的主键表默认使用读时合并模式,数据在写入时并不进行合并,以增量的方式被追加存储,在 Doris 内保留多个版本。查询或 Compaction 时,会对数据进行相同 Key 的版本合并。读时合并适合写多读少的场景,在查询时需要进行多个版本合并,谓词无法下推,可能会影响到查询速度。

在 Doris 中基于主键表更新有两种语义:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ interval:
- [恢复-JOB](../job/RESUME-JOB.md)
- [删除-JOB](../job/DROP-JOB.md)
- [查询-JOB](../../../sql-manual/sql-functions/table-valued-functions/jobs.md)
- [查询-TASKS](../../sql-functions/table-valued-functions/jobs.md)
- [查询-TASKS](../../../sql-manual/sql-functions/table-valued-functions/tasks.md)

## CONFIG

Expand Down
4 changes: 2 additions & 2 deletions versioned_docs/version-4.x/table-design/data-model/unique.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ The Doris Unique Key Model provides two implementations, compared as follows:

| Implementation | Default since version | Merge timing | Query performance | Predicate pushdown | Applicable scenarios |
| --- | --- | --- | --- | --- | --- |
| Merge-on-write | Default since 1.2 | Merged immediately at write time | High | Supported | Most scenarios, balancing query and write performance |
| Merge-on-read | Default before 1.2 | Merged at query or compaction time | Lower | Not supported | Write-heavy, read-light scenarios |
| Merge-on-write | Default since 2.1 (introduced in 1.2) | Merged immediately at write time | High | Supported | Most scenarios, balancing query and write performance |
| Merge-on-read | Default before 2.1 | Merged at query or compaction time | Lower | Not supported | Write-heavy, read-light scenarios |

- **Merge-on-write**: Records with the same Key are merged immediately at write time, ensuring that the storage always holds the latest data. This mode balances query and write performance, avoids merging data across multiple versions, and supports predicate pushdown to the storage layer. **This mode is recommended for most scenarios.**
- **Merge-on-read**: Data is not merged at write time but is appended incrementally, with multiple versions retained inside Doris. At query or compaction time, versions with the same Key are merged. This mode suits write-heavy, read-light scenarios, but queries must merge multiple versions and predicates cannot be pushed down, which may affect query speed.
Expand Down
Loading