diff --git a/blog/xiaomi_vector.md b/blog/xiaomi_vector.md new file mode 100644 index 0000000000000..3609d7f2fb7f1 --- /dev/null +++ b/blog/xiaomi_vector.md @@ -0,0 +1,182 @@ +--- +{ + "title": "Practice and Optimization of Apache Doris in Xiaomi", + "language": "en" +} +--- + + + +> Guide: Xiaomi Group introduced Apache Doris in 2019. At present, Apache Doris has been widely used in dozens of business departments within Xiaomi. A set of data ecology with Apache Doris has been formed. This article is transcribed from an online meetup speech of the Doris community, aiming to share the practice of Apache Doris in Xiaomi. + +> Author: ZuoWei, OLAP Engineer, Xiaomi + +![kv](/images/xiaomi/en/kv.jpg) + +# About Xiaomi +[Xiaomi Corporation](https://www.mi.com/global) (“Xiaomi” or the “Group”; HKG:1810), a consumer electronics and smart manufacturing company with smartphones and smart hardware connected by an Internet of Things (IoT) platform. In 2021, Xiaomi's total revenue amounted to RMB328.3 billion(USD472,231,316,200), an increase of 33.5% year-over-year; Adjusted net profit was RMB22.0 billion(USD3,164,510,800), an increase of 69.5% year-over-year. + +Due to the growing need of data analysis, Xiaomi Group introduced Apache Doris in 2019. As one of the earliest users of Apache Doris, Xiaomi Group has been deeply involved in the open-source community. After three years of development, Apache Doris has been widely used in dozens of business departments within Xiaomi, such as Advertising, New Retail, Growth Analysis, Dashboards, UserPortraits, [AISTAR](https://airstar.com/home), [Xiaomi Youpin](https://www.xiaomiyoupin.com). Within Xiaomi, a data ecosystem has been built around Apache Doris. + +![page_1](/images/xiaomi/en/page_1.jpg) + +At present, Apache Doris already has dozens of clusters in Xiaomi, with an overall scale of hundreds of virtual machines . Among them, the largest single cluster reaches nearly 100 nodes, with dozens of real-time data synchronization tasks. And the largest daily increment of a single table rocket to 12 billion, supporting PB-level storage. And a single cluster can support more than 20,000 multi-dimensional analysis queries per day. + +# Architecture Evolution +The original intention of Xiaomi to introduce Apache Doris is to solve the problems encountered in user behavior analysis. With the development of Xiaomi's Internet business, the demand for growth analysis using user behavior data is becoming stronger and stronger. If each business branch builds its own growth analysis system, it will not only be costly, but also inefficient. Therefore, if there is a product that can help them stop worrying about underlying complex technical details, it would be great to have relevant business personnel focus on their own technical work. In this way, it can greatly improve work efficiency. Therefore, Xiaomi Big Data and the cloud platform jointly developed the growth analysis system called Growing Analytics (referred to as GA), which aims to provide a flexible multi-dimensional real-time query and analysis platform, which can manage data access and query solutions in a unified way, and help business branches to refine operation. + +## Previous Architecture +The growth analysis platform project was established in mid-2018. At that time, based on the consideration of development time and cost, Xiaomi reused various existing big data basic components (HDFS, Kudu, SparkSQL, etc.) to build a growth analysis query system based on Lambda architecture. The architecture of the first version of the GA system is shown in the figure below, including the following aspects: + +- Data Source: The data source is the front-end embedded data and user behavior data. +- Data Access: The event tracking data is uniformly cleaned and ingested into Xiaomi's internal self-developed message queue, and the data is imported into Kudu through Spark Streaming. +- Storage: Separate hot and cold data in the storage layer. Hot data is stored in Kudu, and cold data is stored in HDFS. At the same time, partitioning is carried out in the storage layer. When the partition unit is day, part of the data will be cooled and stored on HDFS every night. +- Compute and Query: In the query layer, use SparkSQL to perform federated queries on the data on Kudu and HDFS, and finally display the query results on the front-end page. + +![page_2](/images/xiaomi/en/page_2.jpg) + +At that time, the first version of the growth analysis platform helped us solve a series of problems in the user operation process, but there were also two problems: + +### Problem No.1: Scattered components +Since the historical architecture is based on the combination of SparkSQL + Kudu + HDFS, too many dependent components lead to high operation and maintenance costs. The original design is that each component uses the resources of the public cluster, but in practice, it is found that during the execution of the query job, the query performance is easily affected by other jobs in the public cluster, and query jitter is prone to occur, especially when reading data from the HDFS public cluster , sometimes slower. + +### Problem No.2: High resource consumption +When querying through SparkSQL, the latency is relatively high. SparkSQL is a query engine designed based on a batch processing system. In the process of exchanging data shuffle between each stage, it still needs to be placed on the disk, and the delay in completing the SQL query is relatively high. In order to ensure that SQL queries are not affected by resources, we ensure query performance by adding machines. However, in practice, we find that there is limited room for performance improvement. This solution cannot make full use of machine resources to achieve efficient queries. A certain waste of resources. + +In response to the above two problems, our goal is to seek an MPP database that integrates computing and storage to replace our current storage and computing layer components. After technical selection, we finally decided to use Apache Doris to replace the older generation of historical architecture. + +## New Choice +Popular MPP-based query engines such as Impala and Presto, can efficiently support SQL queries, but they still need to rely on Kudu, HDFS, Hive Metastore and other storage system, which increase the operation and maintenance costs. At the same time, due to the separation of storage and compute, the query engine cannot easily find the data changes in the storage layer, resulting in bad performance in detailed query optimization. If you want to cache at the SQL layer, you cannot guarantee that the query results are up-to-date. + +Apache Doris is a top-level project of the Apache Foundation. It is mainly positioned as a high-performance, real-time analytical database, and is mainly used to solve reports and multi-dimensional analysis. It integrates Google Mesa and Cloudera Impala technologies. We conducted an in-depth performance tests on Doris and communicated with the community many times. And finally, we determined to replace the previous computing and storage components with Doris. + +## New Architecture Based on Apache Doris +The new architecture obtains event tracking data from the data source. Then data is ingested into Apache Doris. Query results can be directly displayed in the applications. In this way, Doris has truly realized the unification of computing, storage, and resource management tools. + +![page_3](/images/xiaomi/en/page_3.jpg) + +We chose Doris because: +- Doris has excellent query performance and can meet our business needs. +- Doris supports standard SQL, and the learning cost is low. +- Doris does not depend on other external components and is easy to operate and maintain. +- The Apache Doris community is very active and friendly, crowded with contributors. It is easier for further versions upgrades and convenient for maintenance. + +## Query Performance Comparision between Apache Doris & Spark SQL +Note: The comparison is based on Apache Doris V0.13 + +![page_4](/images/xiaomi/en/page_4.jpg) + +We selected a business model with an average daily data volume of about 1 billion, and conducted performance tests on Doris in different scenarios, including 6 event analysis scenarios, 3 retention analysis scenarios, and 3 funnel analysis scenarios. After comparing it with the previous architecture(SparkSQL+Kudu+HDFS), we found out: +- In the event analysis scenario, the average query time was reduced by 85%. +- In the scenarios of retention analysis and funnel analysis, the average query time was reduced by 50%. + +# Real Practice +Below we will introduce our experience of data import, data query, A/B test in the business application of Apache Doris. + +## Data Import + +![page_5](/images/xiaomi/en/page_5.jpg) + +Xiaomi writes data into Doris mainly through Stream Load, Broker Load and a small amount of data by Insert. Usually data is generally ingested into the message queue first, which is divided into real-time and offline data. + +### How to write real-time data into Apache Doris: +After part of real-time data processed by Flink, they will be ingested into Doris through Flink-Doris-Connector provided by Apache Doris. The rest of the data is ingested through Spark Streaming. The bottom layer of these two writing approaches both rely on the Stream Load provided by Apache Doris. + +### How to write offline data into Apache Doris: +After offline data is partially ingested into Hive, they will be ingested into Doris through Xiaomi's data import tool. Users can directly submit Broker Load tasks to the Xiaomi's data import tool and import data directly into Doris, or import data through Spark SQL, which relies on the Spark-Doris-Connector provided by Apache Doris. Spark Doris Connector is actually the encapsulation of Stream Load. + +## Data Qurey + +![page_6](/images/xiaomi/en/page_6.jpg) + +Users can query after data import is done. Inside Xiaomi, we query through our data platform. Users can perform visual queries on Doris through Xiaomi's data platform, and conduct user behavior analysis and user portrait analysis. In order to help our teams conduct event analysis, retention analysis, funnel analysis, path analysis and other behavioral analysis, we have added corresponding UDF (User Defined Function) and UDAF (User Defined Aggregate Function) to Doris. + +In the upcoming version 1.2, Apache Doris adds the function of synchronizing metadata through external table, such as Hive/Hudi/Iceberg and Multi Catalog tool. External table query improves performance, and the ability to access external tables greatly increases ease of use. In the future, we will consider querying Hive and Iceberg data directly through Doris, which builds an architecture of datalake. + +## A/B Test +In real business, the A/B test is a method of comparing two versions of strategies against each other to determine which one performs better. A/B test is essentially an experiment where two or more variants of a page are shown to users at random, and statistical analysis. It is popular approach used to determine which variation performs better for a given conversion goal. Xiaomi's A/B test platform is an operation tool product that conducts the A/B test with experimental grouping, traffic splitting, and scientific evaluation to assist in decision making. Xiaomi's A/B test platform has several query applications: user deduplication, indicator summation, covariance calculation, etc. The query types will involve Count (distinct), Bitmap, Like, etc. + +Apache Doris also provides services to Xiaomi's A/B test platform. Everyday, Xiaomi's A/B test platform needs to process a temendous amount of data with billions of queries. That's why Xiaomi's A/B test platform is eager to improve the query performance. + +Apache Doris V1.1 released just in time and has fully supported vectorization in the processing and storage. Compared with the non-vectorized version, the query performance has been significantly improved. It is time to update Xiaomi's Doris cluster to the latest version. That's why we first launched the latest vectorized version of Doris on Xiaomi's A/B test platform. + +## Test before Launch +Note: The following tests are based on Apache Doris V1.1.2 + +We built a test cluster for Apache Doris V1.1.2, which is as big as that of the Xiaomi online Apache Doris V0.13 version, to test before the vectorization version goes online. The test is divided into two aspects: single SQL parrellel query test and batch SQL concurrent query test. + +The configurations of the two clusters are exactly the same, and the specific configuration information is as follows: +- Scale: 3 FEs + 89 virtual machines +- CPU: Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz 16 cores 32 threads × 2 +- Memory: 256GB +- Disk: 7.3TB × 12 HDD + +### Single SQL Parrellel Query Test +We choose 7 classic queries in the Xiaomi A/B test. For each query, we limited the time range to 1 day, 7 days, and 20 days for testing, where the daily partition data size is about 3.1 billion (the data volume is about 2 TB). The test results are shown in the figures: + +![page_7](/images/xiaomi/en/page_7.jpg) + +![page_8](/images/xiaomi/en/page_8.jpg) + +![page_9](/images/xiaomi/en/page_9.jpg) + +The Apache Doris V1.1.2 has at least 3~5 times performance improvement compared to the Xiaomi online Doris V0.13, which is remarkable. + +## Optimization +Note: The following tests are based on Apache Doris V1.1.2 + +Based on Xiaomi's A/B test business data, we tuned Apache Doris V1.1.2 and conducted concurrent query tests on the tuned Doris V1.1.2 and Xiaomi's online Doris V0.13. The test results are as follows. + +### Optimization in Test 1 +We choose user deduplication, index summation, and covariance calculation query(the total number of SQL is 3245) in the A/B test to conduct concurrent query tests on the two versions. The single-day partition data of the table is about 3.1 billion (the amount of data is about 2 TB) and the query will be based on the latest week's data. The test results are shown in the figures: + +![page_10](/images/xiaomi/en/page_10.jpg) + +Compared with Apache Doris V0.13, the overall average latency of Doris V1.1.2 is reduced by about 48%, and the P95 latency is reduced by about 49%. In this test, the query performance of Doris V1.1.2 was nearly doubled compared to Doris V0.13. + +### Optimization in Test 2 +We choose 7 A/B test reports to test the two versions. Each A/B test report is corresponded to two modules in Xiaomi A/B test platform and each module represents thousands of SQL query. Each report submits query tasks to the cluster where the two versions reside at the same concurrency. The test results are shown in the figure: + +![page_11](/images/xiaomi/en/page_11.jpg) + +Compared with Doris V0.13, Doris V1.1.2 reduces the overall average latency by around 52%. In the test, the query performance of Doris V1.1.2 version was more than 1 time higher than that of Doris V0.13. + +### Optimization in Test 3 +To verify the performance of the tuned Apache Doris V1.1.2 in other cases, we choose the Xiaomi user behavior analysis to conduct concurrent query performance tests of Doris V1.1.2 and Doris V0.13. We choose behavior analysis query for 4 days on October 24, 25, 26 and 27, 2022. The test results are shown in the figures: + +![page_12](/images/xiaomi/en/page_12.jpg) + +Compared with Doris V0.13, the overall average latency of Doris V1.1.2 has been reduced by about 77%, and the P95 latency has been reduced by about 83%. In this test, the query performance of Doris V1.1.2 version is 4~6 times higher than that of Doris V0.13. + +# Conclusion +Since we adopted Apache Doris in 2019, Apache Doris has currently served dozens of businesses and sub-brands within Xiaomi, with dozens of clusters and hundreds of nodes. It completes more than 10,000 user online analysis queries every day and is responsible for most of the online analysis in Xiaomi. + +After performance test and tuning, Apache Doris V1.1.2 has met the launch requirements of the Xiaomi A/B test platform and does well in query performance and stability. In some cases, it even exceeds our expectations, such as the overall average latency being reduced by about 77% in our tuned version. + +Meanwhile, some functions have in the above been released in Apache Doris V1.0 or V1.1, some PRs have been merged into the community Master Fork and should be released soon. Recently the activity of the community has been greatly enhanceed. We are glad to see that Apache Doris has become more and more mature, and stepped forward to an integrated datalake. We truly believe that in the future, more data analysis will be explored and realized within Apache Doris. + + +# Contact Us +Apache Doris Website：http://doris.apache.org + +Github Homepage：https://github.com/apache/doris + +Email to DEV：dev@doris.apache.org + diff --git a/i18n/zh-CN/docusaurus-plugin-content-blog/xiaomi_vector.md b/i18n/zh-CN/docusaurus-plugin-content-blog/xiaomi_vector.md new file mode 100644 index 0000000000000..381ed0e2f9fb4 --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-blog/xiaomi_vector.md @@ -0,0 +1,165 @@ +--- +{ + "title": "最佳实践: Apache Doris 在小米数据场景的应用实践与优化", + "language": "zh" +} +--- + + + +>导读：小米集团于 2019 年首次引入了 Apache Doris ，目前 Apache Doris 已经在小米内部数十个业务中得到广泛应用，并且在小米内部已经形成一套以 Apache Doris 为核心的数据生态。本篇文章转录自 Doris 社区线上 Meetup 主题演讲，旨在分享 Apache Doris 在小米数据场景的落地实践与优化实践。 + +>作者｜魏祚 小米 OLAP 引擎研发工程师 + +![kv](/images/xiaomi/zh/kv.jpg) + +# 关于小米 +小米公司（“小米”或“集团”；HKG：1810），一家消费电子和智能制造公司，其智能手机和智能硬件通过物联网 (IoT) 平台连接。 2021年，小米总收入达到人民币3283亿元（4722.3131.62亿美元），同比增长33.5%；调整后净利润为人民币 220 亿元（316,451.08 万美元），同比增长 69.5%。 + +因分析业务的增长，小米集团于 2019 年首次引入了 Apache Doris 。经过三年时间的发展，目前 Apache Doris 已经在广告投放、新零售、增长分析、数据看板、用户画像、天星数科、小米有品、等小米内部数十个业务和品牌中得到广泛应用，并且在小米内部已经围绕 Apache Doris 为核心建设了数据生态。 + +![1](/images/xiaomi/zh/1.jpg) + +当前 Apache Doris 在小米内部已经具有数十个集群、总体达到数百台 BE 节点的规模，其中单集群最大规模达到近百台节点，拥有数十个实时数据同步任务，每日单表最大增量 120 亿、支持 PB 级别存储，单集群每天可以支持 2W 次以上的多维分析查询。 + +# 架构演进 +小米引入 Apache Doris 的初衷是为了解决内部的用户行为分析中所遇到的问题。随着小米互联网业务的发展，利用用户行为数据进行增长分析的需求越来越强烈。如果每个业务产品线都自己搭建一套增长分析系统，不仅成本高昂，效率也不高。因此如果能有一款产品能够帮助他们不用关心底层的复杂技术细节，让相关业务人员能够专注于自己的技术工作，可以极大提高工作效率。所以，小米大数据和云平台联合开发了增长分析系统 Growing Analytics（下文中简称 GA)，旨在提供一个灵活的多维实时查询和分析平台，可以统一管理数据接入和查询方案，帮助业务线做好精细化运营。 + +## 历史架构 +增长分析平台立项于 2018 年年中，当时基于开发时间和成本，技术栈等因素的考虑，小米复用了现有各种大数据基础组件（HDFS, Kudu, SparkSQL 等），搭建了一套基于 Lamda 架构的增长分析查询系统。GA 系统初代版本的架构如下图所示，包含了以下几个方面： +- 数据源：数据源是前端的埋点数据以及用户行为数据。 +- 数据接入层：对埋点数据进行统一的清洗后写入小米内部自研的消息队列中，并通过 Spark Streaming 将数据导入Kudu 中。 +- 存储层：在存储层中进行冷热数据分离。热数据存放在 Kudu 中，冷数据则会存放在 HDFS 上。同时在存储层中进行分区，当分区单位为天时，每晚会将一部分数据转冷并存储到 HDFS 上。 +- 计算层/查询层：在查询层中，使用 SparkSQL 对 Kudu 与 HDFS 上数据进行联邦查询，最终把查询结果显示在前端页面。 + +![2](/images/xiaomi/zh/2.jpg) + +在当时的历史背景下，初代版本的增长分析平台帮助我们解决了一系列用户运营过程中的问题，但同时在历史架构中也存在了两个问题： + +### 第一个问题：组件分散 +由于历史架构是基于 SparkSQL + Kudu + HDFS 的组合，依赖的组件过多导致运维成本较高。原本的设计是各个组件都使用公共集群的资源，但是实践过程中发现执行查询作业的过程中，查询性能容易受到公共集群其他作业的影响，容易发生查询抖动，尤其在读取 HDFS 公共集群的数据时，有时较为缓慢。 + +### 第二个问题：资源占用高 +通过 SparkSQL 进行查询时，延迟相对较高。SparkSQL 是基于批处理系统设计的查询引擎，在每个 Stage 之间交换数据 Shuffle 的过程中依然需要进行落盘，完成 SQL 查询的时延较高。为了保证 SQL 查询不受资源的影响，我们通过添加机器来保证查询性能，但是实践过程中发现，性能提升的空间有限，这套解决方案并不能充分地利用机器资源来达到高效查询的目的，存在一定的资源浪费。 + +针对上述两个问题，我们的目标是寻求一款计算、存储一体的 MPP 数据库来替代我们目前的存储计算层的组件，在通过技术选型后，最终我们决定使用 Apache Doris 替换老一代历史架构。 + +## 重新选型 + +MPP架构的查询引擎，如Impala,Presto等能够高效地支持SQL查询，但是仍然需要依赖Kudu, HDFS, Hive Metastore等组件, 运维成本依然比较高。同时，由于计算存储分离，查询引擎不能很好地及时感知存储层的数据变化，就无法做更细致的查询优化。如想在SQL层做缓存就无法保证查询的结果是最新的。 + +Doris是Apache基金会顶级项目，主要定位是高性能的、支持实时的分析型数据库， 主要用于解决报表和多维分析。它主要集成了 Google Mesa 和 Cloudera Impala 技术。我们对Doris进行了内部的性能测试并多次和社区沟通交流，确定了用Doris替换原来的计算存储组件的解决方案。我们新的架构如下图所示： + +## 基于 Apache Doris 的新版架构 +新版架构从数据源获取埋点数据后，数据接入后写入 Apache Doris 后可以直接查询结果并在前端进行显示。真正实现了通过Doris统一了计算、存储，和资源管理yarn相关工具。 + +![3](/images/xiaomi/zh/3.jpg) + +我们选择 Doris 原因： +- Doris 具有优秀的查询性能，能够满足业务需求。 +- Doris 支持标准 SQL ，用户使用与学习成本较低。 +- Doris 不依赖于其他的外部系统，运维简单。 +- Doris 社区拥有很高活跃度，版本迭代快。开发者规模大，有利于后续系统的维护升级。 + +## 新旧架构性能对比 + +![4](/images/xiaomi/zh/4.jpg) + +我们选取了日均数据量大约 10 亿的业务，分别在不同场景下对Doris进行了性能测试，其中包含 6 个事件分析场景，3 个留存分析场景以及 3 个漏斗分析场景。经过与【SparkSQL+Kudu+HDFS】的旧方案对比后，我们发现： +- 在事件分析的场景下，平均查询所耗时间降低了 85%。 +- 在留存分析和漏斗分析场景下，平均查询所耗时间降低了 50%。 + +# 应用实践 +下面将介绍我们在Apache Doris应用中数据导入、数据查询、A/B测试的经验。 + +## 数据导入 + +![5](/images/xiaomi/zh/5.jpg) + +小米内部主要通过 Stream Load 与 Broker Load 以及少量 Insert 方式来导入数据到Doris。数据一般会先写入到消息队列中，分为实时数据和离线数据两个部分。 +实时数据如何写入到Apache Doris 中：一部分实时数据通过 Flink数据处理 后， 并通过 Doris 社区提供的 Flink Doris Connector 组件写入到 Doris 中。另一部分数据通过 Spark Streaming 组件写入。这两种写入方式的底层都依赖的是社区提供的 Stream Load。 +离线数据如何写入到Apache Doris 中：离线数据部分写入 Hive 后，通过小米的数据工场将数据导入到 Doris 中。用户可以直接在数据工场提交 Broker Load 任务并将数据直接导入 Doris 中，也可以通过 Spark SQL 将数据导入 。Spark SQL 方式则是依赖了 Doris 社区提供的 Spark Doris Connector 组件，其底层为 Doris 的 Stream Load 的封装。 + +## 数据查询 + +![6](/images/xiaomi/zh/6.jpg) + +用户通过数据工场将数据导入至 Doris 后即可进行查询。在小米内部，可以通过自研的数鲸平台进行查询的。用户可以通过数鲸平台对 Doris 进行可视化的查询，并展开用户行为分析和用户画像分析。其中，为帮助业务进行事件分析、留存分析、漏斗分析、路径分析等行为分析，我们为 Doris 添加了相应的 UDF （User Defined Function）和 UDAF (User Defined Aggregate Function)。 +在即将发布的 1.2 版本中，Doris添加了外表元数据同步的功能，支持 Hive/Hudi/Iceberg 外表并增加了 Multi Catalog。查询外部表提升了性能，接入外表大幅增加了易用性。在未来，我们考虑直接通过 Doris 查询 Hive 与 Iceberg 数据，构建湖仓一体的架构。 + +## A/B测试 +小米的 A/B 实验平台对 Apache Doris 查询性能的提升有着迫切的需求，因此我们选择优先在小米的 A/B 实验平台上线 Apache Doris 向量化版本，也就是 1.1.2 版本。 + +小米的 A/B 实验平台是一款通过 A/B 测试的方式，借助实验分组、流量拆分与科学评估等手段来辅助完成科学的业务决策，最终实现业务增长的一款运营工具产品。在实际业务中，为了验证一个新策略的效果，通常需要准备原策略 A 和新策略 B 两种方案。随后在总体用户中取出一小部分，将这部分用户完全随机地分在两个组中，使两组用户在统计角度无差别。将原策略 A 和新策略 B 分别展示给不同的用户组，一段时间后，结合统计方法分析数据，得到两种策略生效后指标的变化结果，并以此来判断新策略 B 是否符合预期。 + +小米的 A/B 实验平台有几类典型的查询应用：用户去重、指标求和、实验协方差计算等，查询类型会涉及较多的 Count(distinct)、Bitmap计算、Like语句等。 + +### 上线前验证 +我们基于 Apache Doris 1.1.2 版本搭建了一个和小米线上 Apache Doris 0.13 版本在机器配置和机器规模上完全相同的测试集群，用于向量化版本上线前的验证。验证测试分为两个方面：单 SQL 串行查询测试和批量 SQL 并发查询测试。在这两种测试中，我们在保证两个集群数据完全相同的条件下，分别在 Doris 1.1.2 测试集群和小米线上 Doris 0.13 集群执行相同的查询 SQL 来做性能对比。我们的目标是，Doris 1.1.2 版本在小米线上 Doris 0.13 版本的基础上有 1 倍的查询性能提升。 +两个集群配置完全相同，具体配置信息如下： +- 集群规模：3 FE + 89 BE +- BE节点CPU: Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz 16核 32线程 × 2 +- BE节点内存：256GB +- BE节点磁盘：7.3TB × 12 HDD + +#### 单 SQL 串行查询测试 +在该测试场景中，我们选取了小米A/B 实验场景中 7 个典型的查询 Case，针对每一个查询 Case，我们将扫描的数据时间范围分别限制为 1 天、7 天和 20 天进行查询测试，其中单日分区数据量级大约为 31 亿（数据量大约 2 TB），测试结果如图所示： + +![7](/images/xiaomi/zh/7.jpg) + +![8](/images/xiaomi/zh/8.jpg) + +![9](/images/xiaomi/zh/9.jpg) + +根据以上小米 A/B 实验场景下的单 SQL 串行查询测试结果所示，Doris 1.1.2 版本相比小米线上 Doris 0.13 版本至少有 3~5 倍的性能提升，效果显著。 + +### 调优测试结果 +我们基于小米的 A/B实验场景对 Apache Doris 1.1.2 版本进行了一系列调优，并将调优后的 Doris 1.1.2 版本与小米线上 Doris 0.13 版本分别进行了并发查询测试。测试情况如下： + +#### 测试 1 +我们选择了 A/B 实验场景中一批典型的用户去重、指标求和以及协方差计算的查询 Case（SQL 总数量为 3245）对两个版本进行并发查询测试，测试表的单日分区数据大约为 31 亿（数据量大约 2 TB），查询的数据范围会覆盖最近一周的分区。测试结果如图所示，Doris 1.1.2 版本相比 Doris0.13版本，总体的平均延迟降低了大约 48%，P95 延迟降低了大约 49%。在该测试中，Doris 1.1.2 版本相比 Doris0.13 版本的查询性能提升了接近 1 倍。 + +![10](/images/xiaomi/zh/10.jpg) + +#### 测试 2 +我们选择了 A/B实验场景下的 7 份 A/B 实验报告对两个版本进行测试，每份 A/B 实验报告对应小米 A/B实验平台页面的两个模块，每个模块对应数百或数千条查询 SQL。每一份实验报告都以相同的并发向两个版本所在的集群提交查询任务。测试结果如图所示，Doris 1.1.2 版本相比 Doris 0.13 版本，总体的平均延迟降低了大约 52%。在该测试中，Doris 1.1.2 版本相比 Doris 0.13 版本的查询性能提升了超过 1 倍。 + +![11](/images/xiaomi/zh/11.jpg) + +#### 测试 3 +为了验证调优后的 Apache Doris 1.1.2 版本在小米 A/B 实验场景之外的性能表现，我们选取了小米用户行为分析场景进行了 Doris 1.1.2 版本和 Doris 0.13 版本的并发查询性能测试。我们选取了 2022年10月24日、25日、26日和 27日这 4 天的小米线上真实的行为分析查询 Case 进行对比查询，测试结果如图所示，Doris 1.1.2 版本相比 Doris 0.13 版本，总体的平均延迟降低了大约7 7%，P95 延迟降低了大约 83%。在该测试中，Doris 1.1.2 版本相比 Doris 0.13 版本的查询性能有 4~6 倍的提升。 + +![12](/images/xiaomi/zh/12.jpg) + +# 总结 +自从 Apache Doris 从 2019 年上线第一个业务至今，目前 Apache Doris 已经在小米内部服务了数十个业务及子品牌、集群数量达到数十个、节点规模达到数百台。每天完成数万次用户在线分析查询，承担了包括增长分析和报表查询等绝大多数在线分析的需求。 + +经过一个多月的性能调优和测试，Apache Doris 1.1.2 版本在查询性能和稳定性方面已经达到了小米 A/B实验平台的上线要求，在某些场景下的查询性能甚至超过了我们的预期，希望本次分享可以给有需要的朋友一些可借鉴的经验参考。 + +与此同时，在以上小米的实践中，已有部分功能在 Apache Doris 1.0 或 1.1 版本中发布，部分 PR 已经合入社区 Master，将在不久后发布的 1.2 新版本中与大家见面。随着社区的快速发展，有越来越多小伙伴参与到社区建设中，社区活跃度有了极大的提升。Apache Doris 已经变得越来越成熟，并开始从单一计算存储一体的分析型 MPP 数据库走向湖仓一体的道路，相信在未来，还会有更多的数据分析场景被探索和实现。 + + +# 联系我们 +官网：http://doris.apache.org + +Github：https://github.com/apache/doris + +dev邮件组：dev@doris.apache.org + diff --git a/static/images/xiaomi/en/kv.png b/static/images/xiaomi/en/kv.png new file mode 100644 index 0000000000000..e67753693476b Binary files /dev/null and b/static/images/xiaomi/en/kv.png differ diff --git a/static/images/xiaomi/en/page_1.jpg b/static/images/xiaomi/en/page_1.jpg new file mode 100644 index 0000000000000..49d5d75bd712d Binary files /dev/null and b/static/images/xiaomi/en/page_1.jpg differ diff --git a/static/images/xiaomi/en/page_10.jpg b/static/images/xiaomi/en/page_10.jpg new file mode 100644 index 0000000000000..988327264b3cf Binary files /dev/null and b/static/images/xiaomi/en/page_10.jpg differ diff --git a/static/images/xiaomi/en/page_11.jpg b/static/images/xiaomi/en/page_11.jpg new file mode 100644 index 0000000000000..f4f5256b120ef Binary files /dev/null and b/static/images/xiaomi/en/page_11.jpg differ diff --git a/static/images/xiaomi/en/page_12.jpg b/static/images/xiaomi/en/page_12.jpg new file mode 100644 index 0000000000000..97381a4526254 Binary files /dev/null and b/static/images/xiaomi/en/page_12.jpg differ diff --git a/static/images/xiaomi/en/page_2.jpg b/static/images/xiaomi/en/page_2.jpg new file mode 100644 index 0000000000000..1a545f45383a9 Binary files /dev/null and b/static/images/xiaomi/en/page_2.jpg differ diff --git a/static/images/xiaomi/en/page_3.jpg b/static/images/xiaomi/en/page_3.jpg new file mode 100644 index 0000000000000..b5927449226e0 Binary files /dev/null and b/static/images/xiaomi/en/page_3.jpg differ diff --git a/static/images/xiaomi/en/page_4.jpg b/static/images/xiaomi/en/page_4.jpg new file mode 100644 index 0000000000000..15d3f267c58d8 Binary files /dev/null and b/static/images/xiaomi/en/page_4.jpg differ diff --git a/static/images/xiaomi/en/page_5.jpg b/static/images/xiaomi/en/page_5.jpg new file mode 100644 index 0000000000000..2807028871cf8 Binary files /dev/null and b/static/images/xiaomi/en/page_5.jpg differ diff --git a/static/images/xiaomi/en/page_6.jpg b/static/images/xiaomi/en/page_6.jpg new file mode 100644 index 0000000000000..aee12017b0a20 Binary files /dev/null and b/static/images/xiaomi/en/page_6.jpg differ diff --git a/static/images/xiaomi/en/page_7.jpg b/static/images/xiaomi/en/page_7.jpg new file mode 100644 index 0000000000000..4585bea1c1a54 Binary files /dev/null and b/static/images/xiaomi/en/page_7.jpg differ diff --git a/static/images/xiaomi/en/page_8.jpg b/static/images/xiaomi/en/page_8.jpg new file mode 100644 index 0000000000000..f886c98d99613 Binary files /dev/null and b/static/images/xiaomi/en/page_8.jpg differ diff --git a/static/images/xiaomi/en/page_9.jpg b/static/images/xiaomi/en/page_9.jpg new file mode 100644 index 0000000000000..6434b6b8f5b53 Binary files /dev/null and b/static/images/xiaomi/en/page_9.jpg differ diff --git a/static/images/xiaomi/en/test b/static/images/xiaomi/en/test new file mode 100644 index 0000000000000..8b137891791fe --- /dev/null +++ b/static/images/xiaomi/en/test @@ -0,0 +1 @@ + diff --git a/static/images/xiaomi/zh/1.png b/static/images/xiaomi/zh/1.png new file mode 100644 index 0000000000000..46296055a8ffa Binary files /dev/null and b/static/images/xiaomi/zh/1.png differ diff --git a/static/images/xiaomi/zh/10.png b/static/images/xiaomi/zh/10.png new file mode 100644 index 0000000000000..4844293d331e5 Binary files /dev/null and b/static/images/xiaomi/zh/10.png differ diff --git a/static/images/xiaomi/zh/11.png b/static/images/xiaomi/zh/11.png new file mode 100644 index 0000000000000..6ec1f3cdf704c Binary files /dev/null and b/static/images/xiaomi/zh/11.png differ diff --git a/static/images/xiaomi/zh/12.png b/static/images/xiaomi/zh/12.png new file mode 100644 index 0000000000000..16c9b5edb3c55 Binary files /dev/null and b/static/images/xiaomi/zh/12.png differ diff --git a/static/images/xiaomi/zh/2.png b/static/images/xiaomi/zh/2.png new file mode 100644 index 0000000000000..eaa2123583e9e Binary files /dev/null and b/static/images/xiaomi/zh/2.png differ diff --git a/static/images/xiaomi/zh/3.png b/static/images/xiaomi/zh/3.png new file mode 100644 index 0000000000000..6cd9e232bd148 Binary files /dev/null and b/static/images/xiaomi/zh/3.png differ diff --git a/static/images/xiaomi/zh/4.png b/static/images/xiaomi/zh/4.png new file mode 100644 index 0000000000000..42f0642f4e58d Binary files /dev/null and b/static/images/xiaomi/zh/4.png differ diff --git a/static/images/xiaomi/zh/5.png b/static/images/xiaomi/zh/5.png new file mode 100644 index 0000000000000..d0eb413a5a47a Binary files /dev/null and b/static/images/xiaomi/zh/5.png differ diff --git a/static/images/xiaomi/zh/6.png b/static/images/xiaomi/zh/6.png new file mode 100644 index 0000000000000..6ccea54e2c892 Binary files /dev/null and b/static/images/xiaomi/zh/6.png differ diff --git a/static/images/xiaomi/zh/7.png b/static/images/xiaomi/zh/7.png new file mode 100644 index 0000000000000..5eeb11db8aa22 Binary files /dev/null and b/static/images/xiaomi/zh/7.png differ diff --git a/static/images/xiaomi/zh/8.png b/static/images/xiaomi/zh/8.png new file mode 100644 index 0000000000000..f46aad2a333be Binary files /dev/null and b/static/images/xiaomi/zh/8.png differ diff --git a/static/images/xiaomi/zh/9.png b/static/images/xiaomi/zh/9.png new file mode 100644 index 0000000000000..18c411519146f Binary files /dev/null and b/static/images/xiaomi/zh/9.png differ diff --git a/static/images/xiaomi/zh/kv.png b/static/images/xiaomi/zh/kv.png new file mode 100644 index 0000000000000..e67753693476b Binary files /dev/null and b/static/images/xiaomi/zh/kv.png differ diff --git a/static/images/xiaomi/zh/test b/static/images/xiaomi/zh/test new file mode 100644 index 0000000000000..8b137891791fe --- /dev/null +++ b/static/images/xiaomi/zh/test @@ -0,0 +1 @@ +