Skip to content

[improvement](build) Optimize FE incremental build packaging #62915

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
suxiaogang223 merged 10 commits into from
May 7, 2026
Merged

[improvement](build) Optimize FE incremental build packaging #62915

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
a70820e
[doc](build) Document FE build optimization target
suxiaogang223 Apr 26, 2026
caeab42
[improvement](build) Optimize FE runtime lib packaging
suxiaogang223 Apr 26, 2026
ef46fc4
[doc](build) Align FE build optimization document
suxiaogang223 Apr 28, 2026
33a9412
[improvement](build) Use package for FE output build
suxiaogang223 Apr 28, 2026
11d3f16
[improvement](build) Preserve FE release source artifact scope
suxiaogang223 Apr 28, 2026
7807b56
[improvement](build) Align FE build cache schema version
suxiaogang223 Apr 28, 2026
2331641
[doc](build) Remove completed FE build optimization plan
suxiaogang223 Apr 28, 2026
5cd6d37
[improvement](build) Avoid FE lib zip recompression
suxiaogang223 Apr 28, 2026
acd9b51
[fix](build) Enable coverage profile for FE UT
suxiaogang223 Apr 28, 2026
712a3ad
[fix](build) Flatten FE common for CDC client build
suxiaogang223 Apr 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -816,7 +816,7 @@ function build_fe_modules() {
local -a user_settings_opts=()
local -a mvn_cmd=(
"${MVN_CMD}"
install
package
-pl
"${FE_MODULES}"
-am
Expand Down Expand Up @@ -892,7 +892,7 @@ if [[ "${BUILD_FE}" -eq 1 ]]; then
cp -r -p "${DORIS_HOME}/conf/ldap.conf" "${DORIS_OUTPUT}/fe/conf"/
cp -r -p "${DORIS_HOME}/conf/mysql_ssl_default_certificate" "${DORIS_OUTPUT}/fe/"/
rm -rf "${DORIS_OUTPUT}/fe/lib"/*
cp -r -p "${DORIS_HOME}/fe/fe-core/target/lib"/* "${DORIS_OUTPUT}/fe/lib"/
unzip -q -o "${DORIS_HOME}/fe/fe-core/target/doris-fe-lib.zip" -d "${DORIS_OUTPUT}/fe/lib"
cp -r -p "${DORIS_HOME}/fe/fe-core/target/doris-fe.jar" "${DORIS_OUTPUT}/fe/lib"/
if [[ "${WITH_TDE_DIR}" != "" ]]; then
cp -r -p "${DORIS_HOME}/fe/fe-${WITH_TDE_DIR}/target/fe-${WITH_TDE_DIR}-1.2-SNAPSHOT.jar" "${DORIS_OUTPUT}/fe/lib"/
Expand Down
18 changes: 3 additions & 15 deletions fe/.mvn/maven-build-cache-config.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<cache xmlns="http://maven.apache.org/BUILD-CACHE-CONFIG/1.0.0"
<cache xmlns="http://maven.apache.org/BUILD-CACHE-CONFIG/1.2.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/BUILD-CACHE-CONFIG/1.0.0
https://maven.apache.org/xsd/build-cache-config-1.0.0.xsd">
xsi:schemaLocation="http://maven.apache.org/BUILD-CACHE-CONFIG/1.2.0
https://maven.apache.org/xsd/build-cache-config-1.2.0.xsd">
<configuration>
<enabled>true</enabled>
<hashAlgorithm>XX</hashAlgorithm>
Expand Down Expand Up @@ -70,18 +70,6 @@ under the License.
<execId>validate</execId>
</execIds>
</execution>
<!-- dependency:copy-dependencies writes JARs to target/lib/ which the build
scripts later copy into the output directory. The cache extension only
restores the primary JAR artifact on a hit, not the target/lib/ tree.
When combined with a clean build, target/lib/ is
deleted by the clean phase and never recreated, causing "cp: cannot stat
target/lib/*" failures. Force this goal to always run so that the lib
directory is always populated regardless of cache state. -->
<execution artifactId="maven-dependency-plugin">
<execIds>
<execId>copy-dependencies</execId>
</execIds>
</execution>
</executions>
</runAlways>
</executionControl>
Expand Down
42 changes: 25 additions & 17 deletions fe/fe-catalog/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,23 +91,6 @@ under the License.
<libDirectory>src/main/antlr4/org/apache/doris/parser</libDirectory>
</configuration>
</plugin>
<!-- Build source jar -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<configuration>
<attach>true</attach>
</configuration>
<executions>
<execution>
<id>create-source-jar</id>
<goals>
<goal>jar-no-fork</goal>
<goal>test-jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- Build test-jar for projects that may depend on test utilities -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
Expand All @@ -131,4 +114,29 @@ under the License.
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>release</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<configuration>
<attach>true</attach>
</configuration>
<executions>
<execution>
<id>create-source-jar</id>
<goals>
<goal>jar-no-fork</goal>
<goal>test-jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
39 changes: 23 additions & 16 deletions fe/fe-common/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,29 @@ under the License.
<doris.thirdparty>${env.DORIS_THIRDPARTY}</doris.thirdparty>
</properties>
</profile>
<profile>
<id>release</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<configuration>
<attach>true</attach>
</configuration>
<executions>
<execution>
<id>create-source-jar</id>
<goals>
<goal>jar-no-fork</goal>
<goal>test-jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
<dependencies>
<dependency>
Expand Down Expand Up @@ -202,22 +225,6 @@ under the License.
<libDirectory>src/main/antlr4/org/apache/doris/parser</libDirectory>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<configuration>
<attach>true</attach>
</configuration>
<executions>
<execution>
<id>create-source-jar</id>
<goals>
<goal>jar-no-fork</goal>
<goal>test-jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- Build test-jar's for all projects, since some projects depend on tests from others -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
Expand Down
50 changes: 11 additions & 39 deletions fe/fe-core/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -930,25 +930,24 @@ under the License.
</execution>
</executions>
</plugin>
<!-- copy all dependency libs to target lib dir -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<appendAssemblyId>true</appendAssemblyId>
<outputDirectory>${project.build.directory}</outputDirectory>
<recompressZippedFiles>false</recompressZippedFiles>
<descriptors>
<descriptor>src/main/assembly/fe-lib.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<id>copy-dependencies</id>
<id>make-fe-lib</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
<goal>single</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
<overWriteIfNewer>true</overWriteIfNewer>
<includeScope>runtime</includeScope>
<skip>${skip.plugin}</skip>
</configuration>
</execution>
</executions>
</plugin>
Expand All @@ -962,33 +961,6 @@ under the License.
</check>
</configuration>
</plugin>
<!-- clean fe/target dir before building -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-clean-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<skip>${skip.clean}</skip>
<excludeDefaultDirectories>true</excludeDefaultDirectories>
<filesets>
<fileset>
<directory>target</directory>
<excludes>
<exclude>checkstyle-cachefile</exclude>
</excludes>
</fileset>
</filesets>
</configuration>
<executions>
<execution>
<id>auto-clean</id>
<phase>initialize</phase>
<goals>
<goal>clean</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<extensions>
<extension>
Expand Down
36 changes: 36 additions & 0 deletions fe/fe-core/src/main/assembly/fe-lib.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.2.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.2.0 https://maven.apache.org/xsd/assembly-2.2.0.xsd">
<id>lib</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<scope>runtime</scope>
<useProjectArtifact>false</useProjectArtifact>
<unpack>false</unpack>
</dependencySet>
</dependencySets>
</assembly>
41 changes: 25 additions & 16 deletions fe/fe-foundation/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,22 +47,6 @@ under the License.
<finalName>doris-fe-foundation</finalName>
<directory>${project.basedir}/target/</directory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<configuration>
<attach>true</attach>
</configuration>
<executions>
<execution>
<id>create-source-jar</id>
<goals>
<goal>jar-no-fork</goal>
<goal>test-jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
Expand All @@ -85,4 +69,29 @@ under the License.
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>release</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<configuration>
<attach>true</attach>
</configuration>
<executions>
<execution>
<id>create-source-jar</id>
<goals>
<goal>jar-no-fork</goal>
<goal>test-jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
38 changes: 22 additions & 16 deletions fe/fe-grpc/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,28 @@ under the License.
<grpc.java.artifact>io.grpc:protoc-gen-grpc-java:${grpc-java.version}:exe:osx-x86_64</grpc.java.artifact>
</properties>
</profile>
<profile>
<id>release</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<configuration>
<attach>true</attach>
</configuration>
<executions>
<execution>
<id>create-source-jar</id>
<goals>
<goal>jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

<dependencies>
Expand Down Expand Up @@ -89,22 +111,6 @@ under the License.
<finalName>doris-fe-grpc</finalName>
<directory>${project.basedir}/target/</directory>
<plugins>
<!-- Build source jar -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<configuration>
<attach>true</attach>
</configuration>
<executions>
<execution>
<id>create-source-jar</id>
<goals>
<goal>jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- protobuf -->
<plugin>
<groupId>com.github.os72</groupId>
Expand Down
Loading
Loading