From ee7d0bb3cb44912b5352aa8fee9a1e3c72d77507 Mon Sep 17 00:00:00 2001 From: Jatin Kumar Singh Date: Thu, 27 Nov 2025 09:54:29 +0530 Subject: [PATCH] Add PGO documentation section to crate configuration Add section explaining Profile Guided Optimization can provide up to 25% performance improvements. Includes build process instructions and tips for effective PGO usage. References issue #9507. --- docs/source/user-guide/crate-configuration.md | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/source/user-guide/crate-configuration.md b/docs/source/user-guide/crate-configuration.md index eecf7f5bde6e..83a46b50c004 100644 --- a/docs/source/user-guide/crate-configuration.md +++ b/docs/source/user-guide/crate-configuration.md @@ -92,6 +92,36 @@ lto = true codegen-units = 1 ``` +### Profile Guided Optimization (PGO) + +Profile Guided Optimization can improve DataFusion performance by up to 25%. It works by compiling with instrumentation, running representative workloads to collect profile data, then recompiling with optimizations based on that data. + +Build with instrumentation: + +```shell +RUSTFLAGS="-C profile-generate=/tmp/pgo-data" cargo build --release +``` + +Run your workloads to collect profile data. Use benchmarks like TPCH or Clickbench, or your actual production queries: + +```shell +./target/release/your-datafusion-app --benchmark +``` + +Rebuild using the collected profile: + +```shell +RUSTFLAGS="-C profile-use=/tmp/pgo-data" cargo build --release +``` + +Tips: + +- Use workloads that match your production patterns +- Run multiple iterations during profiling for better coverage +- Combine with LTO and CPU-specific optimizations for best results + +See the [Rust compiler guide](https://rustc-dev-guide.rust-lang.org/building/optimized-build.html#profile-guided-optimization) for more details. Discussion and results in [issue #9507](https://github.com/apache/datafusion/issues/9507). + ### Alternate Allocator: `snmalloc` You can also use [snmalloc-rs](https://crates.io/crates/snmalloc-rs) crate as