Add a builder to SimplifyContext to avoid allocating default values#21092
Open
AdamGS wants to merge 4 commits intoapache:mainfrom
Open
Add a builder to SimplifyContext to avoid allocating default values#21092AdamGS wants to merge 4 commits intoapache:mainfrom
SimplifyContext to avoid allocating default values#21092AdamGS wants to merge 4 commits intoapache:mainfrom
Conversation
Signed-off-by: Adam Gutglick <adamgsal@gmail.com>
github-actions
bot
added
logical-expr
Contributor
|
Run benchmark sql_planner |
|
🤖 Criterion benchmark running (GKE) | trigger |
|
Benchmark for this request failed. Last 20 lines of output: Click to expand |
Contributor
Author
|
@Dandandan should I rebase this on top of #21084? |
alamb
approved these changes
Mar 21, 2026
Contributor
alamb
left a comment
alamb
left a comment
There was a problem hiding this comment.
I think this is an API improvement even if it doesn't measurably improve the performance -- thank you @AdamGS
One thing I thought of that might be less invasive (as in require less code changes) would be to use a single global lazy lock for default config options. Something like
/// Global default configuration options
static DEFAULT_CONFIG: LazyLock<Arc<ConfigOptions>> = LazyLock::new(|| Arc::new(ConfigOptions::new()));
impl ConfigOptions {
/// Creates a new [`ConfigOptions`] with default values
pub fn new() -> Self {
Self::default()
}
/// Return a reference to the default configuration options
pub fn default_arc() -> Arc<Self> {
Arc::clone(&DEFAULT_CONFIG)
}Let me see if I can make a PR that does this
| let state = self.state.read(); | ||
| let context = SimplifyContext::default() | ||
| let context = SimplifyContext::builder() | ||
| .with_schema(Arc::clone(prepared.plan.schema())) |
Contributor
There was a problem hiding this comment.
I recommend we also deprecate SimplifyContext::with... methods (can do it as a follow on PR) and direct people to use the builder in their own code.
Contributor
Author
There was a problem hiding this comment.
That makes perfect sense, I'll add it in this one
Contributor
I made I have queued up some benchmark runs (sql_planner) for these branches -- let's see what they say (should be done in an hour or two) |
Contributor
Author
|
What should the edit: FWIW - I'll happily backport this to 53.1.0 if necessary |
Signed-off-by: Adam Gutglick <adamgsal@gmail.com>
AdamGS
force-pushed
the
adamd/simplify-ctx-builder
branch
from
0de841f to
8d0c0d5
Compare
Contributor
54 for now (and we can change it to 53 if we choose to backport) |
Signed-off-by: Adam Gutglick <adamgsal@gmail.com>
|
🤖 |
|
🤖: Benchmark completed Details
|
Contributor
Woah, is this right? |
Contributor
|
run benchmarks |
|
🤖 Benchmark completed (GKE) | trigger Details
Resource Usagetpch — base (merge-base)
tpch — branch
|
|
🤖 Benchmark completed (GKE) | trigger Details
Resource Usagetpcds — base (merge-base)
tpcds — branch
|
|
🤖 Benchmark completed (GKE) | trigger Details
Resource Usageclickbench_partitioned — base (merge-base)
clickbench_partitioned — branch
|
Contributor
I bet it is less of a win after #21084 is merged. I'll update and rerun |
|
🤖 |
|
🤖: Benchmark completed Details
|
Contributor
|
Seems like still a small win |
|
🤖 |
|
🤖: Benchmark completed Details
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
This is a follow up to #21084, where @blaginin realized that allocating
ConfigOptionshas this surprising side effect. Reading through how its used I realized that on mode "real" code paths (and in many tests), DataFusion ends up allocating the default values ofSimplifyContextjust to immediately drop them and override them with pre-existing clone-able data.What changes are included in this PR?
Adds a new type
SimplifyContextBuilderandSimplifyContext::builderAre these changes tested?
Includes a couple of tests to make sure the builder makes sense, in addition to many existing tests.
Are there any user-facing changes?
As noted above, new type and a new function on an existing type.