Skip to content

[chore](conf) Remove deprecated/unused FE/BE configs and stale session variables#63744

Open
jacktengg wants to merge 1 commit into
apache:masterfrom
jacktengg:wt-conf-var-cleanup
Open

[chore](conf) Remove deprecated/unused FE/BE configs and stale session variables#63744
jacktengg wants to merge 1 commit into
apache:masterfrom
jacktengg:wt-conf-var-cleanup

Conversation

@jacktengg
Copy link
Copy Markdown
Contributor

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Doris has accumulated a number of FE configs, BE configs and session variables that are either explicitly marked @Deprecated, or simply have no remaining references anywhere in the codebase (production sources, regression tests, docs, or tools). They add noise to the user-facing configuration surface and to the source tree without changing any behavior. This commit removes them in three coordinated parts and adds the minimum amount of backward-compatibility plumbing required to keep a BE-then-FE rolling upgrade safe.

  1. FE configs (fe/fe-common/.../Config.java)

    Remove 13 @Deprecated fields with no remaining production references: meta_publish_timeout_ms, backup_plugin_path, max_unfinished_load_job, use_new_tablet_scheduler, enable_concurrent_update, cbo_max_statistics_job_num, max_cbo_statistics_task_timeout_sec, cbo_concurrency_statistics_task_num, cbo_default_sample_percentage, finish_job_max_saved_second, enable_array_type, period_analyze_simultaneously_running_task_num, maximum_parallelism_of_export_job.

    ConfigBase.java gains a small warnDeprecatedKeys() helper. After setFields() runs, if any of the legacy keys above is still listed in fe.conf, FE logs a single WARN line so operators notice the stale entry on upgrade. FE startup is not affected: the FE config loader iterates over Java fields, so unknown keys in fe.conf were already silently ignored before this change.

  2. BE configs (be/src/common/config.{cpp,h}, be/src/cloud/config.{cpp,h})

    Remove 24 configs that have zero references in the BE source tree: doris_scanner_max_run_time_ms, multi_get_max_threads, num_disks, read_size, load_max_wg_active_memtable_percent, jdbc_connection_pool_cache_clear_time_sec, big_column_size_buffer, small_column_size_buffer, max_fragment_start_wait_time_seconds, enable_workload_group_for_scan, workload_group_scan_task_wait_timeout_ms, variant_use_cloud_schema_dict_cache, variant_threshold_rows_to_estimate_sparse_column, variant_nested_group_max_depth, enable_ttl_cache_evict_using_lru, file_cache_background_ttl_gc_batch, kerberos_refresh_interval_second, enable_debug_log_timeout_secs, enable_index_compaction, schema_dict_cache_capacity, test_s3_resource, meta_service_use_load_balancer, meta_service_rpc_timeout_ms, delete_bitmap_rpc_retry_times.

    The BE config loader (be/src/common/config.cpp init()) also iterates over the registered field map, so unknown keys in be.conf were already silently ignored — hard removal is safe on a rolling upgrade. None of the removed options affect on-disk data format or the BE↔BE / BE↔FE wire format, so the standard BE-first rolling upgrade ordering is preserved.

  3. Session variables (fe/fe-core/.../qe/SessionVariable.java, .../qe/VariableMgr.java)

    Remove 16 stale Doris-specific session variables along with their public name constants and any getter/setter methods: enable_jdbc_oracle_null_predicate_push_down, use_v2_rollup, rewrite_count_distinct_to_bitmap_hll, enable_variant_access_in_original_planner, extract_wide_range_expr, auto_broadcast_join_threshold, runtime_filters_max_num, disable_inverted_index_v1_for_variant, enable_infer_predicate, limit_rows_for_single_instance, nereids_star_schema_support, enable_cbo_statistics, enable_eliminate_sort_node, drop_table_if_ctas_failed, trace_nereids, enable_sync_mv_cost_based_rewrite.

    Selection criteria for each removed variable: zero references in fe/fe-core/src/main/java and fe/fe-common/src/main/java outside SessionVariable.java itself (after expanding the search to cover get/set/is-prefixed accessors of the field name, the public name constant, the underlying lowercase string literal, the body of toThrift(), and the body of setForwardedSessionVariables()), and zero hits in regression-test/, docs/, tools/, and be/test/.

    Upgrade compatibility is the critical concern here. Unlike fe.conf and be.conf, unknown session-variable names cause VariableMgr to throw ERR_UNKNOWN_SYSTEM_VARIABLE at four sites (setVar, setVarForNonMasterFE, fillValue, getValue). A hard delete would therefore break existing user scripts and JDBC client connection init paths that still issue SET on the old names during a BE-then-FE rolling upgrade. To stay safe, VariableMgr now keeps an explicit REMOVED_SESSION_VAR_NAMES set listing the 16 removed lowercase names; each of the four lookup sites short-circuits on this set so that SET <removed_var> = ... silently no-ops and SELECT @@<removed_var> returns an empty string. The same pattern is already used for mysql_compat_var_whitelist.

The legacy FE configs and BE configs listed above have been removed. The following session variables have been removed; SET and SELECT @@ against them now silently succeed (with empty/no-op behavior) instead of returning the variable's previous default:
enable_jdbc_oracle_null_predicate_push_down, use_v2_rollup, rewrite_count_distinct_to_bitmap_hll,
enable_variant_access_in_original_planner, extract_wide_range_expr, auto_broadcast_join_threshold, runtime_filters_max_num, disable_inverted_index_v1_for_variant, enable_infer_predicate, limit_rows_for_single_instance, nereids_star_schema_support, enable_cbo_statistics, enable_eliminate_sort_node, drop_table_if_ctas_failed, trace_nereids,
enable_sync_mv_cost_based_rewrite. If any of the removed FE keys is still present in fe.conf on upgrade, FE logs a WARN line on startup.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@jacktengg jacktengg requested a review from gavinchou as a code owner May 27, 2026 09:27
@jacktengg
Copy link
Copy Markdown
Contributor Author

/review

@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

…n variables

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Doris has accumulated a number of FE configs, BE configs and session
variables that are either explicitly marked `@Deprecated`, or simply have
no remaining references anywhere in the codebase (production sources,
regression tests, docs, or tools). They add noise to the user-facing
configuration surface and to the source tree without changing any
behavior. This commit removes them in three coordinated parts and adds
the minimum amount of backward-compatibility plumbing required to keep a
BE-then-FE rolling upgrade safe.

1. FE configs (fe/fe-common/.../Config.java)

   Remove 13 `@Deprecated` fields with no remaining production
   references: `meta_publish_timeout_ms`, `backup_plugin_path`,
   `max_unfinished_load_job`, `use_new_tablet_scheduler`,
   `enable_concurrent_update`, `cbo_max_statistics_job_num`,
   `max_cbo_statistics_task_timeout_sec`,
   `cbo_concurrency_statistics_task_num`, `cbo_default_sample_percentage`,
   `finish_job_max_saved_second`, `enable_array_type`,
   `period_analyze_simultaneously_running_task_num`,
   `maximum_parallelism_of_export_job`.

   `ConfigBase.java` gains a small `warnDeprecatedKeys()` helper. After
   `setFields()` runs, if any of the legacy keys above is still listed in
   `fe.conf`, FE logs a single WARN line so operators notice the stale
   entry on upgrade. FE startup is not affected: the FE config loader
   iterates over Java fields, so unknown keys in `fe.conf` were already
   silently ignored before this change.

2. BE configs (be/src/common/config.{cpp,h}, be/src/cloud/config.{cpp,h})

   Remove 24 configs that have zero references in the BE source tree:
   `doris_scanner_max_run_time_ms`, `multi_get_max_threads`, `num_disks`,
   `read_size`, `load_max_wg_active_memtable_percent`,
   `jdbc_connection_pool_cache_clear_time_sec`, `big_column_size_buffer`,
   `small_column_size_buffer`, `max_fragment_start_wait_time_seconds`,
   `enable_workload_group_for_scan`,
   `workload_group_scan_task_wait_timeout_ms`,
   `variant_use_cloud_schema_dict_cache`,
   `variant_threshold_rows_to_estimate_sparse_column`,
   `variant_nested_group_max_depth`, `enable_ttl_cache_evict_using_lru`,
   `file_cache_background_ttl_gc_batch`,
   `kerberos_refresh_interval_second`, `enable_debug_log_timeout_secs`,
   `enable_index_compaction`, `schema_dict_cache_capacity`,
   `test_s3_resource`, `meta_service_use_load_balancer`,
   `meta_service_rpc_timeout_ms`, `delete_bitmap_rpc_retry_times`.

   The BE config loader (`be/src/common/config.cpp` `init()`) also
   iterates over the registered field map, so unknown keys in `be.conf`
   were already silently ignored — hard removal is safe on a rolling
   upgrade. None of the removed options affect on-disk data format or
   the BE↔BE / BE↔FE wire format, so the standard BE-first rolling
   upgrade ordering is preserved.

3. Session variables
   (fe/fe-core/.../qe/SessionVariable.java, .../qe/VariableMgr.java)

   Remove 16 stale Doris-specific session variables along with their
   public name constants and any getter/setter methods:
   `enable_jdbc_oracle_null_predicate_push_down`, `use_v2_rollup`,
   `rewrite_count_distinct_to_bitmap_hll`,
   `enable_variant_access_in_original_planner`, `extract_wide_range_expr`,
   `auto_broadcast_join_threshold`, `runtime_filters_max_num`,
   `disable_inverted_index_v1_for_variant`, `enable_infer_predicate`,
   `limit_rows_for_single_instance`, `nereids_star_schema_support`,
   `enable_cbo_statistics`, `enable_eliminate_sort_node`,
   `drop_table_if_ctas_failed`, `trace_nereids`,
   `enable_sync_mv_cost_based_rewrite`.

   Selection criteria for each removed variable: zero references in
   `fe/fe-core/src/main/java` and `fe/fe-common/src/main/java` outside
   `SessionVariable.java` itself (after expanding the search to cover
   `get`/`set`/`is`-prefixed accessors of the field name, the public
   name constant, the underlying lowercase string literal, the body of
   `toThrift()`, and the body of `setForwardedSessionVariables()`), and
   zero hits in `regression-test/`, `docs/`, `tools/`, and `be/test/`.

   Upgrade compatibility is the critical concern here. Unlike fe.conf and
   be.conf, unknown session-variable names cause
   `VariableMgr` to throw `ERR_UNKNOWN_SYSTEM_VARIABLE` at four sites
   (`setVar`, `setVarForNonMasterFE`, `fillValue`, `getValue`). A hard
   delete would therefore break existing user scripts and JDBC client
   connection init paths that still issue `SET` on the old names during
   a BE-then-FE rolling upgrade. To stay safe, `VariableMgr` now keeps
   an explicit `REMOVED_SESSION_VAR_NAMES` set listing the 16 removed
   lowercase names; each of the four lookup sites short-circuits on this
   set so that `SET <removed_var> = ...` silently no-ops and
   `SELECT @@<removed_var>` returns an empty string. The same pattern is
   already used for `mysql_compat_var_whitelist`.

The legacy FE configs and BE configs listed above have been removed. The
following session variables have been removed; `SET` and `SELECT @@`
against them now silently succeed (with empty/no-op behavior) instead of
returning the variable's previous default:
`enable_jdbc_oracle_null_predicate_push_down`, `use_v2_rollup`,
`rewrite_count_distinct_to_bitmap_hll`,
`enable_variant_access_in_original_planner`, `extract_wide_range_expr`,
`auto_broadcast_join_threshold`, `runtime_filters_max_num`,
`disable_inverted_index_v1_for_variant`, `enable_infer_predicate`,
`limit_rows_for_single_instance`, `nereids_star_schema_support`,
`enable_cbo_statistics`, `enable_eliminate_sort_node`,
`drop_table_if_ctas_failed`, `trace_nereids`,
`enable_sync_mv_cost_based_rewrite`. If any of the removed FE keys is
still present in `fe.conf` on upgrade, FE logs a WARN line on startup.
@jacktengg jacktengg force-pushed the wt-conf-var-cleanup branch from dca393f to 344906a Compare May 27, 2026 09:37
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for one compatibility regression in the session-variable cleanup.

Critical checkpoint conclusions:

  • Goal/test proof: The PR removes deprecated/unused FE/BE configs and stale session variables. Most removals are guarded by warning/no-op compatibility paths, but enable_jdbc_oracle_null_predicate_push_down is still actively consumed by the JDBC connector and is not covered by tests here.
  • Scope/focus: The config cleanup is generally focused, but this session variable is not stale while connector code still depends on it.
  • Concurrency/lifecycle: No new shared mutable concurrency or non-intuitive lifecycle issue was identified.
  • Configuration/session compatibility: Blocking issue found. Existing users can no longer enable Oracle NULL predicate pushdown because SET is ignored and the session map omits the key.
  • Incompatible changes/storage formats: No storage-format or FE-BE protocol compatibility issue was found in the reviewed config removals.
  • Parallel code paths: BE/FE deprecated config warning paths were both added; no missing corresponding config warning was identified.
  • Test coverage: No test covers the still-active JDBC connector session variable path, which is why this regression can slip through.
  • Performance/observability: No new runtime hot-path performance issue was identified in the current PR diff.
  • User focus: No additional user-provided review focus was supplied.

}

// Form map from variable name to its field in Java class.
static {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this session variable breaks the JDBC Oracle NULL predicate pushdown path. JdbcQueryBuilder still reads enable_jdbc_oracle_null_predicate_push_down from the session properties map, but after this change SET enable_jdbc_oracle_null_predicate_push_down=true is silently ignored and VariableMgr.toMap() no longer includes the key, so oracleNullPredicatePushDown always falls back to false. Please keep this variable until the connector no longer depends on it, or replace it with another forwarded setting and update the connector at the same time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants