Skip to content

feat: updates to default memory settings - #26023

Merged
praveen-influx merged 1 commit into
mainfrom
chore/change-default-mem
Feb 20, 2025
Merged

praveen-influx merged 1 commit into
mainfrom
chore/change-default-mem

Conversation

@praveen-influx

@praveen-influx praveen-influx commented Feb 14, 2025 •

Copy link
Copy Markdown
Contributor
  • breaking change, replaced --parquet-mem-cache-size-mb and env var for it with --parquet-mem-cache-size (takes value now in percentage or MB) and it now defaults to 20% of total available memory
  • force snapshotting is set at 50%
  • datafusion mem pool is set to 20%

closes: #26009

@praveen-influx
praveen-influx requested a review from a team February 14, 2025 11:05
@praveen-influx
praveen-influx force-pushed the chore/change-default-mem branch 3 times, most recently from 794cf3f to 54f3637 Compare February 14, 2025 11:25
- breaking change, replaced `--parquet-mem-cache-size-mb` and env var for
  it with `--parquet-mem-cache-size` (takes value now in percentage or
  MB), now defaults to 20% of total available memory
- force snapshotting is set at 50%
- datafusion mem pool is set to 20%

closes: #26009
@praveen-influx
praveen-influx force-pushed the chore/change-default-mem branch from 54f3637 to c8b585e Compare February 14, 2025 12:02
@praveen-influx
praveen-influx marked this pull request as ready for review February 20, 2025 15:58
@praveen-influx
praveen-influx merged commit f1fc498 into main Feb 20, 2025
mgattozzi pushed a commit that referenced this pull request Feb 21, 2025
- breaking change, replaced `--parquet-mem-cache-size-mb` and env var for
  it with `--parquet-mem-cache-size` (takes value now in percentage or
  MB), now defaults to 20% of total available memory
- force snapshotting is set at 50%
- datafusion mem pool is set to 20%

closes: #26009
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set default for datafusion memory pool

2 participants