Skip to content

WIP: Global Allow Disk Use#501

Closed
webysther wants to merge 15 commits intoYooooomi:masterfrom
webysther:GlobalAllowDiskUse
Closed

WIP: Global Allow Disk Use#501
webysther wants to merge 15 commits intoYooooomi:masterfrom
webysther:GlobalAllowDiskUse

Conversation

@webysther
Copy link

@webysther webysther commented Mar 21, 2025

More elegant version of #500

Fixes #499 #272

@webysther webysther changed the title Global Allow Disk Use WIP: Global Allow Disk Use Mar 21, 2025
@webysther
Copy link
Author

webysther commented Mar 21, 2025

Looks like the changes don't did much, I was able to solve part of problem using this server config:

setParameter:
   allowDiskUseByDefault: true
   maxIndexBuildMemoryUsageMegabytes: 2048
   internalQueryMaxPushBytes: 2048576000
   internalQueryMaxBlockingSortMemoryUsageBytes: 2048576000
storage:
  wiredTiger:
    engineConfig:
      journalCompressor: zstd
    collectionConfig:
      blockCompressor: zstd

But now I get another error:

[error]  MongoServerError: PlanExecutor error during aggregation :: caused by :: 
BufBuilder attempted to grow() to 67108867 bytes, past the 64MB limit.

Few possible solutions:

  • Save the consolidate agg by year
  • Using window function aka setWindowFields
  • When the filter is 'all' (or bigger than 1 year) change to get by year and merge in client-side

PS: allowDiskUse is always enabled for mongo > 4

@webysther webysther closed this Apr 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MongoServerError: PlanExecutor error during aggregation

1 participant