perf: improve scaling based on memory (#2459)

We only allowed to use 70% of the available memory, this PR changes the limit to 90%. Tested with a low memory options and it did not have any effect, while it allows to use more memory on the large memory setups - where the 30% could mean 2gb or so, we dont need such a huge buffer. Also increases the scaling steps to 10% instead of 5% so speed up the scaling. Related: [apify.slack.com/archives/C0L33UM7Z/p1715109984834079](https://apify.slack.com/archives/C0L33UM7Z/p1715109984834079)
apify · May 13, 2024 · 2d5d443 · 2d5d443
1 parent 6da86a8
commit 2d5d443
Show file tree

Hide file tree

Showing 2 changed files with 3 additions and 3 deletions.
diff --git a/packages/core/src/autoscaling/autoscaled_pool.ts b/packages/core/src/autoscaling/autoscaled_pool.ts
@@ -242,8 +242,8 @@ export class AutoscaledPool {
             minConcurrency = 1,
             desiredConcurrency,
             desiredConcurrencyRatio = 0.90,
-            scaleUpStepRatio = 0.05,
-            scaleDownStepRatio = 0.05,
+            scaleUpStepRatio = 0.1,
+            scaleDownStepRatio = 0.1,
             maybeRunIntervalSecs = 0.5,
             loggingIntervalSecs = 60,
             taskTimeoutSecs = 0,

diff --git a/packages/core/src/autoscaling/snapshotter.ts b/packages/core/src/autoscaling/snapshotter.ts
@@ -141,7 +141,7 @@ export class Snapshotter {
             clientSnapshotIntervalSecs = 1,
             snapshotHistorySecs = 30,
             maxBlockedMillis = 50,
-            maxUsedMemoryRatio = 0.7,
+            maxUsedMemoryRatio = 0.9,
             maxClientErrors = 3,
             log = defaultLog,
             config = Configuration.getGlobalConfig(),