You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a developer maintaining a large shared pnpm store, I frequently use pnpm fetch to download and update dependencies from a registry mirror.
Occasionally, some package downloads time out due to transient network fluctuations. These timeouts do not necessarily indicate that the registry is overloaded or that the configured concurrency is too high.
However, when a request times out while other requests to the same origin are still active, pnpm permanently reduces the concurrency limit for that origin to 1 for the remainder of the current process.
Even after network conditions recover, all subsequent downloads from that origin remain limited to a single connection.
This significantly reduces download throughput and increases the time required to update a large shared pnpm store.
I would like pnpm to provide more control over this behavior, especially for environments where occasional network timeouts are expected but parallel downloads generally work well.
Describe the solution you'd like
I would like to propose making automatic network concurrency downscaling configurable.
Ideally, pnpm could support the following improvements:
An option to disable automatic downscaling: Allow users to opt out of reducing concurrency after a timeout, while keeping the existing timeout detection and retry mechanisms.
A configurable minimum concurrency: Instead of always reducing concurrency to 1, allow users to specify a minimum concurrency limit.
Automatic concurrency recovery: Consider gradually restoring the original concurrency after a certain number of successful requests or after a cooldown period.
For example, an optional configuration could look like:
Alternatively, for users who prefer to keep adaptive behavior enabled:
networkConcurrency: 32minNetworkConcurrency: 8
I understand that reducing concurrency can be beneficial when a registry or network is genuinely overloaded. Therefore, I am not suggesting that the current behavior should necessarily be removed or changed for everyone.
Instead, providing configuration options would allow users to choose the behavior that best fits their network environment.
The most important requirement for my use case is the ability to prevent a single transient timeout from permanently reducing the concurrency of an origin to one connection.
Describe the drawbacks of your solution
Disabling automatic concurrency downscaling may cause repeated timeouts or additional network pressure when a registry is genuinely overloaded or when the network cannot handle the configured concurrency.
A configurable minimum concurrency may also prevent pnpm from adapting sufficiently to severely degraded network conditions.
Automatic recovery introduces additional complexity, as pnpm needs to determine when network conditions have recovered without causing repeated concurrency fluctuations.
However, these trade-offs could be addressed by keeping the current behavior as the default and allowing users to explicitly opt into alternative strategies.
I believe the flexibility would be valuable for users working with reliable registry mirrors, private registries, CI environments, and large shared package stores.
Describe alternatives you've considered
I have considered the following alternatives:
Increasing fetchTimeout: This may reduce the likelihood of triggering downscaling due to temporary network fluctuations, but it cannot prevent the behavior when a timeout eventually occurs.
Increasing fetchRetries: Retries help recover failed downloads, but they do not restore the concurrency limit after it has been reduced.
Restarting pnpm fetch: Starting a new process resets the per-origin concurrency state, but repeatedly restarting fetch operations is inefficient and inconvenient when maintaining a large shared store.
Patching pnpm's source code: Disabling the downscaling logic directly in the Rust network implementation can avoid this issue, but maintaining a custom pnpm build across upgrades is undesirable.
I believe a built-in configuration option would be a much cleaner and more maintainable solution.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Contribution
Describe the user story
As a developer maintaining a large shared pnpm store, I frequently use
pnpm fetchto download and update dependencies from a registry mirror.Occasionally, some package downloads time out due to transient network fluctuations. These timeouts do not necessarily indicate that the registry is overloaded or that the configured concurrency is too high.
However, when a request times out while other requests to the same origin are still active, pnpm permanently reduces the concurrency limit for that origin to 1 for the remainder of the current process.
Even after network conditions recover, all subsequent downloads from that origin remain limited to a single connection.
This significantly reduces download throughput and increases the time required to update a large shared pnpm store.
I would like pnpm to provide more control over this behavior, especially for environments where occasional network timeouts are expected but parallel downloads generally work well.
Describe the solution you'd like
I would like to propose making automatic network concurrency downscaling configurable.
Ideally, pnpm could support the following improvements:
An option to disable automatic downscaling: Allow users to opt out of reducing concurrency after a timeout, while keeping the existing timeout detection and retry mechanisms.
A configurable minimum concurrency: Instead of always reducing concurrency to 1, allow users to specify a minimum concurrency limit.
Automatic concurrency recovery: Consider gradually restoring the original concurrency after a certain number of successful requests or after a cooldown period.
For example, an optional configuration could look like:
Alternatively, for users who prefer to keep adaptive behavior enabled:
I understand that reducing concurrency can be beneficial when a registry or network is genuinely overloaded. Therefore, I am not suggesting that the current behavior should necessarily be removed or changed for everyone.
Instead, providing configuration options would allow users to choose the behavior that best fits their network environment.
The most important requirement for my use case is the ability to prevent a single transient timeout from permanently reducing the concurrency of an origin to one connection.
Describe the drawbacks of your solution
Disabling automatic concurrency downscaling may cause repeated timeouts or additional network pressure when a registry is genuinely overloaded or when the network cannot handle the configured concurrency.
A configurable minimum concurrency may also prevent pnpm from adapting sufficiently to severely degraded network conditions.
Automatic recovery introduces additional complexity, as pnpm needs to determine when network conditions have recovered without causing repeated concurrency fluctuations.
However, these trade-offs could be addressed by keeping the current behavior as the default and allowing users to explicitly opt into alternative strategies.
I believe the flexibility would be valuable for users working with reliable registry mirrors, private registries, CI environments, and large shared package stores.
Describe alternatives you've considered
I have considered the following alternatives:
Increasing
fetchTimeout: This may reduce the likelihood of triggering downscaling due to temporary network fluctuations, but it cannot prevent the behavior when a timeout eventually occurs.Increasing
fetchRetries: Retries help recover failed downloads, but they do not restore the concurrency limit after it has been reduced.Restarting
pnpm fetch: Starting a new process resets the per-origin concurrency state, but repeatedly restarting fetch operations is inefficient and inconvenient when maintaining a large shared store.Patching pnpm's source code: Disabling the downscaling logic directly in the Rust network implementation can avoid this issue, but maintaining a custom pnpm build across upgrades is undesirable.
I believe a built-in configuration option would be a much cleaner and more maintainable solution.
All reactions