Skip to content

Commit

Permalink
fix: support reloading of sessions with lots of retries
Browse files Browse the repository at this point in the history
When the existing session has very large retry histogram, we failed with
`RangeError: Maximum call stack size exceeded` because of doing `a.push(...b)`.
  • Loading branch information
B4nan committed Nov 7, 2022
1 parent 45bbd8d commit ebc89d2
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion packages/core/src/crawlers/statistics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -305,7 +305,9 @@ export class Statistics {

this.log.debug('Recreating state from KeyValueStore', { persistStateKey: this.persistStateKey });

this.requestRetryHistogram.push(...savedState.requestRetryHistogram);
// the `requestRetryHistogram` array might be very large, we could end up with
// `RangeError: Maximum call stack size exceeded` if we use `a.push(...b)`
savedState.requestRetryHistogram.forEach((idx) => this.requestRetryHistogram.push(idx));
this.state.requestsFinished = savedState.requestsFinished;
this.state.requestsFailed = savedState.requestsFailed;
this.state.requestsRetries = savedState.requestsRetries;
Expand Down

0 comments on commit ebc89d2

Please sign in to comment.