Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

requestHandlerTimeoutSecs upper-bound overflows #1766

Closed
metalwarrior665 opened this issue Feb 1, 2023 · 2 comments
Closed

requestHandlerTimeoutSecs upper-bound overflows #1766

metalwarrior665 opened this issue Feb 1, 2023 · 2 comments
Labels
bug Something isn't working.

Comments

@metalwarrior665
Copy link
Member

Which package is this bug report for? If unsure which one to select, leave blank

@crawlee/basic (BasicCrawler)

Issue description

WARN  PlaywrightCrawler: requestHandlerTimeoutMillis 100000000070000does not fit a signed 32-bit integer. Limiting the value to 2147483647
INFO  PlaywrightCrawler: Starting the crawl
(node:22406) TimeoutOverflowWarning: 200000000140000 does not fit into a 32-bit signed integer.
Timeout duration was set to 1.

This was fixed in the past by setting it to 2147483647 but seems Crawlee now multiplies this so we overflow again.

Code sample

requestHandlerTimeoutSecs: 10000000000


### Package version

latest

### Node.js version

node 16

### Operating system

_No response_

### Apify platform

- [ ] Tick me if you encountered this issue on the Apify platform

### I have tested this on the `next` release

_No response_

### Other context

_No response_
@metalwarrior665 metalwarrior665 added the bug Something isn't working. label Feb 1, 2023
@metalwarrior665
Copy link
Member Author

Even if you set requestHandlerTimeoutSecs: 2000000 (which is bellow safe signed int / 1000), it still fails a bit later so it is again multiplied somewhere.

(node:22706) TimeoutOverflowWarning: 4000140000 does not fit into a 32-bit signed integer.
Timeout duration was set to 1.

@foxt451
Copy link
Collaborator

foxt451 commented Jul 13, 2023

This PR would handle this: #1981. However, @B4nan mentioned it would be a good idea to allow to opt out of timeouts altogether, so I think this issue will be up to discussion

@B4nan B4nan closed this as completed in 8122622 Jul 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working.
Projects
None yet
Development

No branches or pull requests

2 participants