-
Notifications
You must be signed in to change notification settings - Fork 13
feat: update libs for limited perms, fix and speed up Dockerfile build, fix wrong imports, firefox patches and policies, NY TZ #82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…d, fix wrong imports
jirispilka
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we use similar dockerfile as in website-content-crawler: https://github.com/apify/store-website-content-crawler/blob/main/.actor/Dockerfile
Ideally I would keep it as close as possible.
matyascimbulka
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, pre-approving. There are some lint errors that should be fixed before merging.
Honestly I would not do that, we can customize the |
|
Build time is not that important when running in prod. I never said we should copy it blindly. Let me explain my reasoning: the tooling team is actively working on this so why not learn from their Dockerfile and reuse what makes sense for us. For example: TZ settings, patches (if needed), policies, etc. |
Makes sense, will implement the relevant stuff from the WCC dockerfile 👍 |
… WCC (#81) * fix: number of google pages results * Update src/crawlers.ts Co-authored-by: Matyáš Cimbulka <matyas.cimbulka@apify.com> * Update src/crawlers.ts Co-authored-by: Matyáš Cimbulka <matyas.cimbulka@apify.com> * refactor: user data handling for the serp pagination * refactor: remove duplicate code * fix: update request handler to use addRequests for search crawler * make code more typescripty * add var for better readability * feat: update libs for limited perms, fix and speed up Dockerfile build, fix wrong imports, firefox patches and policies, NY TZ (#82) * feat: update libs for limited perms, fix and speed up Dockerfile build, fix wrong imports * magic number to const * lint * fix lint, update crawlee * add firefox policies, patches, set NY TZ * add ghostery blocker * Squashed commit of the following: commit 7aadecd Author: MQ37 <themq37@gmail.com> Date: Wed Nov 26 11:14:09 2025 +0100 add var for better readability commit fd02b0a Author: MQ37 <themq37@gmail.com> Date: Wed Nov 26 11:09:57 2025 +0100 make code more typescripty * fix playwright version, fix types.ts * fix playwright version for the patch * fix merge issue * Update src/utils.ts Co-authored-by: Matyáš Cimbulka <matyas.cimbulka@apify.com> --------- Co-authored-by: Matyáš Cimbulka <matyas.cimbulka@apify.com>
This PR updates the node packages before enabling the limited permissions, just in case - encountered issue with bad cheerio related type imports and then with the Dockerfile build process.
Tested on my personal account