Cloudflare Pages / DigitalOcean

[sharkstate]

Crawlee looks very exciting, just starting out looking into it.

If wanting to deploy to server as opposed to running localy - Is it possible to deploy as a Cloudflare Pages? Or is current best bet to use Docker through for instance DigitalOcean?

[B4nan]

First time hearing about Cloudflare pages, looks like that is for frontend, while Crawlee is backend framework. DigitalOcean should work, or in general any hosting where you can run a docker container. Crawlee templates come with a ready to use dockerfiles.

For best user experience, you can use the Apify platform, a "Crawlee native" solution if you want, there you can schedule to run your crawler periodically, setup webhooks and work with the data in web UI, and many many more. Check the docs about it here:

https://crawlee.dev/docs/guides/apify-platform

[sharkstate]

Thank you.

Trying to use DigitalOcean Apps (cloud.digitalocean.com/apps) for this:

1. Creating a cheerio crawler locally: npx crawlee create crawler
2. run npm
3. upload to github
4. create a DigitalOcean App using above github repository

It fails during deployment with "Deploy Error: Non-Zero Exit Code"

However from the logs I can see the scraper outputs as would expect (see attached)

Anyone tried deploying to CloudFlare Pages or similiar?