You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
Dataset retention_policy is now enforced as a read-time view.item_days hides inactive items whose last_seen_at is older than the window, along with exports created before it. change_days hides changes and run warnings older than its window. Active items are never hidden. Nothing is deleted: shortening, lengthening or clearing the policy takes effect on the next read and is fully reversible. Dataset item_count in list/get/update responses reflects the visible items, and hidden exports return no download URL.
Documentation for hosted plan limits. Scrape, Search and Map document the 429 Concurrency limit reached response. Scrape and Search also document the 403 stealth_proxy_not_allowed response. Proxy Rules explains which proxy modes each plan allows, and Monitors has a new "Plan Limits (Hosted Service)" section. All of these are translated into every supported docs language.
Changed
Docs: scripts are excluded from the Next.js type-check, so the docs app builds on its own in container deployments.
Upgrade notes
No new database migrations. The visibility rules read the existing datasets.retention_policy column. Datasets without a policy behave exactly as before.
Redeploy the API and workers: dataset exports run in workers, which now apply the same visibility rules.
ANYCRAWL_MONITOR_RETENTION_DAYS is unrelated to dataset retention and still deletes old monitor history when set.
Server workspace packages move to 1.2.0. The independently versioned JS SDK remains at 0.0.9.