[pull] main from firecrawl:main#235
Merged
Merged
Conversation
Previously, search endpoints (v1/v2 /search and /x402/search) rejected any team whose flags resolved searchZDR to "forced", returning a 400. Scrape already supports forced ZDR and cleans data up after the job, so search should behave the same way. Now we propagate the team's forced-ZDR state through the controller into executeSearch and the underlying scrape jobs, so scraped pages are cleaned up just like the scrape endpoint. The job-log entries also reflect the resolved ZDR state instead of being hardcoded to false.
The search /enterprise array has two distinct modes ("zdr" and "anon")
with different pricing and upstream behavior, but the previous
searchZDR: "forced" flag conflated them. Split into:
- searchZDR: "forced-zdr" -> always-on ZDR enterprise mode
- searchZDR: "forced-anon" -> always-on anonymous routing
- searchZDR: "forced" -> deprecated alias for forced-zdr
getSearchForcedKind(flags) returns "zdr" | "anon" | null so the
controller can inject the correct value into req.body.enterprise and let
the existing pipeline handle billing and routing.
Also fix the issue identified by cubic: zeroDataRetention in the log
context and applyZdrScope is now derived from the resolved request state
(team-forced OR per-request enterprise:zdr|anon), so "allowed" teams
opting in per-request are no longer misclassified as non-ZDR in logs or
error capture.
fix(api): allow search for teams with searchZDR forced
* Add redactPII to SDK options * Remove unreleased PII wording from SDK PR * Fix Elixir batch redactPII option * Use test site for redactPII scrape snippet
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )