feat(brightdata): add Bright Data integration with 8 tools

[waleedlatif1]

Summary

• Adds complete Bright Data integration with 8 tools: Web Unlocker (scrape URL), SERP search, Discover, sync scrape, async scrape dataset, snapshot status, download snapshot, and cancel snapshot
• Block with operation dropdown, conditional fields per operation, and proper param mapping
• All tools validated against official Bright Data API docs with correct endpoints, params, and response handling

Tools

Tool	API Endpoint	Description
brightdata_scrape_url	POST /request	Fetch content via Web Unlocker
brightdata_serp_search	POST /request	Search Google/Bing/DuckDuckGo/Yandex
brightdata_discover	POST /discover	AI-powered web discovery with intent ranking
brightdata_sync_scrape	POST /datasets/v3/scrape	Synchronous dataset scrape (up to 20 URLs)
brightdata_scrape_dataset	POST /datasets/v3/trigger	Async dataset scrape trigger
brightdata_snapshot_status	GET /datasets/v3/progress/{id}	Check async job status
brightdata_download_snapshot	GET /datasets/v3/snapshot/{id}	Download completed results
brightdata_cancel_snapshot	POST /datasets/v3/snapshot/{id}/cancel	Cancel active job

Files Changed

• apps/sim/tools/brightdata/ — 8 tool files, types, and barrel export
• apps/sim/blocks/blocks/brightdata.ts — Block definition
• apps/sim/components/icons.tsx — BrightDataIcon
• apps/sim/tools/registry.ts — Tool registrations
• apps/sim/blocks/registry.ts — Block registration
• apps/docs/ — Auto-generated docs and icon mapping

Test plan

• Verify block appears in toolbar under Tools category
• Test scrape URL operation with a Web Unlocker zone
• Test SERP search returns structured results
• Test Discover API returns results with relevance scores
• Test async dataset trigger returns snapshot ID
• Test sync scrape returns data or falls back to async (202)
• Test snapshot status/download/cancel lifecycle
• Verify advanced mode fields (country, language, numResults, includeContent) toggle correctly

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Apr 15, 2026 7:36pm

[cursor]

PR Summary

Medium Risk
Adds a new external-API integration that handles user-provided API keys and multiple request/response transforms; misconfiguration or API contract mismatches could break scraping/search workflows.

Overview
Adds a new Bright Data integration end-to-end: an brightdata block with an operation selector and conditional inputs that maps to eight new tools (brightdata_scrape_url, brightdata_serp_search, brightdata_discover, brightdata_sync_scrape, brightdata_scrape_dataset, brightdata_snapshot_status, brightdata_download_snapshot, brightdata_cancel_snapshot).

Registers the block and tools in the SIM registries, adds BrightDataIcon and icon mappings for docs and the landing integrations page, and publishes a new docs page plus tools meta.json entry. Also adjusts Agiloft’s bgColor to #FFFFFF in both the integrations data JSON and block config.

^{Reviewed by Cursor Bugbot for commit eb50282. Configure here.}

[greptile-apps]

Greptile Summary

Adds a complete Bright Data integration with 8 tools covering Web Unlocker scraping, SERP search, Discover, synchronous/asynchronous dataset scraping, and snapshot lifecycle management — each mapped to a conditional sub-block in a single brightdata block.

• The discover tool's transformResponse hardcodes query: null in its output, so any workflow step reading {{brightdata.query}} after a Discover operation will always get null. The value should be extracted from data.query in the parsed response (see inline comment).
• apps/sim/blocks/blocks/agiloft.ts has an unrelated bgColor change (#263A5C → #FFFFFF) that appears to be a stray edit.

Confidence Score: 4/5

Safe to merge after fixing the hardcoded null query output in the Discover tool.

One P1 defect: the Discover tool's query output is always null, silently dropping a documented output field that downstream workflow steps would read. All other tools are well-structured and follow established patterns. The unrelated Agiloft bgColor change is a P2 worth confirming but not blocking.

apps/sim/tools/brightdata/discover.ts — hardcoded null query output

Important Files Changed

Filename	Overview
apps/sim/tools/brightdata/discover.ts	Discover tool implementation; query output is hardcoded to null instead of being extracted from the API response
apps/sim/tools/brightdata/sync_scrape.ts	Sync scrape tool with correct 202/async fallback handling and proper JSON body wrapping in {input: [...]}
apps/sim/tools/brightdata/serp_search.ts	SERP search tool; correctly builds engine-specific URLs with brd_json=1 and handles JSON/raw content types
apps/sim/blocks/blocks/brightdata.ts	Block definition with 8 operations, conditional fields, and params switch; well-structured with good use of mode:advanced for optional fields
apps/sim/tools/brightdata/types.ts	Type definitions for all 8 tool param/response pairs; comprehensive and correctly typed
apps/sim/blocks/blocks/agiloft.ts	Unrelated change: bgColor switched from #263A5C to #FFFFFF, likely accidental or undiscussed cleanup
apps/sim/blocks/registry.ts	BrightDataBlock registered after brandfetch but before box — box was already out of order pre-existing, minor style issue
apps/sim/tools/registry.ts	All 8 Bright Data tools registered correctly; minor alphabetical ordering mismatch with box_ entries (pre-existing)

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    BD[BrightData Block] --> OP{Operation}
    OP -->|scrape_url| WU[Web Unlocker\nPOST /request]
    OP -->|serp_search| SERP[SERP Search\nPOST /request\nvia search engine URL]
    OP -->|discover| DISC[Discover\nPOST /discover]
    OP -->|sync_scrape| SYNC[Sync Scrape\nPOST /datasets/v3/scrape]
    OP -->|scrape_dataset| ASYNC[Scrape Dataset\nPOST /datasets/v3/trigger]
    OP -->|snapshot_status| STATUS[Snapshot Status\nGET /datasets/v3/progress/id]
    OP -->|download_snapshot| DL[Download Snapshot\nGET /datasets/v3/snapshot/id]
    OP -->|cancel_snapshot| CANCEL[Cancel Snapshot\nPOST /datasets/v3/snapshot/id/cancel]
    SYNC -->|202 timeout| ASYNC_FB[Falls back to async\nisAsync=true, returns snapshotId]
    SYNC -->|200 ok| SYNC_DATA[Returns data array\nisAsync=false]
    ASYNC --> SNAP_ID[Returns snapshotId]
    SNAP_ID --> STATUS
    STATUS -->|ready| DL
    STATUS -->|running/starting| STATUS
    SNAP_ID --> CANCEL

Loading

Comments Outside Diff (1)

1. apps/sim/tools/brightdata/discover.ts, line 568-576 (link)

   query output is always null

   The query field in the returned output is hardcoded to null, making the advertised output effectively useless. Any downstream workflow step that reads {{brightdata.query}} from a Discover operation will always receive null. The API response likely contains the echoed query at data.query — extract it the same way serp_search.ts reads data?.general?.query.

_{Reviews (3): Last reviewed commit: "fix(brightdata): disable incompatible Du..." | Re-trigger Greptile}

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit eb50282. Configure here.}