OxyLabs updates #18009
Merged
OxyLabs updates #18009
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 3 Skipped Deployments
|
WalkthroughAdds parse and render propDefinitions to the Oxylabs app and exposes them in scrape-url and scrape-with-query actions (forwarded in the scrape payload). Removes three decommissioned QUERY_SOURCES entries. Bumps versions for several Oxylabs components and the package. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant ScrapeAction as Scrape Action (URL/Query)
participant OxylabsApp as Oxylabs App
participant OxylabsAPI as Oxylabs API
User->>ScrapeAction: Invoke with source/URL, parse?, render?
ScrapeAction->>OxylabsApp: oxylabs.scrape({ ..., parse, render })
OxylabsApp->>OxylabsAPI: POST /scrape (payload includes parse, render)
OxylabsAPI-->>OxylabsApp: Response (structured data or HTML)
OxylabsApp-->>ScrapeAction: Return response
ScrapeAction-->>User: Emit API response
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Assessment against linked issues
Assessment against linked issues: Out-of-scope changes
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (8)
components/oxylabs/actions/create-proxy-session/create-proxy-session.mjs(1 hunks)components/oxylabs/actions/create-schedule/create-schedule.mjs(1 hunks)components/oxylabs/actions/scrape-url/scrape-url.mjs(3 hunks)components/oxylabs/actions/scrape-with-query/scrape-with-query.mjs(3 hunks)components/oxylabs/common/constants.mjs(0 hunks)components/oxylabs/oxylabs.app.mjs(1 hunks)components/oxylabs/package.json(1 hunks)components/oxylabs/sources/new-scheduled-run-completed/new-scheduled-run-completed.mjs(1 hunks)
💤 Files with no reviewable changes (1)
- components/oxylabs/common/constants.mjs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Lint Code Base
- GitHub Check: Verify TypeScript components
- GitHub Check: Publish TypeScript components
🔇 Additional comments (10)
components/oxylabs/package.json (1)
3-3: Version bump looks goodPackage version increment to 0.1.1 is consistent with action/source bumps.
components/oxylabs/actions/create-proxy-session/create-proxy-session.mjs (1)
8-8: Version bump only — OKNo logic changes; safe increment to 0.0.2.
components/oxylabs/actions/create-schedule/create-schedule.mjs (1)
8-8: Version bump only — OKNo functional diffs; 0.0.2 is fine.
components/oxylabs/sources/new-scheduled-run-completed/new-scheduled-run-completed.mjs (1)
9-9: Version bump only — OKSource version to 0.0.2 with no runtime changes.
components/oxylabs/oxylabs.app.mjs (1)
23-28: New parse prop — LGTMOptional boolean parse prop aligns with the requirement to toggle structured parsing.
components/oxylabs/actions/scrape-with-query/scrape-with-query.mjs (3)
29-34: Added parse prop — OKProp wired via propDefinition and forwarded in payload. Matches requirement.
35-40: No remainingrenderdependencies; decommissioned sources removed
- The constants for
amazon_reviews,amazon_questions, andgoogle_imagesare no longer present anywhere undercomponents/oxylabs.- The only places referencing
parse: this.parseandrender: this.renderare:
•components/oxylabs/actions/scrape-url/scrape-url.mjs(lines 50–51)
•components/oxylabs/actions/scrape-with-query/scrape-with-query.mjs(lines 49–50)Once you mark
renderas optional inoxylabs.app.mjs, these existing prop definitions will work as intended.
49-51: Confirm parse + render compatibilityForwarding
parseandrenderhere matches the Oxylabs Web Scraper API contract—no changes needed:
- Valid
rendervalues are"html","png", or""(to disable rendering).- You can combine
render:"html"withparse:true(the parser runs on the rendered HTML).- If you need custom parsing, include
parsing_instructionsalongsideparse:true; note that some sources use a built-in parser whenparse:trueis set, and only one parser type may run at a time.- Remember that rendering incurs extra time and traffic—consider increasing client timeouts where appropriate.
✔️ components/oxylabs/actions/scrape-with-query/scrape-with-query.mjs (lines 49–51)
components/oxylabs/actions/scrape-url/scrape-url.mjs (2)
30-35: Added parse prop — OKPropDefinition reference and optional behavior are appropriate.
50-51: Payload forwards parse and render — OKNo issues forwarding both options to the API.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #17903
Summary by CodeRabbit