Add Oxylabs Document Loader

[oxy-rostyslav]

Description

Adds an Oxylabs Document loader that allows to load the data from multiple sources efficiently.

Example

[HenryHengZJ]

can you do pnpm lint-fix to fix the linting issues?

[Copilot]

Pull Request Overview

Adds a new Oxylabs document loader that integrates with the Oxylabs real-time scraping API and exposes it as a Flowise node.

• Introduces OxylabsLoader for making authenticated requests to various Oxylabs sources.
• Wraps the loader in an INode implementation (Oxylabs_DocumentLoaders) with UI inputs and output handling.
• Adds a credential definition for Oxylabs API credentials.

Reviewed Changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 3 comments.

File	Description
packages/components/nodes/documentloaders/Oxylabs/Oxylabs.ts	New loader class, API request methods, and Flowise node wrapper
packages/components/credentials/OxylabsApi.credential.ts	New credential class for Oxylabs API username/password

Comments suppressed due to low confidence (3)

packages/components/nodes/documentloaders/Oxylabs/Oxylabs.ts:69

• [nitpick] The OxylabsLoader class lacks JSDoc or inline comments. Adding a brief description and parameter/docs would improve readability and help future maintainers.

export class OxylabsLoader extends BaseDocumentLoader {

packages/components/nodes/documentloaders/Oxylabs/Oxylabs.ts:165

• [nitpick] Class name Oxylabs_DocumentLoaders uses an underscore and plural form. Consider renaming to OxylabsDocumentLoaderNode for consistency with the project's naming conventions.

class Oxylabs_DocumentLoaders implements INode {

packages/components/nodes/documentloaders/Oxylabs/Oxylabs.ts:1

• No unit tests were added for OxylabsLoader. Consider adding tests to cover each source type, parameter filtering, and error handling paths.

import { TextSplitter } from 'langchain/text_splitter'

[Copilot]

Using btoa may not work in Node.js environments; consider using Buffer.from(\${this.params.username}:${this.params.password}`).toString('base64')` for server-side base64 encoding.

Suggested change

	const auth = btoa(`${this.params.username}:${this.params.password}`)
	const auth = Buffer.from(`${this.params.username}:${this.params.password}`).toString('base64')

[Copilot]

Only the first result is converted into a document. To support multiple pages or results, map over response.data.results and create a document for each entry.

Suggested change

	const docs: OxylabsDocument[] = [
	{
	id: response.data.job.id.toString(),
	pageContent: response.data.results[0].content,
	metadata: {}
	}
	]
	const docs: OxylabsDocument[] = response.data.results.map((result, index) => ({
	id: `${response.data.job.id.toString()}-${index}`,
	pageContent: result.content,
	metadata: {}
	}))

[Copilot]

[nitpick] The getUniversal, getGoogleSearch, getAmazonProduct, and getAmazonSearch methods are almost identical. Consider unifying them into a single method that accepts source as an argument to reduce duplication.