Skip to content

Conversation

@zees-dev
Copy link
Collaborator

Description

This pull request enhances the serialization of multiple structs in src/bless_crawl/mod.rs by adding the #[serde(skip_serializing_if = "Option::is_none")] attribute to optional fields.
This change ensures that None values are excluded from serialized output, reducing payload size and improving efficiency.

Serialization Enhancements:

  • ScrapeOptions struct: Added #[serde(skip_serializing_if = "Option::is_none")] to optional fields like include_tags, exclude_tags, viewport, user_agent, and headers.
  • Viewport, MapOptions, and CrawlOptions structs: Applied the same attribute to optional fields such as width, height, link_types, base_url, limit, max_depth, and others.
  • PageMetadata struct: Enhanced serialization by adding the attribute to a comprehensive list of optional metadata fields, including title, description, language, keywords, og_*, twitter_*, and more.
  • Response struct: Added the attribute to the error field to exclude it from serialization when None.
  • CrawlData struct: Applied the attribute to the link_map field for more efficient serialization.…etadata, Response, and CrawlData structs
  • Introduced new optional fields with serde serialization conditions to enhance flexibility in data handling.
  • Updated ScrapeOptions, MapOptions, CrawlOptions, PageMetadata, Response, and CrawlData to include additional metadata and configuration options.

…etadata, Response, and CrawlData structs

- Introduced new optional fields with serde serialization conditions to enhance flexibility in data handling.
- Updated ScrapeOptions, MapOptions, CrawlOptions, PageMetadata, Response, and CrawlData to include additional metadata and configuration options.
@zees-dev zees-dev requested a review from Copilot June 30, 2025 03:12
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances the serialization logic in the bless_crawl module by adding the #[serde(skip_serializing_if = "Option::is_none")] attribute to various optional fields, ensuring that None values are omitted from the output.

  • Serialization attributes added to optional fields in ScrapeOptions, MapOptions, CrawlOptions, PageMetadata, Response, and CrawlData.
  • This change reduces payload size and may improve serialization efficiency.

@zees-dev zees-dev requested review from Joinhack and uditdc June 30, 2025 03:12
@zees-dev zees-dev merged commit 53b5284 into main Jun 30, 2025
1 check passed
@zees-dev zees-dev deleted the feat/bless-crawl-improvements branch June 30, 2025 04:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants