Skip to content

Releases: watercrawl/watercrawl-nodejs

Release v1.2.1

Choose a tag to compare

@amirasaran amirasaran released this 27 Jul 19:32

Changed

  • Enhanced getCrawlRequestResults Method
    • Added optional page, pageSize, and download parameters to the getCrawlRequestResults method for better pagination control and direct result downloading.
    • Updated documentation and examples to reflect the new parameters.
    • Added comprehensive test cases to validate the new functionality.

Fixed

  • Security: Patched a critical vulnerability in the form-data dependency by updating it to a secure version.

Full Changelog: v1.2.0...v1.2.1

Release v1.2.0

Choose a tag to compare

@amirasaran amirasaran released this 30 Jun 21:27

[1.2.0] - 2025-06-30

Added

  • Batch crawl API support for processing multiple URLs in a single request
  • Comprehensive sitemap API with improved functionality:
    • Dedicated methods for creating, retrieving, listing, and deleting sitemap requests
    • Support for different output formats (JSON, Markdown, Graph) via getSitemapResults
    • Real-time monitoring with event streaming via monitorSitemapRequest
  • New getCrawlRequestSitemap method to retrieve sitemap data from a crawl request with format options
  • Example code demonstrating batch crawl and sitemap API usage in the examples directory

Changed

  • Updated streaming implementation to use Axios streams instead of Fetch API for better Node.js compatibility
  • Improved test suite organization with clear categorization (Core API, Crawl API, Search API, Sitemap API)
  • Enhanced error handling in tests and async operations

Fixed

  • Fixed type issues with SitemapOptions and enum usage in tests
  • Improved test reliability for asynchronous streaming operations
  • Better handling of potentially long-running operations in examples

Deprecated

  • downloadResult - Use alternative methods for downloading result data
  • downloadSitemap - Use the new getCrawlRequestSitemap method with JSON format
  • downloadSitemapGraph - Use the new getCrawlRequestSitemap method with Graph format
  • downloadSitemapMarkdown - Use the new getCrawlRequestSitemap method with Markdown format

Full Changelog: v1.1.1...v1.2.0

Release v1.1.1

Choose a tag to compare

@amirasaran amirasaran released this 03 May 09:57

What's Changed

  • fix(stream): implement improved async event streaming mechanism by @amirasaran in #3

Full Changelog: v1.1.0...v1.1.1

Release v1.1.0

Choose a tag to compare

@amirasaran amirasaran released this 30 Apr 21:27
d22871e

What's Changed

  • Add sitemap download and visualization endpoints
  • Add search functionality with query, monitoring, and result handling
  • Update documentation with simplified examples
  • Update license copyright to WaterCrawl-NodeJS
  • Add tests for new sitemap and search API functionality
  • Update CHANGELOG.md for version 1.1.0

Full Changelog: v1.0.0...v1.1.0

Release v1.0.0

Choose a tag to compare

@amirasaran amirasaran released this 30 Apr 21:14

Initial release of the WaterCrawl Node.js client library providing a simple and intuitive way to interact with WaterCrawl's web crawling service.

Features Added

  • Full API coverage for WaterCrawl endpoints
  • TypeScript support with complete type definitions
  • Synchronous and asynchronous crawling capabilities
  • Real-time crawl monitoring with event streaming
  • Automatic result downloading and processing
  • Promise-based API with async/await support
  • Comprehensive error handling
  • ESLint and Prettier configuration
  • Continuous Integration with GitHub Actions
  • Automated npm publishing with version tags
  • Examples for basic usage
  • Complete documentation in README.md
  • MIT License file

Dependencies

  • axios ^1.6.5
  • eventsource-parser ^1.1.1
  • url-join ^5.0.0