Skip to content

v0.2.4

Latest

Choose a tag to compare

@amineremache amineremache released this 03 Jul 23:44
· 2 commits to main since this release

feat(server): Refactor scraper, add LLM parsing, and switch to HTTP

This commit introduces a major refactoring of the Daft.ie scraper and adds several new features, including natural language query parsing and a move to an HTTP-based server.

Refactoring:

  • The core scraping logic in daftScraper.service.ts has been decomposed into smaller, single-responsibility services for fetching (fetch.service.ts), parsing (parser.service.ts), and filtering (filter.service.ts). This significantly improves modularity, maintainability, and testability.
  • Input parameters across the application have been standardized to use camelCase (e.g., minPrice instead of min_price) for consistency.

New Features:

  • LLM Query Parsing: A new parse_query tool has been added. It leverages llamaindex with an Ollama model to interpret natural language queries (e.g., "find a 2-bed apartment in Dublin under €2000") and convert them into structured JSON for the search tool.
  • HTTP SSE Server: The server transport has been upgraded from StdioServerTransport to SSEServerTransport using Express, transforming the application into a web server that communicates over HTTP and Server-Sent Events.
  • Docker Support: A Dockerfile is now included, enabling easy containerization and deployment of the server.

Enhancements:

  • Error Handling: A robust, centralized error handling system has been implemented using custom error classes (ScraperError, ValidationError, etc.) and a utility function (createErrorResponse) to generate standardized error payloads.
  • Testing: Comprehensive new unit tests have been added for the server, scraper, and filter services, with coverage reporting configured in package.json.

Other Changes:

  • Added .session to .gitignore.
  • Updated package.json with new dependencies (dotenv, llamaindex, @types/express) and bumped the version to 0.2.4.