Skip to content

feat: Implement unit tests and enhance scraper reliability

Choose a tag to compare

View all tags
@amineremache amineremache released this 20 Jun 18:13
· 5 commits to main since this release
v0.2.1
6cfbfb1

feat: Implement unit tests and enhance scraper reliability

This commit introduces Vitest for unit testing and significantly refactors the Daft.ie scraper for improved reliability, error handling, and maintainability.

Key changes include:

  • Unit Testing:
    • Added Vitest and configured it to run tests from the src directory.
    • Implemented comprehensive unit tests for parsePrice, parseBeds, and slugify helper functions in daftScraper.ts.
  • Scraper Enhancements (daftScraper.ts):
    • Refined client-side location filtering for "Ringsend".
    • Added detailed logging for CSS selector fallbacks.
    • parsePrice now returns a structured object { value, type }.
    • Enhanced pagination robustness with retries and better logging.
  • Error Handling & Logging:
    • MCP tools return structured JSON errors.
    • Server-side logs for errors are more detailed.
    • Added logging for selector fallbacks and pagination termination.
  • API Disclaimer (daftApi.ts):
    • Added disclaimer about API key requirement.
  • Build & Dependencies:
    • Updated package.json for Vitest.
    • Updated .gitignore.

Addresses BUG-001, ENH-001, ENH-002, ENH-003, ENH-004, ENH-005, TASK-001 (disclaimer part), and TASK-002 from the project roadmap.

Assets 2
Loading