Skip to content

v1.0.0

Latest

Choose a tag to compare

@sancho1952007 sancho1952007 released this 02 Oct 19:39
  1. Optimizations:
  • 99% reduction in database round-trips: Replaced row-by-row inserts (100 individual queries per page) with native batch operations across all databases (jsonb_array_elements for Postgres, bulkWrite for MongoDB, multi-row INSERT for MySQL, and batch insert for ClickHouse).
  • ~50x–100x faster database write throughput: Database ingestion overhead dropped from ~2.5s per page to ~30ms per page, making the overall end-to-end sync ~10x faster and no longer bottlenecked by database latency.
  • Fixes serverless data loss (Vercel / AWS Lambda): All database writes are now explicitly awaited before run() resolves, preventing serverless runtimes from freezing containers while writes are in flight.
  • Added .disconnect() / .close(): Enables clean TCP connection teardown in serverless finally blocks, preventing connection pool exhaustion.
    Replaced recursion with iterative loops: Constant stack depth, eliminating stack overflow risks on large catalogs.
  1. Added a better secrets input (@inquirer/password) for interactive mode in CLI (for Database URL & API Key).
  2. Updated dependencies.
  3. Fixed depreciated MySQL update method.
  4. Fixed Licence id key for MongoDB.
  5. Updated MongoDB connection string parsing to make it easily accessible to a wide range of database environments.