You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
99% reduction in database round-trips: Replaced row-by-row inserts (100 individual queries per page) with native batch operations across all databases (jsonb_array_elements for Postgres, bulkWrite for MongoDB, multi-row INSERT for MySQL, and batch insert for ClickHouse).
~50x–100x faster database write throughput: Database ingestion overhead dropped from ~2.5s per page to ~30ms per page, making the overall end-to-end sync ~10x faster and no longer bottlenecked by database latency.
Fixes serverless data loss (Vercel / AWS Lambda): All database writes are now explicitly awaited before run() resolves, preventing serverless runtimes from freezing containers while writes are in flight.
Added .disconnect() / .close(): Enables clean TCP connection teardown in serverless finally blocks, preventing connection pool exhaustion.
Replaced recursion with iterative loops: Constant stack depth, eliminating stack overflow risks on large catalogs.
Added a better secrets input (@inquirer/password) for interactive mode in CLI (for Database URL & API Key).
Updated dependencies.
Fixed depreciated MySQL update method.
Fixed Licence id key for MongoDB.
Updated MongoDB connection string parsing to make it easily accessible to a wide range of database environments.