Dealing with thousands of files

This repository provides an example of using queues to handle thousands of files.

🤖 How?

1. First, make it boom.

   Clone this project, and install it's dependencies:

   git clone https://github.com/jonalvarezz/use-queues.git
   cd use-queue
   yarn

   💥 Let's start off by trying to generate 40,000 files asynchronously (parallel)

   yarn start

   It should have boomed, just like this:

2. Second, and finally, let's fix it.

   Switch to the make-it-queue branch, which refactors the code to make use of better-queue

   git checkout -b make-it-queue origin/make-it-queue
   yarn
   yarn start

   It should have succeeded and thus, you should have 40,000 files now under the data folder.

   ls -l data | wc -l
   # -> 40000

   Compare the changes of the branches. Happy hacking!