- Calculating primes is one of the most computationally intensive task, and an important component in computer security.
- This repo aims at calculating prime numbers, in a certain range, using multithreading concepts in NodeJS.
- Basic sieve method is used to collect prime numbers. The range is divided between worker-threads here.
- And the number of worker threads spawned is equal to the number of cores, to maximise output. If more spawned need to switch between threads, which takes time, if more spawned, cores underutilized.
- But, if Node needs to do heavy computation, it cannot rely on EventLoop, since it runs on single-thread, and all the cores aren't optimally utilized. Here is when,
- From the Worker-Thread Docs-
- For tasks like these, even
clustercan be used. Although, they don't provide support for transferring or sharing memory as of now.
- For initialising a worker, passing data in workerData actually clones the data
- Whereas, for passing messages, using
worker.postMessage(), data is passed either by cloning, transfering or sharing
- Cloning :
Arraysare serialised then cloned, and desearialised. Since, the raw data segregation isn't know.
- Transfering :
ArrayBufferoperate on this, since, they use
Uint8Arrayas layer, owing to which, their raw segment sizes are known, and can be transfered easily.
- Sharing :
SharedArrayBufferis similar to ArrayBuffer, where you need to wrap it using
TypedArrays, but they can be shared. To maintain concurrency,
Atomicslibrary is used alongwith it, to perform concurrent opertations
- Array - MDN
- ArrayBuffer - MDN
- SharedArrayBuffer - MDN
- TypedArray - MDN
- A crash course in memory management - Mozilla Hacks
Other Tools used
- Inquirer.JS - To create command line interfaces, in a whim.
- Ora - Elegant terminal spinners for your command line interfaces.
- Benchmark.JS - A robust benchmarking library, with high-resolution timers.
Benchmarking and write to CSV
- Add serialised Array and benchmark
- Plot the benchmark data
- Benchmark message passing
- Implement communication between workers
- To calculate prime numbers upto a Range.
CPU family: 6
Model name: Intel(R) Core(TM) i5-8600K CPU @ 3.60GHz
- Over large inputs, worker thread works better than main threads.
- Also, sharedArrayBuffer gives best performance over various data structures.
- When using sharedArrayBuffer beware, to use Atomics library to maintain data consistency.
- Notes and errors encounterd, have been added to NOTES.md
- Always open for