Notable Features
The entire task is designed to consists of 4 modules, 1 abstraction layer and 1 pipeline layer (DAG) so that the number of modules can be extended or added for maintainability and flexibility purpose.
The Abstraction layer has the ability to run every module independent from each other. Also, a single module can be executed in parallel with different configurations on different docker containers to improve performance.
