The high throughput compute grid project (HTC-Grid) is a container based cloud native HPC/Grid environment. The project provides a reference architecture that can be used to build and adapt a modern High throughput compute solution using underlying AWS services, allowing users to submit high volumes of short and long running tasks and scaling environments dynamically.
Warning: This project is an Open Source (Apache 2.0 License), not a supported AWS Service offering.
When should I use HTC-Grid ?
HTC-Grid should be used when the following criteria are meet:
- A high task throughput is required (from 250 to 10,000+ tasks per second).
- The tasks are loosely coupled.
- Variable workloads (tasks with heterogeneous execution times) are expected and the solution needs to dynamically scale with the load.
When should I not use the HTC-Grid ?
HTC-Grid might not be the best choice if :
- The required task throughput is below 250 tasks per second: Use AWS Batch instead.
- The tasks are tightly coupled, or use MPI. Consider using either AWS Parallel Cluster or AWS Batch Multi-Node workloads instead
- The tasks uses third party licensed software.
How do I use HTC-Grid ?
The full documentation of the HTC grid can be accessed here https://awslabs.github.io/aws-htc-grid/