docs: Document cost optimizations. Fixes #1139 #2972
Conversation
alexec
changed the title
|
@jessesuen would you like to make any suggestions or edits please? |
|
@jessesuen do you think you might be able to review this week please? |
|
@sarabala1979 @jessesuen this is now two weeks old. Could I please chase for a code review? |
|
Kudos, SonarCloud Quality Gate passed!
|
docs/cost-optimisation.md
Outdated
|
|
||
| > Suitable if you have a workflow that passes a lot of artifacts within itself. | ||
|
|
||
| Copying artifacts to and from storage outside of a cluster can be expensive. You can use a volume claim template to mount a volume that is attached to each step within the cluster |
There was a problem hiding this comment.
Suitable if you have a workflow that passes a lot of artifacts within itself.
Theres so many variables in this that it's hard to make any recommendation other than "consider doing this depending on this."
Volumes vs. VolumeClaimTemplates vs. Artifacts is highly dependent on what their artifact storage provider is vs. what volume they are using. For example, I believe it is more expensive to allocate and delete new EBS volumes every workflow using the PVC feature, than it is to upload and download some small files to S3.
On the other hand if they are using a NFS volume shared between all their workflows with large artifacts, that might be cheaper than the data transfer and storage costs of S3.
Lets reword this so we outline the variables to consider when choosing when to use artifacts vs. volumes:
- data transfer costs (upload/download vs. copying)
- data storage costs (s3 vs. volume)
- requirement for parallel access to data (NFS vs. EBS vs. artifact)
Checklist:
"fix(controller): Updates such and such. Fixes #1234".See #2934,
Fixes #1139