Description
What
I believe caching is arguably in-scope of taskfile interface, since it is naturally part of workflows. We could choose to delegate caching to a end user, but then, the user would need to orchestrate the workflow on top of taskfiles, or write very complicated tasks to support this.
In fact, there is little to do for supporting a simple KV store caching solution.
Why
Caching is always a very important functionality in every products. Use case may happen to have back and forth task build (with identical content at times). In that case, KV cache is a obvious solution to checksum based solution in Taskfile.
As a argument against not supporting it, Taskfile currently support :
Prevent unnecessary work
via sources and generates fields in the interface. That is exactly the definition of caching : if input output same, dont do again.
How
Consider
sources:
- file.txt
- blabla.txt
generates:
- output.txt
It then creates already a hash in .task/checksum/myTaskName, and it contains (in other words, cat shows) a hash like 23bc5743a223c33397efb435f7cf59a6.
To implement KV store based cache, we just need to :
- support
cache: [false|true] default to false field, to enable the functionality
- use the checksum of the task as the key and the
generates/ listed files as the content
- bundle the content in an archive, e.g.
checksum.tar.zstd
We would end up with
sources:
- file.txt
- blabla.txt
generates:
- output.txt
cache: true
I am pretty sure I can implement that easily and it would have great impact. We would let backwards compatibility by defaulting to false.
Then, we can expand the cache by adding eviction policies such as LRU as part of .taskrc for example. But that can be implemented in a second PR.
Description
What
I believe caching is arguably in-scope of taskfile interface, since it is naturally part of workflows. We could choose to delegate caching to a end user, but then, the user would need to orchestrate the workflow on top of taskfiles, or write very complicated tasks to support this.
In fact, there is little to do for supporting a simple KV store caching solution.
Why
Caching is always a very important functionality in every products. Use case may happen to have back and forth task build (with identical content at times). In that case, KV cache is a obvious solution to checksum based solution in Taskfile.
As a argument against not supporting it, Taskfile currently support :
via
sourcesandgeneratesfields in the interface. That is exactly the definition of caching : if input output same, dont do again.How
Consider
It then creates already a hash in
.task/checksum/myTaskName, and it contains (in other words,catshows) a hash like23bc5743a223c33397efb435f7cf59a6.To implement KV store based cache, we just need to :
cache: [false|true] default to falsefield, to enable the functionalitygenerates/listed files as the contentchecksum.tar.zstdWe would end up with
I am pretty sure I can implement that easily and it would have great impact. We would let backwards compatibility by defaulting to
false.Then, we can expand the cache by adding eviction policies such as LRU as part of
.taskrcfor example. But that can be implemented in a second PR.