Informative messages for some side-effect functions

[mattwarkentin]

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.
• Format your code according to the tidyverse style guide.

Proposal

Hi @wlandau,

Any interest in providing suppressible informative messages when using some of the side-effect functions like:

• tar_delete()
• tar_deduplicate()
• tar_invalidate()
• tar_prune()

This is maybe most useful when pruning and deduplicating, since it is a little harder to reason about whether anything will be done at all.

For invalidation and deletion, it's a bit more obvious what will happen since the user has to specify names, but since tidyselect syntax is supported, might still be informative to know what targets were ultimately invalidated or deleted.

Just a thought.

[wlandau]

I will think about it. I would probably go with these:

• tar_delete()
• tar_invalidate()
• tar_prune()
• tar_load()
• tar_workspace()

Alternatively we could just write helpers to introspect the data store.

• tar_list(): basically list.files("_targets/objects/")
• tar_workspaces(): list.files("_targets/workspaces/")`

Between that and tar_meta(), I think we have all the information covered.

[wlandau]

tar_deduplicate() is a funny one because tar_meta() always reads deduplicated data (without modifying storage) so you would have to count the lines in _targets/meta/meta to check if deduplication actually happened. But tar_make() always deduplicates the file when it starts, so this is something users rarely have to worry about anyway.

[mattwarkentin]

Regarding the possibility of tar_list() (or maybe tar_objects() for consistency), do you think it should return literal objects (i.e. including branches), or just the root target names?

Does tar_prune() prune whole targets only, or is it able to prune branches that are no longer needed too? Seems like the former. This is probably relevant to what tar_list() might return.

[wlandau]

Regarding the possibility of tar_list() (or maybe tar_objects() for consistency), do you think it should return literal objects (i.e. including branches), or just the root target names?

I like tar_objects() more, now that you mention it. We could probably support a logical branches argument to choose whether we want branch names. tar_outdated() supports this.

Does tar_prune() prune whole targets only, or is it able to prune branches that are no longer needed too? Seems like the former. This is probably relevant to what tar_list() might return.

tar_prune() does account for branches in its reasoning:

https://github.com/wlandau/targets/blob/b6c338e77f85177dbcebc52261bc986ea6574231/R/tar_prune.R#L45-L61

[mattwarkentin]

Okay, good to know. I think I like the idea of tar_objects(names = NULL, branches = FALSE). I think this function would give enough information to allow the user to introspect for themselves what the implications would be when calling tar_delete(), tar_invalidate(), tar_prune(), or tar_load(), right?

tar_prune() is the only one I'm not sure of. tar_objects() would list all of the objects in cache, but what information lets the user know which objects still belong to the current pipeline? tar_meta() %>% filter(type %in% c("stem", "branches"), format != "file")??

[wlandau]

Okay, good to know. I think I like the idea of tar_objects(names = NULL, branches = FALSE). I think this function would give enough information to allow the user to introspect for themselves what the implications would be when calling tar_delete(), tar_invalidate(), tar_prune(), or tar_load(), right?

Actually, on reflection I think branches = FALSE would raise some unnecessarily difficult issues. We need the metadata to reason about branches, and stuff could be present in _targets/objects/ and not the metadata due to tar_invalidate(). Plus, we would need to think about whether representation in tar_objects() means all the branches are present or any branches are present, and that could throw some users off. tar_objects() as list.files("_targets/objects/") (+ safeguards) seems clearer.

tar_prune() is the only one I'm not sure of. tar_objects() would list all of the objects in cache, but what information lets the user know which objects still belong to the current pipeline? tar_meta() %>% filter(type %in% c("stem", "branches"), format != "file")??

tar_manifest() gets all the targets in the pipeline, which can be left-joined with tar_meta() to get branch info.

[wlandau]

Just implemented tar_objects() and tar_workspaces().

[mattwarkentin]

I think this example needs fixing

https://github.com/wlandau/targets/blob/c048af7c3e615cfa0643e5e51ad53ba51e3f8baf/R/tar_objects.R#L5-L15

[wlandau]

Done.