Add --include --exclude to --restore, --copyfrom

[RhetTbull]

I don't think adding these to --backup is a good idea but it makes sense to add them to --copyfrom and --restore

    **I inspected the JSON** and now realized that timestamps and permissions were indeed not backed up.

• The plethora of Mac timestamps really tell you a lot and are so useful. Would be worthy to have them in --backup, --restore and also in --copyfrom. I am glad you recognize the backup value of this too.

• The permissions matter a bit less for general purposes, but matter in some use cases (restoring software directories, or shared directories).

For all use cases (--backup, --restore, --copyfrom (possibly in combo with paste and xargs) ) you may want to be able to set that not ALL attributes are included but only a certain subset:

1. --backup

• a) the default is to include all.
• b) but there are possible scenarios where you may not want to include all, e.g. where one may only want to backup tags (which are used uniformly across a team) but not comments (which are individual or potentially private) or not file owner names or numerical-IDs (for privacy concerns).
• c) Ofc opening the possibility that --backup does not include all attributes complicates the exception handling logic if on the --restore or --copyfrom side you request certain attributes which were not included in the --backup. So in doubt --backup shall not have that attribute filtering capabilities.

2. --restore : Likely that you want only certain attributes to be restored (e.g. comments) or certain attributes NOT restored on purpose (e.g. the modification date shall be now, to have an indication of the restoration).
3. --copyfrom: Even more likely that you want to use that very specifically.

So the companion parameter originally suggested as --copy should have a more general name if it can be used together with --restore, --copyfrom and potentially also --backup. I propose --include and --exclude. That way you must not work in an enumerate fashion. But can say "all but A,B" or "only X,Y".

Originally posted by @porg in #71 (comment)

[porg]

I agree then!

• The only reasons for --backup to support --include and --exclude were data privacy/sparsity concerns.
• But as I suspected and you seem to confirm, opening that possibility on the --backup side complicates things.
• Edge case users can still manually cleanup the .osxmeta.json via a RegEx find/replace with awk or similar and use --include / --exclude accordingly on the --restore or --copyfrom side.

[RhetTbull]

The only reasons for --backup to support --include and --exclude were data privacy/sparsity concerns.

Understand, but the data that might be of concern is still in the actual filesystem metadata so I don't really see a privacy concern if the same data is in the backup. You could use osxmetadata --clear ATTRIBUTE to remove the sensitive attribute from the file and then do the backup which eliminates the privacy issue.

That said, if you really wanted to strip a key from the backup file you could do something like the following. For example, if your Finder comment contained sensitive information:

jq '[.[] | del(.kMDItemWhereFroms)]' .osxmetadata.json > backup.json && rm .osxmetadata.json && mv backup.json .osxmetadata.json

The above command deletes the key kMDItemFinderComment from the backup file.

The issue with excluding keys from the backup is that the user could change their mind later and be in a place where they had lost the ability to restore important data.

[porg]

As long as the .osxmeta files sits on the APFS volume were it got created then that metadata is still in the metadata of the corresponding APFS files yes. But if you copy those files plus the .osxmeta to a NAS which does not support/carry-on the same metadata or where it gets stripped on purpose (e.g. individual's comments on files get stripped, from then on commented collaboratively) then this would be a concern.

But it's far fetched. And you laid out tools/workflows how to remove. It is just that usually its easiest "leaving out stuff" during creation rather than "operating it out later". But as already written initially, in doubt --backup shall contain all.

[RhetTbull]

Thinking about how to do this: --include and --exclude will be mutually exclusive. If neither is specified, all attributes are copied/restored/printed. If --include is specified, only the specified attributes are handled. If --exclude is specified, all attributes except the excluded attributes are handled. If both are specified, an error is generated.

[porg]

100% agree.