Bag validation failed: data/.DS_Store exists on filesystem but is not in the manifest

[jjkoehorst]

Related to #33 as I was regenerating the dataset my Mac decided it was time to create a .DS_Store file on the fly which of course cause some conflicts. I am not sure if this should be hard coded but .DS_Store are "useless" files for RO Crates as far as I can tell.

http://download.systemsbiology.nl/unlock/cwl/issues/PROV_DS_Store.zip

data/.DS_Store exists on filesystem but is not in the manifest
Traceback (most recent call last):
  File "/Users/jasperk/mambaforge/bin/runcrate", line 8, in <module>
    sys.exit(cli())
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/runcrate/cli.py", line 67, in convert
    builder = ProvCrateBuilder(root, workflow_name, license, readme)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/runcrate/convert.py", line 188, in __init__
    self.ro = ResearchObject(BDBag(str(root)))
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/cwlprov/ro.py", line 66, in __init__
    bag.validate()
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/bdbag/bdbagit.py", line 490, in validate
    self._validate_contents(processes=processes, fast=fast, completeness_only=completeness_only, callback=callback)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/bdbag/bdbagit.py", line 519, in _validate_contents
    self._validate_completeness()
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/bdbag/bdbagit.py", line 549, in _validate_completeness
    raise BagValidationError(_("Bag validation failed"), errors)
bagit.BagValidationError: Bag validation failed: data/.DS_Store exists on filesystem but is not in the manifest

[simleo]

This is not a runcrate bug. It's the RO bundle that's invalid due to the presence of the extra files generated by the Mac. For instance, with cwlprov:

$ cwlprov -d PROV/ validate
WARNING:bdbag.bdbagit:data/.DS_Store exists on filesystem but is not in the manifest
ERROR:cwlprov.tool:BagIt validation failed for: /tmp/temp/PROV: Bag validation failed: data/.DS_Store exists on filesystem but is not in the manifest

Maybe you browsed the RO bundle with after it's been generated by cwltool and the Mac added its metadata files, making the structure invalid. If those files are removed, the RO bundle becomes valid again.

$ find PROV/ -name .DS_Store -exec rm -fv {} \;
removed 'PROV/data/.DS_Store'
removed 'PROV/metadata/.DS_Store'
removed 'PROV/.DS_Store'
$ cwlprov -d PROV/ validate
Valid CWLProv RO: PROV