KeyError: 'fastqc.cwl' while cwl file is in the provenance folder

[jjkoehorst]

This issue is around http://download.systemsbiology.nl/unlock/cwl/issues/PROV_ngtax.zip.

It tries to obtain the cwl_tool = self.cwl_defs[tool_fragment] where the tool_fragment is 'fastqc.cwl' however only the packed.cwl is available in the cwl_defs.

find PROV_ngtax/ | grep cwl
PROV_ngtax//snapshot/workflow_ngtax.cwl
PROV_ngtax//snapshot/files_to_folder_tool.cwl
PROV_ngtax//snapshot/fastqc.cwl
PROV_ngtax//snapshot/ngtax.cwl
PROV_ngtax//snapshot/ngtax_to_tsv-fasta.cwl
PROV_ngtax//workflow/packed.cwl
PROV_ngtax//metadata/provenance/primary.cwlprov.jsonld
PROV_ngtax//metadata/provenance/primary.cwlprov.json
PROV_ngtax//metadata/provenance/primary.cwlprov.xml
PROV_ngtax//metadata/provenance/primary.cwlprov.nt
PROV_ngtax//metadata/provenance/primary.cwlprov.ttl
PROV_ngtax//metadata/provenance/primary.cwlprov.provn

Traceback (most recent call last):
  File "/Users/jasperk/mambaforge/bin/runcrate", line 8, in <module>
    sys.exit(cli())
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/runcrate/cli.py", line 68, in convert
    crate = builder.build()
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/runcrate/convert.py", line 273, in build
    self.add_workflow(crate)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/runcrate/convert.py", line 329, in add_workflow
    self.add_step(crate, workflow, s)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/runcrate/convert.py", line 341, in add_step
    tool = self.add_tool(crate, workflow, cwl_step.run)
  File "/Users/jasperk/mambaforge/lib/python3.10/site-packages/runcrate/convert.py", line 352, in add_tool
    cwl_tool = self.cwl_defs[tool_fragment]
KeyError: 'fastqc.cwl'

[mr-c]

Did you force upgrade cwl-utils? Then you'll need at least this oneline change:

https://github.com/ResearchObject/runcrate/pull/40/files#diff-2ab79087d832fc1db2cbc22ff9b4bc31df5d518c69fd32d21aa92dff48d8675dR162

[jjkoehorst]

Not sure if that alone is sufficient as

    hints_map = {_["class"]: _ for _ in cwl_tool.hints}
TypeError: 'DockerRequirement' object is not subscriptable

[mr-c]

Not sure if that alone is sufficient as

    hints_map = {_["class"]: _ for _ in cwl_tool.hints}
TypeError: 'DockerRequirement' object is not subscriptable

Yeah, then grab the rest of that PR (or update your copy, as it was merged)

[jjkoehorst]

fixed with the latest updates as mentioned by @mr-c