Cwltool fails with type:record and no fields defined

[ThomasHickman]

Expected Behavior

The cwl specification states that record types has an optional record field (http://www.commonwl.org/v1.0/CommandLineTool.html#CommandInputRecordSchema), so this file shouldn't yield an error when run:

cwlVersion: v1.0
class: CommandLineTool
inputs:
  - id: input
    type:
      type: record
outputs: []

with an input file of:

input:
  value: 7

Actual Behavior

When running cwltool test.cwl test.yaml or cwltool test.cwl, the task fails with the error below:

Without --debug:

/usr/local/bin/cwltool 1.0.20180111185617
Resolved 'test.cwl' to '.../test.cwl'
I'm sorry, I couldn't load this CWL file, try again with --debug for more information.
The error was: 'fields'

With --debug:

/usr/local/bin/cwltool 1.0.20180111185617
Resolved 'test.cwl' to '.../test.cwl'
I'm sorry, I couldn't load this CWL file.  The error was:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/cwltool/main.py", line 895, in main
    makeTool, make_tool_kwds)
  File "/usr/local/lib/python3.6/site-packages/cwltool/load_tool.py", line 322, in make_tool
    tool = makeTool(processobj, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/cwltool/workflow.py", line 36, in defaultMakeTool
    return draft2tool.CommandLineTool(toolpath_object, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/cwltool/draft2tool.py", line 198, in __init__
    super(CommandLineTool, self).__init__(toolpath_object, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/cwltool/process.py", line 515, in __init__
    c["type"] = avroize_type(c["type"], c["name"])
  File "/usr/local/lib/python3.6/site-packages/cwltool/process.py", line 420, in avroize_type
    avroize_type(field_type["fields"], name_prefix)
  File "/usr/local/lib/python3.6/site-packages/ruamel/yaml/comments.py", line 680, in __getitem__
    return ordereddict.__getitem__(self, key)
KeyError: 'fields'

Your Environment

• cwltool version: 1.0.20180111185617

[ThomasHickman]

I could have a look at fixing this issue with a pull request, however the CWL 1.0 specification doesn't specify what should happen when no field property is defined. I've added my ideas as to how this should be implemented here: common-workflow-language/common-workflow-language#428 and I'll have a look at fixing it when the expected behaviour is clarified.

[mr-c]

@ThomasHickman thank you for your issue! Can you help us by giving a concrete real world example?

[ThomasHickman]

I'm trying to write a cwl file for an tool called capmq (https://github.com/mcshane/capmq). In capmq, there are options where you can pass arguments to htslib (-I and -O from https://github.com/mcshane/capmq/blob/master/capmq.c#L288). One way to express the type of this parameter would be to use a map e.g. to describe the inputs from example given in the help text of capmq, you would write the following as an input:

htslib_input_options:
  file_format: cram
  lossy_names: true
  seqs_per_slice: 100000

where file_format is a special property that is put first in the output array. From the cwl documentation, it looks like the way to implement an map would be to use type: record without a field parameter defined and using a valueFrom expression for the inputBinding. However, in cwltool, using this results in a KeyError being thrown. A workaround to this is to use the type Any, but this doesn't describe the required type as fully as type: record. If this behaviour is not supported, it may be a good idea to report a more descriptive error than KeyError

[stain]

I thought I could use the fact that files was optional to have a wildcard JSON record field, which I really need to avoid Record-izing the world (which can't cover all of possible JSON), yet now this seems to not be possible to have a nested "any JSON" field because of this bug.

[cwl-bot]

This issue has been mentioned on Common Workflow Language Discourse. There might be relevant details there:

https://cwl.discourse.group/t/creating-config-json-file-from-a-record/172/1