Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should enums with identical structure but defined in separate locations be considered the same type? #308

Closed
michael-kotliar opened this issue Feb 27, 2017 · 8 comments

Comments

@michael-kotliar
Copy link
Contributor

Hi all!
Found Type mismatch between source and sink when trying to pass enum from workflow to task.
To run the workflow: cwl-runner --debug workflow.cwl job.json

workflow.cwl

class: Workflow
cwlVersion: v1.0

requirements:
  - class: StepInputExpressionRequirement

inputs:

  enum_workflow_input:
    type:
      name: "foo"
      type: enum
      symbols: ["a","b","c"]

outputs: []

steps:
  task:
    in:
      enum_task_input: enum_workflow_input
    out: []
    run: task.cwl

task.cwl

class: CommandLineTool
cwlVersion: v1.0
inputs:

  enum_task_input:
    type:
      name: "foo"
      type: enum
      symbols: ["a","b","c"]

outputs: []

arguments: [echo, $(inputs.enum_task_input) ]

job.json

{
  "enum_workflow_input": "b"
}

Output log

/home/michael_kotliar/.local/bin/cwl-runner 1.0.20170224141733
Resolved 'workflow.cwl' to 'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl'
[workflow workflow.cwl] initialized from file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl
[workflow workflow.cwl] workflow starting
[step task] Cannot make job: Type mismatch between source 'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input' (ordereddict([('name', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input/foo'), ('type', 'enum'), ('symbols', [u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input/foo/a', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input/foo/b', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input/foo/c'])])) and sink 'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#task/enum_task_input' (ordereddict([('name', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/task.cwl#enum_task_input/foo'), ('type', 'enum'), ('symbols', [u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/task.cwl#enum_task_input/foo/a', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/task.cwl#enum_task_input/foo/b', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/task.cwl#enum_task_input/foo/c'])]))

Traceback (most recent call last):
  File "/home/michael_kotliar/python/lib/python2.7/site-packages/cwltool/workflow.py", line 391, in job
    for newjob in step.iterable:
  File "/home/michael_kotliar/python/lib/python2.7/site-packages/cwltool/workflow.py", line 264, in try_make_job
    self.state, inputparms, False, supportsMultipleInput, "source")
  File "/home/michael_kotliar/python/lib/python2.7/site-packages/cwltool/workflow.py", line 174, in object_from_state
    inp["type"]))
WorkflowException: Type mismatch between source 'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input' (ordereddict([('name', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input/foo'), ('type', 'enum'), ('symbols', [u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input/foo/a', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input/foo/b', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#enum_workflow_input/foo/c'])])) and sink 'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/workflow.cwl#task/enum_task_input' (ordereddict([('name', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/task.cwl#enum_task_input/foo'), ('type', 'enum'), ('symbols', [u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/task.cwl#enum_task_input/foo/a', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/task.cwl#enum_task_input/foo/b', u'file:///mnt/data/HOME/michael_kotliar/workspaces/cwl_ws/cwl_sandbox/enum/task.cwl#enum_task_input/foo/c'])]))
[workflow workflow.cwl] outdir is /tmp/tmppOsPjo
Removing intermediate output directory /tmp/tmppOsPjo
{}
Final process status is permanentFail
@qiukunlong
Copy link
Contributor

I met this too. Is this fixed?

tetron added a commit that referenced this issue Jul 23, 2017
…kflow-test

Add test for embedded subworkflow
@serge2016
Copy link

Hello! Anybody here? I have the same behavior:

  1. A tool with non-required enum input.
  2. A Workflow with the same enum input to pass it to the step --> tool.
    Only while such inputs are not required the validation is passed (with warnings), but if I make this input required, then it's a validation error. It seems to me, that symbols inside the enum should not be converted to URIs, they should stay unchanged in the enum tpye.

@mr-c
Copy link
Member

mr-c commented Dec 10, 2017

Hello all,

Sorry for the delay. I invite @tetron and @Stian to discuss if this is a bug or not.

One approach is to externalize the type definition and import it into each CWL document that needs to refer to it:

enum.yaml

name: foo
type: enum
symbols: [a, b, c]

enum-task.cwl

#!/usr/bin/env cwl-runner
cwlVersion: v1.0
class: CommandLineTool

requirements:
  SchemaDefRequirement:
    types:
     - $import: enum.yaml

inputs:
  enum_task_input: enum.yaml#foo

outputs: []

arguments: [echo, $(inputs.enum_task_input) ]

enum-workflow.cwl

#!/usr/bin/env cwl-runner
cwlVersion: v1.0
class: Workflow

requirements:
  SchemaDefRequirement:
    types:
     - $import: enum.yaml

inputs:
  enum_workflow_input: enum.yaml#foo

outputs: []

steps:
  task:
    in:
      enum_task_input: enum_workflow_input
    out: []
    run: enum-task.cwl
$ cwltool --validate enum-workflow.cwl 
/home/michael/cwltool/env/bin/cwltool 1.0.20171205210500
Resolved 'enum-workflow.cwl' to 'file:///home/michael/cwltool/enum-workflow.cwl'
Tool definition is valid

Here is the relevant section of the user guide (which did not exist when this question was originally asked): http://www.commonwl.org/user_guide/19-custom-types/

[edited to remove the un-needed StepInputExpressionRequirement ]

@mr-c mr-c changed the title Type mismatch when using enum in workflow Should enums with identical structure but defined in separate locations be considered the same type? Dec 10, 2017
@mr-c
Copy link
Member

mr-c commented Dec 10, 2017

Since we have a workaround I am going to close this issue and we can continue at common-workflow-language/common-workflow-language#568 to discuss CWL's syntax and model for types

@mr-c mr-c closed this as completed Dec 10, 2017
@serge2016
Copy link

@mr-c, sorry, but this doesn't look like a solution! It's not obvious to use 2 non-standard requirements to get expected behaviour!

@mr-c
Copy link
Member

mr-c commented Dec 10, 2017

@serge2016 The StepInputExpressionRequirement was un-needed. I've updated my comment above.

I agree that this could be cleaner, so let's continue that discussion over in the CWL specification repository common-workflow-language/common-workflow-language#568

@serge2016
Copy link

@mr-c, ok, thank you.

@mr-c
Copy link
Member

mr-c commented Dec 11, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants