Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support output from cwltool's --pack #103

Closed
mr-c opened this issue May 8, 2017 · 16 comments
Closed

support output from cwltool's --pack #103

mr-c opened this issue May 8, 2017 · 16 comments

Comments

@mr-c
Copy link
Member

mr-c commented May 8, 2017

Example: https://view.commonwl.org/workflows/github.com/ProteinsWebTeam/ebi-metagenomics-cwl/tree/packed/workflows/emg-pipeline-v3-packed.cwl

Or in other words, support visualizing subworkflows in situ

@hmenager
Copy link
Member

hmenager commented Jun 8, 2017

+1 that would be really helpful!

@MarkRobbo
Copy link
Member

Support for subworkflows has improved in v1.1.0 due to cwltool support.

It is now a lot easier for us to implement visualisation of subworkflows in their own clusters in-place if the workflow is packed.

@mr-c
Copy link
Member Author

mr-c commented Aug 11, 2017

@MarkRobbo Or worse case, you can --pack it for the user?

@MarkRobbo
Copy link
Member

@mr-c The problem is with the workflows being packed rather than the opposite.

It is then difficult to give each page a unique URL as # cannot be used due to those parameters not being passed to the server.

Currently the application assumes one URL refers to one cwl file, which is not the case with packed workflows. Though there is room for support in the code if we can come up with a strategy for the URLs.

@mr-c
Copy link
Member Author

mr-c commented Aug 14, 2017

@MarkRobbo Why not URL encode the hash as %23?

@MarkRobbo
Copy link
Member

MarkRobbo commented Aug 14, 2017

@mr-c that's possible and would work in the normal application flow, but users manually typing # into the browser to navigate to different subworkflows would not have it URL encoded which could make that confusing.

Edit: I've also done some quick testing on this with some weird behaviour on the part of the browsers - not sure if this would work as an approach

@mr-c
Copy link
Member Author

mr-c commented Aug 14, 2017

Can't you detect the presence of an unescaped hash and convert appropriately if it's a valid sub document?

@MarkRobbo
Copy link
Member

MarkRobbo commented Aug 14, 2017

@mr-c The only way to read them would be on the client side - I suppose it may be possible having loaded the page to redirect you to the escaped hash page using Javascript, but that is a little messy and there's no way to check if it actually exists at that point without an AJAX call.

As they are used for anchors etc, that information is not sent to the server at all.

@mr-c
Copy link
Member Author

mr-c commented Aug 14, 2017

It is your call -- I think some messy code is often worth a better user experience.

@alexiswl
Copy link

I'm having issues with this.

If I try the following link I get:

https://view.commonwl.org/?url=https://github.com/umccr/cwl-ica/blob/main/.dockstore/dragen-somatic-with-germline-pipeline/4.2.4/dragen-somatic-with-germline-pipeline__4.2.4.packed.cwl.json

Error: No workflow files were found in the given directory

@mr-c
Copy link
Member Author

mr-c commented Oct 26, 2023

I'm having issues with this.

If I try the following link I get:

https://view.commonwl.org/?url=https://github.com/umccr/cwl-ica/blob/main/.dockstore/dragen-somatic-with-germline-pipeline/4.2.4/dragen-somatic-with-germline-pipeline__4.2.4.packed.cwl.json

Error: No workflow files were found in the given directory

Can you try from a branch where the file extension is .cwl?

@kinow
Copy link
Member

kinow commented Oct 26, 2023

Running the viewer locally in IntelliJ/Docker/etc should give more details if there's any exception. My env is currently broken as I was working on the Spring Boot 3 upgrade, but the Docker setup is not too complicated. It starts a Java and a Postgres container locally. Then you try to upload this workflow, wait for the error and check the exception in the logs. Might be something in the code that needs to be fixed and might be getting a nullpointerexception or some other runtime error.

@alexiswl
Copy link

Thanks for the suggestion Bruno (turns out I needed to add #main at the end of my url).

So

https://view.commonwl.org/workflows/github.com/umccr/cwl-ica/blob/test/no-extension-cwl-viewer/.dockstore/dragen-somatic-with-germline-pipeline/4.2.4/dragen-somatic-with-germline-pipeline__4.2.4.packed.json.cwl%23main

gets at-least to here:

image

But then fails shortly after

image

with the error message

Error: cwltool version failed to run on this workflow:

An error occurred when executing a query on the SPARQL store

Using #main worked locally and successfully generated a graph, the logs differ with

cwlviewer-postgres-1  | 2023-10-26 23:42:30.023 UTC [28] DETAIL:  parameters: $1 = '{"url": "https://github.com/umccr/cwl-ica/blob/test/no-extension-cwl-viewer/.dockstore/dragen-somatic-with-germline-pipeline/4.2.4/dragen-somatic-with-germline-pipeline__4.2.4.packed.json.cwl#", "path": "dragen-somatic-with-germline-pipeline__4.2.4.packed.json.cwl#", "type": "GITHUB", "branch": "test/no-extension-cwl-viewer/.dockstore/dragen-somatic-with-germline-pipeline/4.2.4", "rawUrl": "https://raw.githubusercontent.com/umccr/cwl-ica/test/no-extension-cwl-viewer/.dockstore/dragen-somatic-with-germline-pipeline/4.2.4/dragen-somatic-with-germline-pipeline__4.2.4.packed.json.cwl#", "repoUrl": "https://github.com/umccr/cwl-ica.git", "packedId": null}'

to

cwlviewer-postgres-1  | 2023-10-26 23:43:39.136 UTC [28] DETAIL:  parameters: $1 = '{"url": "https://github.com/umccr/cwl-ica/blob/test/no-extension-cwl-viewer/.dockstore/dragen-somatic-with-germline-pipeline/4.2.4/dragen-somatic-with-germline-pipeline__4.2.4.packed.json.cwl#main", "path": "no-extension-cwl-viewer/.dockstore/dragen-somatic-with-germline-pipeline/4.2.4/dragen-somatic-with-germline-pipeline__4.2.4.packed.json.cwl", "type": "GITHUB", "branch": "test", "rawUrl": "https://raw.githubusercontent.com/umccr/cwl-ica/test/no-extension-cwl-viewer/.dockstore/dragen-somatic-with-germline-pipeline/4.2.4/dragen-somatic-with-germline-pipeline__4.2.4.packed.json.cwl", "repoUrl": "https://github.com/umccr/cwl-ica.git", "packedId": "main"}'

(note the packedId is different).

For .packed.cwl.json suffix however, '#main' does not work and the packedId remains as 'null'.

I don't know what is causing the different between my local server and commonwl.org (maybe it's a timeout thing and that is why it can't render the graph).

@alexiswl
Copy link

alexiswl commented Oct 26, 2023

With the ridiculous number of inputs in this pipeline, there isn't really any surprise that this graph doesn't generate anything useful anyway.

image

@mr-c
Copy link
Member Author

mr-c commented Nov 6, 2023

@GlassOfWhiskey sounds like it is time to re-deploy to the server

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants