Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add several checks in the code to detect containers unavailable for the current hardware architecture #34

Open
jmfernandez opened this issue Jan 30, 2023 · 0 comments
Assignees

Comments

@jmfernandez
Copy link
Member

Thanks to the tests from @dcl10 some issues have been uncovered related to workflows which depend on container images which are not available for the current processor architecture.

A way to reproduce the chain of issues is trying to execute cosifer workflow, which depends on a single container prepared for x86_64 / amd64 architecture, in a different architecture like linux arm64.

cosifer "toy" workflow uses a single custom container which is only available for x86_64. WfExS-backend tries materializing the container by itself, and most probably it is doing wrongly despite the architecture mismatch, but it should have complained before even trying to run cwltool. So, when cwltool tries running it, it is surely failing because either the previously materialized container is for the wrong architecture, or because cwltool is not able to fetch any container suitable for the task. So, cwltool is returning an empty description of its outputs which is deserialized to None instead to a dictionary, and the code is failing trying to access key "class" because None is not a dictionary.

Also, the caching directory should have container images directory per supported architecture, so it can hold cached versions for x86_64 and arm64, in case the caching directory is used in an heterogeneous HPC environment.

jmfernandez added a commit that referenced this issue Jun 6, 2023
@jmfernandez jmfernandez self-assigned this Jun 6, 2023
jmfernandez added a commit that referenced this issue Jun 7, 2023
…ssor architecture and operating system.

This one is related to issue #34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant