284 add support for arm architecture for singularity build

[munishchouhan]

add arm architecture support for singularity builds using --platform flag

[munishchouhan]

Singularity 4 docker is out, I will move forward with this PR
https://quay.io/repository/singularity/singularity

[munishchouhan]

we have platform in two places while building singularity image
--platform linux/arm64 quay.io/singularity/singularity:v4.0.1-slim sh -c singularity build image.sif /Users/munish.chouhan/main_ground/wave/284/wave/build-workspace/93569c0c19462261/Containerfile --platform linux/arm64

@pditommaso should i create another field in buildrequest to specify singularity platform?

[pditommaso]

Ideally this should be enough

[munishchouhan]

@pditommaso so we should use singularity image when the platform is arm46

[marcodelapierre]

I think multi-arch builds are not supported yet in Singularity 4.0.

The user guide only mentions support for pulling architecture-/platform- specific containers:
https://docs.sylabs.io/guides/4.0/user-guide/singularity_and_docker.html#specifying-a-platform-architecture

The user case for this feature seems to be to provide emulation support to run containers on a certain arch, that were built for a different arch. (I can think e.g. of those x86_64 only containers, that someone may need to run on ARM, RISC or other).

So for now the only available way for multi-arch seems to be docker builds.

[marcodelapierre]

Evoking the one and mighty Vanessasaurus @vsoch :

are multi arch builds with Singularity currently under consideration?

(I browsed issues / discussions in the sylabs singularity repo, but could not find mention of it)

[marcodelapierre]

[update] after some additional study on the command line (still undocumented), I see that singularity build has indeed got --arch and --platform options. The use case, as inferred by the CLI help screen, is to pick the build architecture when using Sylabs remote build service.

However, the machine on which singularity is running needs to have matching architecture.

So I have two questions @munishchouhan :

1. I assume that your current implementation exploit the Wave server capability to route the builds on machines of appropriate architecture, right?

2. if 1. is the case, do we need the --arch flag at all? I would assume (haven't tested it yet myself) that a Singularity build on an ARM machine would build an ARM container by default, right?

Thanks,

[munishchouhan]

Is the support for K8s missing, isn't it? if so please mark draft until isn't complete

This feature needs two changes, one is singularity arm image, and arm node

1. I have made changes for singularity arm image
2. Node selection is already there

[marcodelapierre]

One more request:

please add appropriate testing for the K8s case in src/test/groovy/io/seqera/wave/service/builder/KubeBuildStrategyTest.groovy

[munishchouhan]

One more request:

please add appropriate testing for the K8s case in src/test/groovy/io/seqera/wave/service/builder/KubeBuildStrategyTest.groovy

done

[marcodelapierre]

good to go for me

[pditommaso]

Suggested change

	@Value('${wave.build.singularity-image-arm}')
	String singularityImageArm
	@Value('${wave.build.singularity-image-arm64}')
	String singularityImageArm64

Let's rename to arm64 suffix for consistency.

Also, this can be made nullable, and the default to singularityImage + '-arm64'

[munishchouhan]

done

[pditommaso]

Maybe i'm wrong here, but it looks like that when singularityImageArm64 is not specified null it will be used.

My suggestion was instead that when singularityImageArm64 is not specied (nullable) the value of singularityImage is appending to it the -arm64, suffix.

To avoid duplicating the code, it could make sense to introduce an object BuildConfig (along the same way of SpackConfig) that holds all the wave.build.* settings

[munishchouhan]

But in this way, we are coupling our code with the naming convention of singularity docker image. If somewhere down the line they change this, then our code will be broken

[pditommaso]

True, but still remain the ability to override it.

https://en.m.wikipedia.org/wiki/Convention_over_configuration#:~:text=Convention%20over%20configuration%20(also%20known,repeat%20yourself%20(DRY)%20principles.

[munishchouhan]

ok i understood now, thanks for the clarification

[munishchouhan]

done

[marcodelapierre]

@munishchouhan @pditommaso how are we going here?

[munishchouhan]

@marcodelapierre everything is done from my side

[marcodelapierre]

@pditommaso when you get a chance please have a look at Munish' commits that tackle your request

[pditommaso]

For sake of simplicity I've removed the @nullable on arm64 image name. Let's implement it in the BuildConfig object