-
-
Notifications
You must be signed in to change notification settings - Fork 14k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
singularity-tools: refactor and add definition-based image buider #224636
base: master
Are you sure you want to change the base?
Conversation
This builder takes a list `path` and reduces the `writeMultipleReferencesToFile` result of each element to a file. It would be helpful when building a container image from a list of packages.
Questions:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional questions about the original implementation.
It's the contents of the
Not quite, it's executed inside the container via a chroot, but has the host store available. So it allows arbitrary setup of things outside of the nix store prior to instantiating the container specific store. This is analagous to dockerTools.
Not sure how much isolation affects the process of building an image, more of a runtime thing.
One thing to consider is that many places have namespaces disabled so lighter alternatives are not possible. The squashfs approach of @posch is great and wasn't possible when I wrote the initial implementation, but runAsRoot isn't possible with this. This probably doesn't matter for a majority of use cases. |
What if we have user namespaces? posch's script is great, but it's a bit hacky as the neither Apptainer nor SingularityCE have published the specification about the image format. (Hand-made sandbox build also has similar problem.) |
(I haven't processed all of the previous notifications, but) I just wanted to note that this isn't too often the case for HPC clusters (Aalto's "Triton" only offers setuid)... |
Split the build steps used by buildImage into * buildSandboxFromContents * buildImageFromSandbox * buildImageInLinuxVM
5a88f78
to
53a1065
Compare
Just noticed that the layout of an Apptainer/Singularity "sandbox" is very similar to the extracted content of a SIF image. If we could figure out a way to create a sandbox, maybe we could just pack it into a squashfs image. On the other hand, I'm facing a strange |
"${coreutils}/bin/ln" -s "${coreutils}/bin/env" "''${SINGULARITY_ROOTFS}/usr/bin/env" | ||
''; | ||
environment = { | ||
PATH = "${lib.makeBinPath contents}:\${PATH:-}"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose this is the alternative to the previous approach of generating a FHS-like tree (/bin
and such). We can address this in a separate PR, but I wanted to explore (ab)using ext3 and populate both /nix/store
and /bin
using hardlinks. This way I could hide the container's /nix
with --bind /tmp:/nix
and still having working sh
and nix
(assuming I've built them statically linked)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose this is the alternative to the previous approach of generating a FHS-like tree (
/bin
and such).
Yes. The PATH
-based approach seems cleaner for me, which is also how nix shell
works. As both of them has its use cases (Nixy idiom vs. FHS PATH compatibility), we could allow users to decide which to use.
We can address this in a separate PR, but I wanted to explore (ab)using ext3 and populate both
/nix/store
and/bin
using hardlinks. This way I could hide the container's/nix
with--bind /tmp:/nix
and still having workingsh
andnix
(assuming I've built them statically linked)
HPC might be based upon a distributed filesystems; great chance that they don't support hard links. Former versions of Singularity simply extracts the SquashFS image on the host. If the machine have some sort of TMPDIR where local filesystems like ext4 or tmpfs are mounted, you could have it extract there and (hopefully) enjoy the hard links. If you don't have access hard-link supported spaces, there might be problems extracting an image like that.
A good news is that both Apptainer and SingularityCE now tries to mount the image with the squashfuse
command from the environment before falling back to extracting, and that SquashFS does support hard links on its own. I added squashfuse
to the defaultPath and wrapper PATH of both apptainer
and singularity
along with the version bump #224683. Let's make it land on the stable release of Nixpkgs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
great chance that they don't support hard links
I guess I don't really care if they're extracted as copies on the host (in fact, only one would be extracted, and the other would get hidden), as long as they are represented as hardlinks inside the image that we create.
I added squashfuse to the defaultPath and wrapper PATH of both apptainer and singularity along with the version bump
Awesome! I should have a look tomorrow!
Let's make it land on the stable release of Nixpkgs
Sounds great! But just in case it takes longer, we can and should backport later
Today I've tried to use "apptainer build" instead of "mksquashfs" to create the container from the sandbox-like tree. It worked with only a small change to apptainer (see apptainer/apptainer#1312 ). Maybe that's a little less hacky. |
Great! That would be more likely to be megred by the upstrem than mine (apptainer/apptainer#1284). |
During the work on #224828, I found that As most of the distributions (I have heard of) uses an interactive shell as its |
Right now it seems like the contents of `.singularity.d/runscript.sh are overwritten by singularity/apptainer |
Description of changes
Status: work in progress
The original
buildImage
implementation is commented out to make discussion easier.Also update trivial-builders.nix:
writeMultipleReferencesToFile: init (#178717)
writeScriptBin, writeShellScriptBin: add meta.mainProgram automatically
Things done
sandbox = true
set innix.conf
? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"
. Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/
)