Enable a limited tmpfs for shared memory

[jb3]

This PR introduces a small 40mb tmpfs mounted at /dev/shm to allow shared memory, utilised by multiprrocessing to work. This means that multiprocessing.Pool should work in snekbox (and we have a new test that verifies this).

I'd appreciate if anyone would like to test that you try break the 40mb limit and other general trickery surrounding these new capabilities.

[coveralls]

coverage: 90.947%. remained the same when pulling 08e7636 on jb3/shared-mem into afda301 on main.

[jchristgit]

This looks good to me.

As for your request to test breaking the memory limit, writing there using multiprocessing.shared_memory would have also been my one and only test, and since we're doing that here, I personally can't think of another possibility to break this. Since it says "File too large", what happens if multiple shared_memory.SharedMemory objects are created that exceed the limit together? I would assume that the first file exceeding the limit hit the error?

[MarkKoz]

Someone made me realise that shared memory could be accessed across NsJail instances. Can you think of a way have isolated mounts for each instance?

[jb3]

How could that happen? We create a new tmpfs for every nsjail instance — it's not shared.

[MarkKoz]

How could that happen? We create a new tmpfs for every nsjail instance — it's not shared.

Is it really a new one or is it just mounting the same directory over and over?

[MarkKoz]

Is that what is_bind false does?

[jb3]

Is it really a new one or is it just mounting the same directory over and over?

snekbox/config/snekbox.cfg

Lines 107 to 113 in e6cf1c4

	mount {
	dst: "/dev/shm"
	fstype: "tmpfs"
	rw: true
	is_bind: false
	options: "size=40m"
	}

If this works the same way that it would in the equivalent fstab, it will be a new tmpfs instance for each execution. I'm having trouble reading through how nsjail actually allocates this though, I can't tell if it is doing something special or just mounting a filesystem in the typical way.

Is that what is_bind false does?

is_bind enables:

• MS_BIND ("A bind mount makes a file or a directory subtree visible at another point within the single directory hierarchy")
• MS_REC ("Used in conjunction with MS_BIND to create a recursive bind mount, and in conjunction with the propagation type flags to recursively change the propagation type of all of the mounts in a subtree.")
• MS_PRIVATE ("Make this mount point private. Mount and unmount events do not propagate into or out of this mount point.")

I'm not sure any of these are required, it might be a dud option for tmpfs mounts.

[MarkKoz]

Are these standard semantics for tmpfs? For the sake of my curiosity, is it even possible to mount the same tmpfs under different directories?

[jb3]

You can achieve that through symlinking, I don't think there is a "same tmpfs" due to the nature of tmpfs allocation (a new tmpfs for every mountpoint, as far as I understand).

[MarkKoz]

Okay, great. Just need to address the comments I left a couple weeks ago.

[wookie184]

Thoughts on adding a test that creates lots of empty directories to ensure there are limits on that? Memory taken up by them doesn't count towards the tmpfs size afaict. Currently the limit seems to be due to it hitting cgroup_mem_max, which is fine, although we could also consider setting nr_inodes in the mount options to explicitly limit that if we wanted to.

[MarkKoz]

I don't know why it needs so much overhead 🤷 but in any case, the point of this test isn't to test the normal NsJail OOM killer — there are other tests for that.

[MarkKoz]

@wookie184 Testing for inode limits (and having inode limits) sounds like a good idea. Do you mind writing up a new issue for that? I think that situation would also apply to our temporary file system feature.

[wookie184]

Great stuff, thanks for working on this

All resolved