Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FreeBSD 12.0 jails dying #89

Open
denisSal opened this issue May 9, 2019 · 3 comments

Comments

@denisSal
Copy link
Member

commented May 9, 2019

In FreeBSD 12.0, sometimes jails cannot be destroyed and removed from the existing jail list, but rather stay in the "dying" state (this can be seen with jls -ad). This shouldn't be a problem except when trying to restart the experiment, when jails are destroyed and created with the same ID.

We could bypass this issue by matching the current experiment ID with the running jail IDs before running every experiment and changing it if it already exists.

@denisSal denisSal added the bug label May 9, 2019

@Firerouge

This comment has been minimized.

Copy link
Contributor

commented May 30, 2019

The problem of repeating jail IDs could be addressed by using random jail IDs in collisions, however; my use case requires repeatedly using the same experiment ID.
To fix one without breaking the other, the change would require supporting cases where experiment and jail ID are no longer the same.

The underlying problem is FreeBSD 12 (stable) doesn't complete deleting jails upon experiment termination.

@denisSal

This comment has been minimized.

Copy link
Member Author

commented Aug 19, 2019

I've been digging a bit on my FreeBSD 12.0 virtual machine so I just wanted to write an update on this issue...

First of all, this is definitely primarily a bug with FreeBSD jails (even without IMUNES). I can replicate it with:

# jail -c name=foo persist
# jexec foo sh
# jail -r foo

There shouldn't be any, but there are dying jails left when I run jls -d. This bug report also shows this.

I accidentally found out that the dying jails disappear when copying/creating a big file on my VM, for example:

# dd if=/dev/zero of=/tmp/bigfile bs=1M count=500

This also works with IMUNES dying jails. Of course, this isn't a fix or anything, I just wanted to acknowledge this 'hack' (because this and this tells us that there is no fix for this, only that 'keeping the same JID or ... recycling freed JID is not recommended' and to just 'ignore it'. :)

This and again this suggest that there is a problem with credentials holding a jail.

To fix one without breaking the other, the change would require supporting cases where experiment and jail ID are no longer the same.

This seems like a correct solution to the IMUNES issue. I will look into this but it could be a bigger challenge for me to test and make sure I don't break anything so it could take a while.

@denisSal

This comment has been minimized.

Copy link
Member Author

commented Aug 29, 2019

This patch resolves the issue, and jails now don't remain in the dying state:
patch.zip

WARNING: I am not a FreeBSD developer and this is not an official patch, but according to this it should be ok to revert this one line to an older revision.

How to patch:

# cd /tmp/
# unzip patch.zip
# cd /usr/src
# patch < /tmp/vfs_bio.patch
# cd sys/amd64/conf # or i386
# config GENERIC
# cd ../compile/GENERIC
# make -j4 depend && make -j4
# make -j4 install
# reboot
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.