New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ruleset exists but iocage does not find it #952
Comments
For fun, I did this:
Ruleset 10 does not exist in Restarting the jail results and there is no 'does not exist' exist message, despite the fact that the ruleset does not in fact exist.
Next, let us renumber the ruleset in the file:
Restarting the jail, and again, no message about 'does not exist', which is correct this time. 2019/06/17 13:50:50 (INFO) * Stopping toiler Now My questions:
|
I recommend leaving that on the default 4 with dhcp on, as iocage will handle the devfs rulesets for you and unhide the same devices. |
From man poudriere:
Do we both understand this the same way? This jail has a static IP address. dhcpd runs inside this jail. I think the iocage dhcp configuration setting is not related to this use case. |
In which case just set the bpf property and vnet. Don't set dhcp. iocage will handle the rulesets for you as long as the property is at the default (4) |
Here is something interesting:
Wait... I changed it to 4, then back to 10, and it's already 10? Let's try it again:
OK, let's look in there again:
|
Trying that again, it seems to work. WTF is going on?
|
That's because iocage get will return the dynamic ruleset, you know this as you commented in the issue (#694) :P |
TLDR; Set bpf and vnet to 1, set devfs_ruleset to 4 and enjoy it working as it should. |
You attribute much to me but I have no memory of that one. |
;) Well it did indeed happen lol |
Yeah, but expecting people to recall obscure bits is obscure. |
I'm off to configure vnet for this dhcp jail. |
"VNET is considered experimental. Unexpected system crashes can occur. " - https://iocage.readthedocs.io/en/latest/networking.html Wait, what? I must now use this? Is there another way? |
That's for earlier RELEASEs, it's stabilized significantly since then. To pass the bpf device in, vnet AFAIK is required. |
This jail has used bpf since it was created. The requirement to use vnet must be new. |
The interesting part here:
The way it takes down ix0:
I'm guessing that is because 10.55.0.13 is static on ix0 and must now be dynamic. OK, I can do that.
Add the IP addresses back in:
But still:
I am not sure why this fails at all. |
Do you have defaultrouter set? |
Yes, it is set to my gateway.
|
I have no idea what this should look like:
|
I am reading https://iocage.readthedocs.io/en/latest/networking.html The examples under VIMAGE/VNET have references to what is required in configuration files but no mention of how to invoke that configuration item from the command line. bridge0 already existed when I got here. That is likely related to:
Let's try what I found at https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-bridging.html
Oh, I have to take ix0 down for this? |
It’s because it’s already a member of bridge0 (iocage handles the creation and teardown of the bridges you supply with the interfaces prop).
Brandon
…On 24 Jun 2019, at 14:48, Dan Langille wrote:
I am reading https://iocage.readthedocs.io/en/latest/networking.html
The examples under VIMAGE/VNET have references to what is required in configuration files but no mention of how to invoke that configuration item from the command line.
bridge0 already existed when I got here. That is likely related to:
```
cloned_interfaces="lo1"
```
Let's try what I found at https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-bridging.html
```
$ sudo ifconfig bridge create
bridge1
$ ifconfig bridge1
bridge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:ca:b7:29:ce:01
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 0 ifcost 0 port 0
groups: bridge
nd6 options=1<PERFORMNUD>
$ sudo ifconfig bridge1 addm ix0
ifconfig: BRDGADD ix0: Device busy
```
Oh, I have to take ix0 down for this?
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#952 (comment)
|
Oh. Is the following required?
Where em0 is my main interface? ix0 in my case. I ask because that's what the documentation says. |
In most circumstances no, but that documentation is for users who prefer to do things the manual way. iocage grew support for doing that automated a little while ago.
Brandon
…On 24 Jun 2019, at 14:55, Dan Langille wrote:
Oh. Is the following required?
```
# set up bridge interface for iocage
cloned_interfaces="bridge0"
# plumb interface em0 into bridge0
ifconfig_bridge0="addm em0 up"
ifconfig_em0="up"
```
Where em0 is my main interface? ix0 in my case. I ask because that's what the documentation says.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#952 (comment)
|
I have the jail up and I can There is no firewall on this host.
If I go into the jail via I think I am missing a step. |
Got it.
Then:
NOTE the lack of UPS messages. The missing mask is the key. |
This is py36-iocage-devel-1.0.0.20190801,1 (built from master on that date) After rebooting this server today, no jails started up:
Trying manually, to reproduce the message:
I first searched the code to fix that whitespace typo, see #1015 Based on the previous conversation, I thought iocage would take care of the bridge, hence:
So I did this:
Reading above, I thought this would work:
|
It looks like
So much for not starting the jails one by one. |
OK, that's the other jails started, let's see if I can figure this problem out. |
I've been playing in the code. get_interface_bridge_map() outputs: vnet0 bridge0 interface is: ix0 It seems like ix0 should be in the map too. This seems OK though:
|
That is exactly what is happening:
|
cloned/copied into dynamic ones. This makes devfs rule handling more symmetrical to what is done when the default ruleset 4 is configured (which in fact never applies devfs ruleset 4, but creates an iocage specific dynamic ruleset instead - which can be quite confusing). As a result, this addresses the problem of non-dynamic rulesets being removed on `iocage stop` raised in iocage#952. This also makes iocage fail to start a jail if the configured devfs ruleset is not available - beforehand it would start with a default ruleset in this case, which can have severe unwanted side effects. Finally, this sets the minimum dynamically assigned devfs rule id to 1000 to reserve the lower ids for static configuration by the admin in devfs.rules (this is mostly for convenience).
cloned/copied into dynamic ones. This makes devfs rule handling more symmetrical to what is done when the default ruleset 4 is configured (which in fact never applies devfs ruleset 4, but creates an iocage specific dynamic ruleset instead - which can be quite confusing). As a result, this addresses the problem of non-dynamic rulesets being removed on `iocage stop` raised in iocage#952. This also makes iocage fail to start a jail if the configured devfs ruleset is not available - beforehand it would start with a default ruleset in this case, which can have severe unwanted side effects. Finally, this sets the minimum dynamically assigned devfs rule id to 1000 to reserve the lower ids for static configuration by the admin in devfs.rules (this is mostly for convenience).
@dlangille I created a pull request that fixes the issue by creating dynamic rules from configured static rules, please see #1106 for details. It also stops jail start in case a configured devfs rule isn't available (before it would start with default rules instead). For a quick fix/testing, download and apply the raw patch to ioc_common.py. |
Thank you for this work. I appreciate it. This is the result. EDIT: this was an invalid test. The jail must be started with the patched code. It was not.
|
I will retest. I think the above was an invalid test. |
After the above, I did this:
Success. Thank you. |
@dlangille Cool, thanks for testing/confirming. Could you maybe also check |
cloned/copied into dynamic ones. This makes devfs rule handling more symmetrical to what is done when the default ruleset 4 is configured (which in fact never applies devfs ruleset 4, but creates an iocage specific dynamic ruleset instead - which can be quite confusing). As a result, this addresses the problem of non-dynamic rulesets being removed on `iocage stop` raised in iocage#952. This also makes iocage fail to start a jail if the configured devfs ruleset is not available - beforehand it would start with a default ruleset in this case, which can have severe unwanted side effects. Finally, this sets the minimum dynamically assigned devfs rule id to 1000 to reserve the lower ids for static configuration by the admin in devfs.rules (this is mostly for convenience).
I picked the
Yeah, that's not right is it?
Oh. Same thing. Oh.. |
@dlangille Well, what is that jail's configuration? I assume that jail has ruleset=4 configured (it was still running from before the update, that's why it had rule id 6). So having the same rules here makes sense, right? |
The jail is indeed configured with devfs_ ruleset=4 But given that, I would expect:
|
Perhaps this is the result of the clone? |
Well, that's why I wrote in #1106 :
This isn't something I introduced, that's how iocage behaved already (as you can compare with an unpatched version). That's why it never killed ruleset 4, but only custom ones: It always creates a dynamic custom ruleset if "4" is magically configured, but it didn't if you defined one yourself. The dynamic one was removed on stop, which didn't cause any harm (as it's not 4), but it caused harm with any other id. Therefore, any changes to ruleid 4 in devfs.rules has (and had) no impact whatsoever. That's a problem in itself, but I can't solve this, as it seems really hard to migrate away from it. My patch basically treats statically configured ruleset ids != 4 the same - copy and create a dynamic one, which then can be safely disposed. So what I was hoping for was a test like described with a custom ruleset != 4 configured. p.s. There's also a problem with |
@dlangille The unpatched/unaltered code in question can be found here: iocage/iocage_lib/ioc_common.py Lines 751 to 818 in 793fae1
There you can see how a ruleset configuration of "4" receives special treatment by synthesising a ruleset that's not related to what is actually configured as ruleset 4 in devfs. This ruleset is later deleted on My patch alters this by cloning the configured devfs_ruleset into a dynamic one, so that the cleanup mechanism works the same as for ruleset "4". It only does that for anything != 4 though, as it otherwise would break existing iocage installations by replacing what iocage does with what a default FreeBSD install has in devfs.rules. That's why my patch doesn't change any behaviour for iocage has that default behaviour for a reason. I'm unhappy that they used "4" as a magic number for that (it feels wrong on so many different levels and is super confusing), but that's just what it is. |
Or in pseudo-code: Original behaviour:
New behaviour:
@dlangille Does this make it more clear/understandable? |
I follow. Thank you. |
@dlangille can you please confirm if it's safe to close this issue as I think @grembo's patch fixes your concern ? ( Thank you @grembo btw for the contribution ;) ) |
Confirmed, I have had no recurrence of this issue. |
Thank you for confirming @dlangille. Closing this as #1106 will fix this. |
…rulesets != 4 are (#1106) * Change devfs ruleset handling so that configured rulesets != 4 are cloned/copied into dynamic ones. This makes devfs rule handling more symmetrical to what is done when the default ruleset 4 is configured (which in fact never applies devfs ruleset 4, but creates an iocage specific dynamic ruleset instead - which can be quite confusing). As a result, this addresses the problem of non-dynamic rulesets being removed on `iocage stop` raised in #952. This also makes iocage fail to start a jail if the configured devfs ruleset is not available - beforehand it would start with a default ruleset in this case, which can have severe unwanted side effects. Finally, this sets the minimum dynamically assigned devfs rule id to 1000 to reserve the lower ids for static configuration by the admin in devfs.rules (this is mostly for convenience). * Change devfs_ruleset config parsing on jail start, so that: - Users get a helpful error message in case a configured devfs_ruleset doesn't exist (also shows the configured ruleset and not the dynamically created one, which was not helpful). - Users learn on jail start about how the devfs_ruleset is created (show id it was cloned from or that it is based on iocage's default). - Avoid leaking devfs_rulesets on starting plugins that define devfs_paths/devfs_includes (would lose one ruleset everytime otherwise). - Show a warning in case a plugin with devfs_paths/devfs_includes is started with a manually configured devfs_ruleset (as this won't - and never did - apply those. - Move magic numbers to constants in ioc_common.py. This doesn't fix the iocage man page, which shows (and had shown) inaccurate information on this for a while. * Move "min_dyn_devfs_ruleset" to jail property, change jail start devfs_ruleset message to be one-liner containing information on configured devfs ruleset (configured or iocage generated). * Validate 'devfs_ruleset' and 'min_dyn_devfs_ruleset' while setting.
cloned/copied into dynamic ones. This makes devfs rule handling more symmetrical to what is done when the default ruleset 4 is configured (which in fact never applies devfs ruleset 4, but creates an iocage specific dynamic ruleset instead - which can be quite confusing). As a result, this addresses the problem of non-dynamic rulesets being removed on `iocage stop` raised in iocage#952. This also makes iocage fail to start a jail if the configured devfs ruleset is not available - beforehand it would start with a default ruleset in this case, which can have severe unwanted side effects. Finally, this sets the minimum dynamically assigned devfs rule id to 1000 to reserve the lower ids for static configuration by the admin in devfs.rules (this is mostly for convenience). (cherry picked from commit 30c8f2d)
…rulesets != 4 are (iocage#1106) * Change devfs ruleset handling so that configured rulesets != 4 are cloned/copied into dynamic ones. This makes devfs rule handling more symmetrical to what is done when the default ruleset 4 is configured (which in fact never applies devfs ruleset 4, but creates an iocage specific dynamic ruleset instead - which can be quite confusing). As a result, this addresses the problem of non-dynamic rulesets being removed on `iocage stop` raised in iocage#952. This also makes iocage fail to start a jail if the configured devfs ruleset is not available - beforehand it would start with a default ruleset in this case, which can have severe unwanted side effects. Finally, this sets the minimum dynamically assigned devfs rule id to 1000 to reserve the lower ids for static configuration by the admin in devfs.rules (this is mostly for convenience). * Change devfs_ruleset config parsing on jail start, so that: - Users get a helpful error message in case a configured devfs_ruleset doesn't exist (also shows the configured ruleset and not the dynamically created one, which was not helpful). - Users learn on jail start about how the devfs_ruleset is created (show id it was cloned from or that it is based on iocage's default). - Avoid leaking devfs_rulesets on starting plugins that define devfs_paths/devfs_includes (would lose one ruleset everytime otherwise). - Show a warning in case a plugin with devfs_paths/devfs_includes is started with a manually configured devfs_ruleset (as this won't - and never did - apply those. - Move magic numbers to constants in ioc_common.py. This doesn't fix the iocage man page, which shows (and had shown) inaccurate information on this for a while. * Move "min_dyn_devfs_ruleset" to jail property, change jail start devfs_ruleset message to be one-liner containing information on configured devfs ruleset (configured or iocage generated). * Validate 'devfs_ruleset' and 'min_dyn_devfs_ruleset' while setting.
…rulesets != 4 are (iocage#1106) * Change devfs ruleset handling so that configured rulesets != 4 are cloned/copied into dynamic ones. This makes devfs rule handling more symmetrical to what is done when the default ruleset 4 is configured (which in fact never applies devfs ruleset 4, but creates an iocage specific dynamic ruleset instead - which can be quite confusing). As a result, this addresses the problem of non-dynamic rulesets being removed on `iocage stop` raised in iocage#952. This also makes iocage fail to start a jail if the configured devfs ruleset is not available - beforehand it would start with a default ruleset in this case, which can have severe unwanted side effects. Finally, this sets the minimum dynamically assigned devfs rule id to 1000 to reserve the lower ids for static configuration by the admin in devfs.rules (this is mostly for convenience). * Change devfs_ruleset config parsing on jail start, so that: - Users get a helpful error message in case a configured devfs_ruleset doesn't exist (also shows the configured ruleset and not the dynamically created one, which was not helpful). - Users learn on jail start about how the devfs_ruleset is created (show id it was cloned from or that it is based on iocage's default). - Avoid leaking devfs_rulesets on starting plugins that define devfs_paths/devfs_includes (would lose one ruleset everytime otherwise). - Show a warning in case a plugin with devfs_paths/devfs_includes is started with a manually configured devfs_ruleset (as this won't - and never did - apply those. - Move magic numbers to constants in ioc_common.py. This doesn't fix the iocage man page, which shows (and had shown) inaccurate information on this for a while. * Move "min_dyn_devfs_ruleset" to jail property, change jail start devfs_ruleset message to be one-liner containing information on configured devfs ruleset (configured or iocage generated). * Validate 'devfs_ruleset' and 'min_dyn_devfs_ruleset' while setting. (cherry picked from commit a69d58b)
…rulesets != 4 are (#1106) * Change devfs ruleset handling so that configured rulesets != 4 are cloned/copied into dynamic ones. This makes devfs rule handling more symmetrical to what is done when the default ruleset 4 is configured (which in fact never applies devfs ruleset 4, but creates an iocage specific dynamic ruleset instead - which can be quite confusing). As a result, this addresses the problem of non-dynamic rulesets being removed on `iocage stop` raised in #952. This also makes iocage fail to start a jail if the configured devfs ruleset is not available - beforehand it would start with a default ruleset in this case, which can have severe unwanted side effects. Finally, this sets the minimum dynamically assigned devfs rule id to 1000 to reserve the lower ids for static configuration by the admin in devfs.rules (this is mostly for convenience). * Change devfs_ruleset config parsing on jail start, so that: - Users get a helpful error message in case a configured devfs_ruleset doesn't exist (also shows the configured ruleset and not the dynamically created one, which was not helpful). - Users learn on jail start about how the devfs_ruleset is created (show id it was cloned from or that it is based on iocage's default). - Avoid leaking devfs_rulesets on starting plugins that define devfs_paths/devfs_includes (would lose one ruleset everytime otherwise). - Show a warning in case a plugin with devfs_paths/devfs_includes is started with a manually configured devfs_ruleset (as this won't - and never did - apply those. - Move magic numbers to constants in ioc_common.py. This doesn't fix the iocage man page, which shows (and had shown) inaccurate information on this for a while. * Move "min_dyn_devfs_ruleset" to jail property, change jail start devfs_ruleset message to be one-liner containing information on configured devfs ruleset (configured or iocage generated). * Validate 'devfs_ruleset' and 'min_dyn_devfs_ruleset' while setting.
This worked before upgrading
iocage
today and upgrading the jail to12.0-RELEASE-p5
Make sure to follow and check these boxes before submitting an issue! Thank you.
The above is head from 2019-06-16:
Today I upgraded
iocage
from 1.0.0.20190519_1,1When the jail is started, the rule set is not invoked/found:
NOTE the 'does not exist' in the log extra above.
Within
/etc/devfs.rules
we haveAt present, dhcpd in the jail cannot be started because:
Jun 17 01:18:07 toiler dhcpd[61983]: No bpf devices. Please read the README section for your operating system.
The text was updated successfully, but these errors were encountered: