Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

restart modified host containers #1722

Merged
merged 1 commit into from Sep 21, 2021

Conversation

gthao313
Copy link
Member

@gthao313 gthao313 commented Aug 24, 2021

When a host container is running and its settings have changed,
bottlerocket will restart and apply new changes to this host container.

Issue number:
#1531

Description of changes:
If a host container is running, and its settings have changed, we should restart it so the settings take effect.

Testing done:
start a host-container and change some setting that might affect it like superpower, image URL, proxy....

Control container

source: apiclient set -j '{"host-containers": {"control": {"source": "328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-control:v0.5.0"}}}'

result:

"control":{"source":"328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-control:v0.5.0"}}
Sep 15 23:27:18  ip-192-168-78-239.us-west-2.compute.internal systemd[1]: host-containers@control.service: Succeeded.
Sep 15 23:27:18 ip-192-168-78-239.us-west-2.compute.internal systemd[1]: Stopped Host container: control.
Sep 15 23:27:18  ip-192-168-78-239.us-west-2.compute.internal systemd[1]: Started Host container: control.
Sep 15 23:27:18  ip-192-168-78-239.us-west-2.compute.internal systemd[1]: Reloading.
Sep 15 23:27:18  ip-192-168-78-239.us-west-2.compute.internal host-ctr[19285]: time="2021-08-23T23:50:49Z" level=info msg="pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-control:v0.5.0"
Sep 15 23:27:18  ip-192-168-78-239.us-west-2.compute.internal host-ctr[19285]: time="2021-08-23T23:50:52Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-control:v0.5.0"

Superpower: apiclient set -j '{"host-containers": {"control": {"superpowered": true}}}'

result:

"control":{superpowered":true}}
Sep 15 23:29:08 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: host-containers@control.service: Succeeded.
Sep 15 23:29:08 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Stopped Host container: control.
Sep 15 23:29:08 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Started Host container: control.
Sep 15 23:29:08 ip-192-168-80-31.us-west-2.compute.internal host-ctr[4131]: time="2021-09-15T23:29:08Z" level=info msg="pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-control:v0.5.0"

user-data:

apiclient set -j '{"host-containers": {"control": {"user-data": "eyJzc2giOnsiYXV0aG9yaXplZC1rZXlzIjpbInNzaC1yc2EgQUFBQUIzTnphQzF5YzJFQUFBQURBUUFCQUFBQkFRRHlEVWZteTJxcS9Dbkh1something"}}}'

result

Sep 15 23:37:10 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: host-containers@control.service: Succeeded.
Sep 15 23:37:10 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Stopped Host container: control.
Sep 15 23:37:10 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Started Host container: control

Admin container

source: apiclient set -j '{"host-containers": {"admin": {"source": "328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.7.1"}}}'

result:

{"admin":{"source":"328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.7.1"}}}
Sep 15 23:41:48 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: host-containers@admin.service: Succeeded.
Sep 15 23:41:48 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Stopped Host container: admin.
Sep 15 23:41:48 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Started Host container: admin.
Sep 15 23:41:48 ip-192-168-80-31.us-west-2.compute.internal host-ctr[8932]: time="2021-09-15T23:41:48Z" level=info msg="pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.7.1"

superpowered: When I set to false, I'll unable to access to sheltie. If anyway that I can check the log?
But when I set from false to true, Host-container restart

apiclient set -j '{"host-containers": {"admin": {"superpowered": true}}}'

result:

{"admin":{"superpowered":true}},
Sep 15 23:43:08 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: host-containers@admin.service: Succeeded.
Sep 15 23:43:08 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Stopped Host container: admin.
Sep 15 23:43:08 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Started Host container: admin.

user-data:

apiclient set -j '{"host-containers": {"admin": {"user-data": "eyJzc2giOnsiYXV0aG9yaXplZC1rZXlzIjpbInNzaC1yc2EgQUFBQUIzTnphQzF5YzJFQUFBQURBUUFCQUFBQkFRRHlEVWZteTJxcS9something"}}}'

result

Sep 15 23:44:34 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: host-containers@admin.service: Succeeded.
Sep 15 23:44:34 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Stopped Host container: admin.
Sep 15 23:44:34 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Started Host container: admin.

Proxy:

  1. launch an ubuntu instance
  2. follow these instructions: https://ubuntu.com/server/docs/proxy-servers-squid to set proxy
  3. set the proxy to the public ip of your ubuntu instance settings.network.http_proxy=35.84.178.198: 3128

result

Sep 15 23:50:02 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Stopped Host container: admin.
Sep 15 23:50:02 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Stopped Host container: control.
Sep 15 23:50:02 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Started Host container: admin.
Sep 15 23:50:02 ip-192-168-80-31.us-west-2.compute.internal systemd[1]: Started Host container: control

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@gthao313 gthao313 marked this pull request as ready for review August 30, 2021 17:39
@gthao313
Copy link
Member Author

Currently I have tested host-container settings and proxy setting which I think might affect host-container restart. If you think there are another more settings that would affect host-container restart, Please let me know! I would like to do more test. Thanks.

Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍄

@@ -397,6 +406,10 @@ where
if host_containerd_unit.is_active()? && !systemd_unit.is_enabled()? {
command(constants::HOST_CTR_BIN, &["clean-up", "--container-id", name])?;
}
else if host_containerd_unit.is_active()? && systemd_unit.is_enabled()? {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change should be done in a different part. For a little context, we only want to enable host-containers' units during the multi-user.target execution.

With your current approach, during the execution of preconfigured.target, and with user data as:

[settings.host-containers.control]
source  = "<an-image>"

This condition will be false:

host_containerd_unit.is_active()? && systemd_unit.is_enabled()?

And since we have this conditional:

match command(constants::SYSTEMCTL_BIN, &["get-default"])?.trim().as_ref() {
            "multi-user.target" => {
                debug!("Enabling and starting container: '{}'", unit_name);
                systemd_unit.enable_and_start()?
            }
            _ => {
                debug!("Enabling: '{}'", unit_name);
                systemd_unit.enable()?
            }
};

The unit will be just enabled, not started. Now, say you have a bootstrap container that changes the host-containers' settings, this new condition will now be true, since the unit was enabled during preconfigured.target:

host_containerd_unit.is_active()? && systemd_unit.is_enabled()?

Thus, your change effectively affects when host containers' units are started. I think a better approach is to modify the enable_and_start method, since it will only be executed when multi-user.target is the current target.

I think within that method, you should check if the unit is active with self.active()?, if it is, then you can run your method try_reload_or_restart.

This will keep the consistency of only starting host container units in multi-user.target.

When a host container is running and its settings have changed,
bottlerocket will restart and apply new changes to this host container.
@gthao313
Copy link
Member Author

Push above address @arnaldo2792 's comment which it should only affect containers in the multi-user.target.

Copy link
Contributor

@arnaldo2792 arnaldo2792 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥾

Copy link
Member

@jpculp jpculp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⛰️

Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🖌️

@gthao313 gthao313 merged commit f858d68 into bottlerocket-os:develop Sep 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants