Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Container hostname from the snapshot #727

Open
kirs opened this issue Jun 25, 2019 · 4 comments

Comments

Projects
None yet
3 participants
@kirs
Copy link

commented Jun 25, 2019

Hey there. Thanks a lot for your work on CRIU and especially on the container aspect of it.

I'm experimenting with making a checkpoint on one host and doing docker create + docker start --checkpoint on another host, to use CRIU in order to optimize the container start time.

I've noticed that CRIU / Docker ignores the hostname of the new container and forces it to use the hostname from the original (checkpointed) container:

# create the donor
$ docker run --name fat-app-original --hostname original fat-app:latest <entrypoint>
$ docker checkpoint create fat-app-original checkpoint-1

# instantiate the clone
$ docker create --name fat-app-clone --hostname clone fat-app:latest <entrypoint>

$ cp -r /var/lib/docker/containers/<id-of-donor>/checkpoints/checkpoint-1 /var/lib/docker/containers/<id-of-clone>/checkpoints/checkpoint-1

$ docker start --checkpoint=checkpoint-1 fat-app-clone

# the actual hostname gets inferred from the checkpoint:
$ docker exec <id-of-cloned-container> hostname
original

# however docker inspect shows the intended hostname:
$ docker inspect <id-of-cloned-container>  | grep Hostname
        "HostnamePath": "/var/lib/docker/containers/867c560097618a7d0cdacc236e599b49456005508b358a896a70d66ab3cbfc9b/hostname",
            "Hostname": "clone",

Is this a bug or the intentional design?
I'd expect the new hostname to take the preference for the clone.

I understand this might be not the CRIU bug by itself. Feel free to redirect me to the more appropriate project (moby/runc/containerd?).

Thanks!

@adrianreber

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

The following is not verified but from how I expect CRIU to work.

When CRIU checkpoint and restores a container it usually handles all the related namespace. One of the namespace usually involved is the UTS namespace which handles the hostname.

Upon restore CRIU restores the UTS namespace just as it used to be, that is why you get the old hostname. An example where CRIU does not restore the namespace but where CRIU restores into an existing namespace is Podman and the network namespace. So it could be implemented that CRIU does not restore the same UTS namespace but restores into a, by the container engine, pre-created UTS namespace. No code exists in CRIU to do this, but as we already do it for the network namespace it should be doable.

The reason why I ignored it for Podman is that your application in the container might have already used gethostname() and has already a copy of the old hostname stored somewhere in memory.

So you need to trigger the containered application to re-read the hostname anyway.

So it is not really bug, more a not yet implemented, because not sure how useful it is.

@rst0git

This comment has been minimized.

Copy link
Collaborator

commented Jun 25, 2019

As Adrian already pointed out, the old hostname might be stored in memory and a copy of all memory pages are stored as part of the container checkpoint.

Another issue could be the /etc/hostname file. When CRIU creates a checkpoint it stores information about all open file descriptors, and for successful restore, it is required to have the same root filesystem.

IMHO it would be easier/simpler to explicitly change the container's hostname after restore.

docker exec -it <id-of-cloned-container> sh -c "hostname '$NEW_HOSTNAME'"
@kirs

This comment has been minimized.

Copy link
Author

commented Jun 26, 2019

Thanks for the context!

Let me share the use case to illustrate the reason why I have asked about it. In orchestrated environments, it's typical to create N replicas of the same container over multiple hosts. If the UoW is called web, then containers created will be named something like web-xua1h, web-ro2ui, web-xec1u (assuming replicas=3), where the suffixes are randomly generated with the uniqueness in mind.

What I'm looking for is to run a "donor" container, snapshot it and let those replicas be created from the snapshot of the donor. It would be a bit confusing from the application side if all of them shared the same name. Of course, the app would have to avoid persisting the hostname and be ready that it might change.

IMHO it would be easier/simpler to explicitly change the container's hostname after restore.

I think this would require --cap-add SYS_ADMIN added to the container, but yeah, I think that would work. Thanks!

@adrianreber

This comment has been minimized.

Copy link
Member

commented Jun 26, 2019

You could also use the crit tool to modify the utsns*img in the checkpoint:

# crit show utsns-11.img 
{
    "magic": "UTSNS",
    "entries": [
        {
            "nodename": "25c9295d0e47",
            "domainname": "(none)"
        }
    ]
}

You can use the decode and encode command to decode, modify and encode the utsns image file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.