Using --env for nerdctl doesn't pass through current environment values

[tigris]

I'm unsure if this is expected behaviour or not, I was attempting to replace docker with nerdctl.lima on Mac OSX (M1) and almost all other runtime options seem to work the same, so seemed a bit odd that this one doesn't quite get there. Feel free to close if it's expected behaviour.

#-❯ export FOO=bar

#-❯ echo $FOO
bar

#-❯ nerdctl.lima run --rm -it -e FOO ubuntu bash
root@4b71f410d6a4:/# echo $FOO
<empty>

vs

#-❯ nerdctl.lima run --rm -it -e FOO=baz ubuntu bash
root@1697d3d01eac:/# echo $FOO
baz

When using docker, usage of -e FOO with no value, would pass through whatever it is in the current environment. Perhaps it's a problem with lima not passing through the current environment to the linux VM on mac OS. Perhaps this is expected behaviour of native nerdctl? I'll move this issue over to there if that's the case.

FWIW- I'm using the default VM that came with brew install lima. Perhaps there is a way to configure it so the entire current ENV vars are always pushed through the SSH?

[afbjorklund]

Apparently containerd uses "FOO" to mean it should be removed, and "FOO=bar" to mean that it shall be added.

https://pkg.go.dev/github.com/containerd/containerd@v1.5.7/oci#WithEnv

https://github.com/containerd/containerd/blob/v1.5.7/oci/spec_opts.go#L178_L214

For instance, -e PATH doesn't start. (With docker, it would pass on the current value of $PATH to the container)

FATA[0000] failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown

I suppose the workaround for this change would be to use -e FOO=$FOO.

[jandubois]

I suppose the workaround for this change would be to use -e FOO=$FOO.

Yes, the issue is that nerdctl.lima is just a shell wrapper to execute limactl shell $INSTANCE nerdctl ... inside the VM, and FOO is not set in the guest, but only the host.

To get this working automatically, we would need to pass all the local environment variables over ssh into the guest as well (with a blocklist for things like PATH, because those would break stuff).

[afbjorklund]

It didn't seem specific to lima at all, but a containers/nerdctl specific flag

[jandubois]

It didn't seem specific to lima at all, but a containers/nerdctl specific flag

That can also be true, but either way, it wouldn't work because the environment variable is not set when nerdctl is being run.

[tigris]

I suppose the workaround for this change would be to use -e FOO=$FOO.

This would work for some cases, but not all. As an example, my employer has an executable we use to retrieve AWS credentials and put them in the ENV at runtime. We then chain that with another command, so the entire shell doesn't need the credentials.

get-creds -- nerdctl.lima run -e AWS_CREDS ...

In this scenario, using AWS_CREDS=$AWS_CREDS won't work, because the shell will interpret it too early.

#-❯ FOO=bar echo $FOO
<empty>

vs

#-❯ FOO=bar
#-❯ echo $FOO
bar

Or, to show how it might work with docker for example

#-❯ FOO=bar docker run --rm -it -e FOO ubuntu bash
root@625cee3f5c7c:/# echo $FOO
bar

[afbjorklund]

Yeah, I think it would break the mirage and you would have to use lima nerdctl and quote the shell variables accordingly...

$ FOO=bar sh -c "echo \$FOO"
bar

i.e. the user would have to know that some things run on host and some things run on a virtual machine different from it

Changing nerdctl sounds easier ?

[afbjorklund]

it wouldn't work because the environment variable is not set when nerdctl is being run.

right, because nerdctl is run over ssh unlike the docker and podman which run locally ?

$ nerdctl.lima
$ limactl shell ${LIMA_INSTANCE:=default} nerdctl

$ export DOCKER_HOST=unix://$HOME/docker.sock
$ docker ...

$ export CONTAINER_HOST=unix://$HOME/podman.sock
$ podman ...

[tigris]

Sounds like the problem is 2-fold.

• nerdctl --env FOO doesn't pass through FOO like docker does
• lima doesn't send the current ENV to the guest VM

Happy for this to be closed / updated knowing the actual --env problem isn't specific to lima.

[jandubois]

right, because nerdctl is run over ssh unlike the docker and podman which run locally ?

Yes. And it is difficult to pass the variables over ssh, as you need to configure SendEnv on the host for each variable you want to send, and AcceptEnv in the guest for each one you want to accept (and restart sshd after changing the config).

I think you could use *, but that runs into issues with PATH that you explicitly don't want to inherit from the host.

Not sure this is worth the effort; maybe it should just be a FAQ item?

[afbjorklund]

Tricky... It's like sudo, but worse.

[jandubois]

get-creds -- nerdctl.lima run -e AWS_CREDS ...

@tigris Maybe you can get it to work like this:

get-creds -- sh -c 'nerdctl.lima run -e AWS_CREDS="${AWS_CREDS}" ...'