ENV variables in secret targetpath

[danielschlegel]

I try to mount a secret as a file inside the $HOME directory. Unfortunately this does not work.

  RUN echo $HOME # outputs /root
  RUN --mount=type=secret,target=/root/.ssh/id_rsa_akeneo,id=+secrets/AKENEO_SSH_KEY \ 
      cat /root/.ssh/id_rsa_akeneo  # works
  RUN --mount=type=secret,target=$HOME/.ssh/id_rsa_akeneo,id=+secrets/AKENEO_SSH_KEY \ 
      cat $HOME/.ssh/id_rsa_akeneo # does not work

I assume that earthly is not replacing the env variable $HOME in the target.

[alexcb]

I would have expected this to work with the VERSION --shell-out-anywhere 0.6 feature enabled, but even then this doesn't seem to work. Thanks for raising the issue.

[alexcb]

It's interesting to note that the HOME env does not exist in a standard image such as alpine; as we can see by running

$ docker image inspect alpine | jq '.[].Config.Env'

which only shows that PATH is set in the alpine image:

[
  "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
]

Under the hood, when earthly is run with the --shell-out-anywhere feature, if it detects a var such as $KEY, it simply looks up the equivalence of Config.Env.

However if the shell-out-anywhere lexer encounters $(....), it will shell out to the container, and execute a command within /bin/sh; and it appears that /bin/sh sets additional envs upon execution (e.g. HOME, SHLVL, TERM, PWD).

so as a work-around, you can force the run mount path to be expanded using a shellout by running $(echo $HOME).

Here's a example:

VERSION --shell-out-anywhere 0.6
test:
  FROM alpine
  RUN --mount=type=secret,target="$(echo $HOME)/.ssh/id_rsa",id=+secrets/mykey,mode=0600 \
    ls -la $HOME/.ssh/id_rsa && cat $HOME/.ssh/id_rsa

Then running earthly --no-cache --secret mykey=helloworld +test produces:

              alpine | --> Load metadata alpine linux/amd64
               +test | --> FROM alpine
               +test | [----------] 100% resolve docker.io/library/alpine@sha256:8914eb54f968791faf6a8638949e480fef81e697984fba772b3976835194c6d4
               +test | --> expandargs echo $HOME
               +test | --> RUN ls -la $HOME/.ssh/id_rsa && cat $HOME/.ssh/id_rsa
               +test | -rw-------    1 root     root            10 Dec  2 21:07 /root/.ssh/id_rsa
               +test | helloworld
              output | --> exporting outputs

It's an unexpected work-around -- perhaps we should add some sort of heuristic to automatically have $HOME expand via a shell-out?

[danielschlegel]

Thanks for your workaround, yes i think the automatic expand could help here.
I just started with earthly but i can easily tackle a complicated use case and i love it!

[danielschlegel]

There is another strange behaviour i just got related to ENV variables. If a variable is expanded through shell-out it should still work.

push:
ARG AWS_REGION=eu-west-1

+push | --> RUN --no-cache echo "$AWS_REGION"
+push | eu-west-1
+push | --> RUN --no-cache echo "$(AWS_REGION)"
+push | /bin/sh: 1: AWS_REGION: not found

[alexcb]

$(...) runs the command ..., so in you're case you're literally trying to run a command AWS_REGION which doesn't exist.

instead you would need to do $(echo $AWS_REGION), or $(/usr/bin/echo $AWS_REGION), or perhaps $(env | grep AWS_REGION), or maybe even $(python3 -c 'import os; print(os.environ["AWS_REGION"])'

[danielschlegel]

Thanks for pointing this out.