Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/borgbackup: fix env vars not being passed verbatim in job wrappers #179899

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

erikarvstedt
Copy link
Member

@erikarvstedt erikarvstedt commented Jul 2, 2022

Copy of commit msg

Like in the borg-job systemd service, env var values are now passed verbatim to the borg process.
Previously, the var values were evaluated as bash double quoted strings in the Nix build environment for the job wrapper .

Test this with:

read -d '' container <<'EOF' || :
{
  containers.borg-test = {
    config = {
      documentation.enable = false;
      services.borgbackup.jobs.main = {
        startAt = [];
        paths = [ "/" ];
        repo = "myrepo";
        encryption.mode = "none";
        environment.BORG_REMOTE_PATH = "$HOME/bin/borg";
      };
    };
  };
}
EOF
extra-container shell --nixpkgs-path /home/main/d/nix-dev/nixpkgs -E "$container" --run c bash -c 'cat $(which borg-job-main)' | grep BORG_REMOTE_PATH

Output before this PR:

export BORG_REMOTE_PATH='/homeless-shelter/bin/borg'

Output after this PR:

export BORG_REMOTE_PATH='$HOME/bin/borg'

@erikarvstedt
Copy link
Member Author

@ofborg test borgbackup

@dotlambda dotlambda changed the title borgbackup: don't evaluate env vars in job wrapper nixos/borgbackup: don't evaluate env vars in job wrapper Jul 5, 2022
@erikarvstedt erikarvstedt changed the title nixos/borgbackup: don't evaluate env vars in job wrapper nixos/borgbackup: fix env vars not being passed verbatim in job wrappers Jul 6, 2022
Copy link
Contributor

@pennae pennae left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will definitely need a changelog entry, just in case anyone does rely on expansion for envs. escaping these does look like a good idea in general (can't think of anything that would need env vars to be expanded like this).

@erikarvstedt
Copy link
Member Author

erikarvstedt commented Oct 23, 2022

I've added a changelog entry.

Example: For cheap VPS nodes that only allow access to a $HOME directory, one would install borg in $HOME and set BORG_REMOTE_PATH = "$HOME/path/to/bin/borg".

@pennae
Copy link
Contributor

pennae commented Oct 23, 2022

Example: For cheap VPS nodes that only allow access to a $HOME directory, one would install borg in $HOME and set BORG_REMOTE_PATH = "$HOME/path/to/bin/borg".

but the builder does not have access to the correct $HOME expansion for the target system, and even if it did borg itself does not expand its variables so it'd either see something completely wrong (as you noted) or not fully expanded. or am i missing something?

@erikarvstedt
Copy link
Member Author

BORG_REMOTE_PATH is evaluated via SSH in the remote context, so $HOME expands correctly.

@pennae
Copy link
Contributor

pennae commented Oct 23, 2022

right, that makes sense then.

@pennae
Copy link
Contributor

pennae commented Oct 23, 2022

while testing this we noticed that systemd specifiers in the environment remain unescaped in the generated unit, giving another source of differences between the unit and the job wrapper. that'll also need to be solved somehow, but it looks like that would be a multi-year effort since it'd have to touch unit file generation 😕

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/695

@erikarvstedt
Copy link
Member Author

erikarvstedt commented Nov 30, 2022

Rebased to master.

This rebase can be reviewed with:

git fetch https://github.com/NixOS/nixpkgs 77382a5a242d72c4f4e60590aeadd6bb55fa5bcf
git range-diff FETCH_HEAD...HEAD

Like in the borg-job systemd service, env var values are now passed
verbatim to the borg process.
Previously, the var values were evaluated as bash double quoted strings.
@@ -551,4 +551,6 @@ Available as [services.patroni](options.html#opt-services.patroni.enable).

- `haskellPackages.callHackage` and `haskellPackages.callCabal2nix` (and related functions) no longer keep a reference to the `cabal2nix` call used to generate them. As a result, they will be garbage collected more often.

- `services.borgbackup`: In the borg job wrapper, environment variables are now passed verbatim to the borg process, so that its process environment is the same as in the systemd service. Previously, vars were expanded in the Nix build environment for the job wrapper.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to move this to the 23.05 release notes

@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 19, 2024
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants