Behavior of path_regex changed in patch release and broke some usecases

[michael-k]

Since release 3.3.1 (#478) path regexes check against the full path as implemented in PR #466 (issue #465). This solved the usecase mentioned in the issue but broke another one.

Consider this config (taken from #466):

creation_rules:
    - path_regex: .*/development/.*
      kms: '…'

    - path_regex: .*/production/.*
      kms: '…'

    - kms: '…'

If the repo that contains the secrets is located under a path like /home/user/development/secrets/, the first rule will always match. I have no control over the location where other developers store their repos.

We tried to prevent this by using ^(./)?development/.+\.yaml as a path_regex and calling sops always from the project root, but this no longer works with sops 3.3.1 and fails with error loading config: no matching creation rules found. It even fails for existing encrypted files, despite the irrelevance of creation rules for them.

Possible solution

Instead of simply converting the path passed to sops to an absolute path, the absolute path should be made relative to the config file (if one exists).

This should support both the use case from #465 and ours.

[ajvb]

Hey @michael-k

Instead of simply converting the path passed to sops to an absolute path, the absolute path should be made relative to the config file (if one exists).

That sounds good to me. Can get onto this shortly (or will be happy to review a PR).

[cilindrox]

Any updates on this? Currently running into the same issue - can you point me towards what needs updating @ajvb ? Happy to open a PR to address this

[schollii]

If I understand this correctly, the above proposal would break the following use-case: we have a .sops.yaml in a folder, soft-linked to a sops.yaml in "configuration" folder. So given /a/.sops.yaml -> /b/sops.yaml, the above would cause sops -e /c/somefile.yaml to end up seeing path as ../../c/somefile.yaml. Not sure.

[michael-k]

Relative to the project root is closer to what I'm looking for than relative to the config file. Whatever the project root is. Maybe traverse the path up until a config file, a .git/.hg folder or / is found.

And I'm also happy if this is available as an option. Maybe another config next to path_regex or path_regex being split into path_regex_absolute/path_regex_relative or something else.

I'm also able to pass the prefix that path_regex should ignore to sops.

[michael-k]

While writing my latest comment, I thought of a solution that works for us. Since we're calling sops from a small wrapper script anyway, we're now adjusting the configuration on the fly:

sed -r "s|- (path_regex):\s+/*|- \1: ^$(pwd)/|g" ./.sops.template.yaml > ./.sops.yaml

This takes path_regex: .*/development/.* from the template and converts it to path_regex: ^/home/user/development/secrets/.*/development/.*.