Support using comments to select parts to encrypt #1392

felixfontein · 2023-12-27T16:44:00Z

This PR adds support to annotate comments with a string (e.g., sops:enc) which can then be matched with a regex. If it matches, the corresponding value (the one which follows the comment) is encrypted while other values are not. (There is also the opposite regex available, to select those values which should not be encrypted.)

This enables the YAML file to have the same structure encrypted and decrypted, without having to add suffixes or manage complex regexes to match keys. See #543 for more discussion.

(This PR currently contains #1389 and will be rebased once that's merged. It is a lot more likely that #1389 will be merged before this PR or any of its variants, so I decided to use that one as a basis.)

This PR continues the work of #974 by rebasing #974 upon #1389 and adding some final touches (see #974 (comment)).

Closes #974.

felixfontein · 2023-12-27T17:28:17Z

@mitar I think the two comments I added address the issues from my review in #974 (review). Can you take a look at the commits?

mitar · 2024-01-04T12:56:49Z

sops.go

+				if ok && tree.Metadata.UnencryptedCommentRegex != "" {
+					// If an encrypted comment matches tree.Metadata.UnencryptedCommentRegex, decryption will fail
+					// as the MAC does not match, and the commented value will not be decrypted.
+					matched, _ := regexp.Match(tree.Metadata.UnencryptedCommentRegex, []byte(in.(string)))


Why not use here MatchString?

Didn't we say we will also try to match against EncryptedCommentRegex? So both of them should not match the resulting encrypted string?

I originally implemented this, and then when trying to test it I removed the code again. As opposed to UnencryptedCommentRegex, encrypted comments matching EncryptedCommentRegex aren't a problem since comments are only encrypted/decrypted if they are part of an encrypted subtree that's encrypted due to a comment higher up.

For example, you can encrypt

--- #ENC[AES256_GCM,data:MI3A,iv:c9lHpZCbz3W9ooGuGbmQTcWcXAH7JEL/5JyJdC41dec=,tag:LrQLBEgQayguId5RrNiOCA==,type:comment] foo: bar: barbar #ENC[AES256_GCM,data:MI3A,iv:c9lHpZCbz3W9ooGuGbmQTcWcXAH7JEL/5JyJdC41dec=,tag:LrQLBEgQayguId5RrNiOCA==,type:comment] # something else baz: bam: bambam #ENC[AES256_GCM,data:MI3A,iv:c9lHpZCbz3W9ooGuGbmQTcWcXAH7JEL/5JyJdC41dec=,tag:LrQLBEgQayguId5RrNiOCA==,type:comment]

with --encrypted-comment-regex ENC and then decrypt it again without any problems.

We could still implement the check to be extra cautious, but it isn't needed (or I'm missing something :) ).

OK. I think it is then fine and we can revisit this if anyone finds a realistic case where it is breaking some user flow.

mitar · 2024-01-04T12:57:38Z

Thanks. I think this looks good as it is already. Thank you for all the work. I made a small comment, but I think it is not critical to do it.

felixfontein · 2024-02-06T14:03:45Z

Now that #1389 has been merged, this is ready for review! 🎉

hiddeco · 2024-02-09T13:35:43Z

sops.go

+		if vIsComment {
+			// If v is a comment, we add it to the slice of active comments.
+			// This allows us to also encrypt comments themselves by enabling encryption in a prior comment.
+			commentsStack[len(commentsStack)-1] = append(commentsStack[len(commentsStack)-1], c.Value)


Is there (n)ever a theoretical chance of len(commentsStack) == 0?

That can't happen since in the first line of the function, one element is appended to commentsStack, and during the execution of this function no value is removed from commentsStack.

Gui13 · 2024-04-07T08:55:56Z

This looks good, any idea when this could be merged ?

felixfontein · 2024-04-07T10:16:45Z

It's currently waiting for further reviews / approval by maintainer(s).

mitar · 2024-05-13T23:21:36Z

Is there anything to do to push this further?

felixfontein · 2024-06-02T11:02:51Z

Like several other PRs, this is waiting for a review / approval from someone else in @getsops/maintainers.

devstein

Looks good to me, but would be nice to get another review from @hiddeco @onedr0p if possible; otherwise, I can merge

sabre1041

LGTM

Signed-off-by: Mitar <mitar.git@tnode.com>

…ments. Signed-off-by: Felix Fontein <felix@fontein.de>

Signed-off-by: Felix Fontein <felix@fontein.de>

felixfontein · 2024-06-27T07:26:27Z

@mitar thank you very much for your contribution! I'm glad it finally got merged...
Also thanks to everyone who helped reviewing and testing this!

mitar · 2024-06-27T08:54:25Z

Thank you everyone who was involved into getting this in, and especially you @felixfontein for all the help getting it in.

How could I bring #1404 further?

c33s · 2024-06-27T10:15:43Z

@felixfontein

thank you very much for this awesome feature!

two questions:

can you please add a simple example to the readme on how to use this new feature?
is it possible to invert the features behavior (encrypt by default and use a comment to keep the value unencrypted)?

mitar · 2024-06-27T10:50:42Z

README does have some text about this here.

is it possible to invert the features behavior (encrypt by default and use a comment to keep the value unencrypted)?

Yes, you can use unencrypted-regex for that.

c33s · 2024-06-27T18:31:42Z

i meant some examples for the config file like:
.sops.yaml

- encrypted_regex: '.'
  unencrypted_comment_regex: 'keep_decrypted'
  path_regex: '*\.yaml$'

file.yaml

foo:
  # keep_decrypted
  bar: 1234
  password: will be enrcypted

oh in missed the --unencrypted-comment-regex. really awesome it can work both ways

This was referenced Dec 27, 2023

Move extraction of encryption and rotation options to separate functions #1387

Closed

Support using comments to select parts to encrypt #974

Closed

mitar reviewed Jan 4, 2024

View reviewed changes

felixfontein force-pushed the pr-974-continue branch from de3e3ac to 46c488a Compare February 6, 2024 13:58

felixfontein marked this pull request as ready for review February 6, 2024 14:03

hiddeco reviewed Feb 9, 2024

View reviewed changes

felixfontein force-pushed the pr-974-continue branch from 46c488a to e55463f Compare February 9, 2024 20:13

c33s mentioned this pull request Mar 21, 2024

Allow to encrypt specific nodes in a file with specific keys (muliple matching creation_rules) #1467

Open

felixfontein force-pushed the pr-974-continue branch from e55463f to 0b498e9 Compare June 3, 2024 04:37

devstein self-requested a review June 10, 2024 23:47

devstein reviewed Jun 12, 2024

View reviewed changes

felixfontein force-pushed the pr-974-continue branch from 0b498e9 to 54298f8 Compare June 15, 2024 15:54

This was referenced Jun 26, 2024

Release 3.9.0 #1546

Merged

New patch version please #1453

Closed

felixfontein added this to the v3.9.0 milestone Jun 26, 2024

sabre1041 approved these changes Jun 27, 2024

View reviewed changes

mitar and others added 3 commits June 27, 2024 09:21

Support using comments to select parts to encrypt

f63e844

Signed-off-by: Mitar <mitar.git@tnode.com>

Add check to avoid that UnencryptedCommentRegex matches encrypted com…

f6ece99

…ments. Signed-off-by: Felix Fontein <felix@fontein.de>

Update documentation.

6cfb8ed

Signed-off-by: Felix Fontein <felix@fontein.de>

felixfontein force-pushed the pr-974-continue branch from 54298f8 to 6cfb8ed Compare June 27, 2024 07:21

felixfontein merged commit aa800f1 into getsops:main Jun 27, 2024
10 checks passed

felixfontein deleted the pr-974-continue branch June 27, 2024 07:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support using comments to select parts to encrypt #1392

Support using comments to select parts to encrypt #1392

felixfontein commented Dec 27, 2023 •

edited

Loading

felixfontein commented Dec 27, 2023

mitar Jan 4, 2024

felixfontein Jan 5, 2024

mitar Jan 5, 2024

mitar commented Jan 4, 2024

felixfontein commented Feb 6, 2024

hiddeco Feb 9, 2024

felixfontein Feb 9, 2024

Gui13 commented Apr 7, 2024

felixfontein commented Apr 7, 2024

mitar commented May 13, 2024

felixfontein commented Jun 2, 2024

devstein left a comment

sabre1041 left a comment

felixfontein commented Jun 27, 2024

mitar commented Jun 27, 2024

c33s commented Jun 27, 2024

mitar commented Jun 27, 2024

c33s commented Jun 27, 2024

Support using comments to select parts to encrypt #1392

Support using comments to select parts to encrypt #1392

Conversation

felixfontein commented Dec 27, 2023 • edited Loading

felixfontein commented Dec 27, 2023

mitar Jan 4, 2024

Choose a reason for hiding this comment

felixfontein Jan 5, 2024

Choose a reason for hiding this comment

mitar Jan 5, 2024

Choose a reason for hiding this comment

mitar commented Jan 4, 2024

felixfontein commented Feb 6, 2024

hiddeco Feb 9, 2024

Choose a reason for hiding this comment

felixfontein Feb 9, 2024

Choose a reason for hiding this comment

Gui13 commented Apr 7, 2024

felixfontein commented Apr 7, 2024

mitar commented May 13, 2024

felixfontein commented Jun 2, 2024

devstein left a comment

Choose a reason for hiding this comment

sabre1041 left a comment

Choose a reason for hiding this comment

felixfontein commented Jun 27, 2024

mitar commented Jun 27, 2024

c33s commented Jun 27, 2024

mitar commented Jun 27, 2024

c33s commented Jun 27, 2024

felixfontein commented Dec 27, 2023 •

edited

Loading