Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

shamir treshold breaks env files (panic: interface conversion: interface {} is float64, not string) #857

Closed
cohadar opened this issue Apr 16, 2021 · 2 comments · Fixed by #1394 · May be fixed by #1401
Closed

shamir treshold breaks env files (panic: interface conversion: interface {} is float64, not string) #857

cohadar opened this issue Apr 16, 2021 · 2 comments · Fixed by #1394 · May be fixed by #1401
Labels
bug priority/medium Medium priority issues (e.g. breaking changes that have a workaround)

Comments

@cohadar
Copy link

cohadar commented Apr 16, 2021

sops: 3.7.1 (ubuntu)

.sops.yaml

creation_rules:
- shamir_threshold: 2
  key_groups:
  - pgp:
    - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  - pgp:
    - YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

command:
sops example.env

produces:

panic: interface conversion: interface {} is float64, not string

goroutine 1 [running]:
go.mozilla.org/sops/v3/stores/dotenv.(*Store).EmitPlainFile(0x1a17488, 0xc0005d24e0, 0x1, 0x1, 0x3, 0xc0005af040, 0x8, 0x10, 0x0)
	/Users/ajvb/gocode/src/go.mozilla.org/sops/stores/dotenv/store.go:122 +0x3f6
go.mozilla.org/sops/v3/stores/dotenv.(*Store).EmitEncryptedFile(0x1a17488, 0x21816938, 0xed80b80e1, 0x0, 0x11adb59, 0xc, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/ajvb/gocode/src/go.mozilla.org/sops/stores/dotenv/store.go:107 +0x2fa
main.editTree(0x137d960, 0xc000490fc0, 0x138f380, 0x1a17488, 0x138f380, 0x1a17488, 0xc0001a4840, 0x5c, 0x0, 0xc00048d1d0, ...)
	/Users/ajvb/gocode/src/go.mozilla.org/sops/cmd/sops/edit.go:164 +0xa35
main.editExample(0x137d960, 0xc000490fc0, 0x138f380, 0x1a17488, 0x138f380, 0x1a17488, 0xc0001a4840, 0x5c, 0x0, 0xc00048d1d0, ...)
	/Users/ajvb/gocode/src/go.mozilla.org/sops/cmd/sops/edit.go:84 +0x51b
main.main.func8(0xc0001e4dc0, 0x0, 0x0)
	/Users/ajvb/gocode/src/go.mozilla.org/sops/cmd/sops/main.go:943 +0x24bc
gopkg.in/urfave/cli%2ev1.HandleAction(0xfdafa0, 0x11f8d28, 0xc0001e4dc0, 0x0, 0x0)
	/Users/ajvb/gocode/src/go.mozilla.org/sops/vendor/gopkg.in/urfave/cli.v1/app.go:490 +0x82
gopkg.in/urfave/cli%2ev1.(*App).Run(0xc00047c4e0, 0xc00018c000, 0x2, 0x2, 0x0, 0x0)
	/Users/ajvb/gocode/src/go.mozilla.org/sops/vendor/gopkg.in/urfave/cli.v1/app.go:264 +0x5f5
main.main()
	/Users/ajvb/gocode/src/go.mozilla.org/sops/cmd/sops/main.go:987 +0x3885
@ajvb ajvb added bug priority/medium Medium priority issues (e.g. breaking changes that have a workaround) labels May 7, 2021
@joshdk
Copy link

joshdk commented Jul 22, 2021

👋🏻 Hello folks,

I've also been hitting this exact issue, and can add some color to the configuration that causes it.

This first configuration defines multiple key_groups, and will panic when running sops updatekeys. Additionally, it there is no panic when running sops --encrypt or sops --decrypt.

creation_rules:
  - path_regex: .*
    key_groups:
      - pgp:
          - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      - pgp:
          - YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

Whereas this second configuration a single key_groups with multiple keys, and will not panic under any sops operation.

creation_rules:
  - path_regex: .*
    key_groups:
      - pgp:
          - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
          - YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

Hopefully this help to resolve this bug. Cheers!

Additionally, this observation holds even if you're mixing pgp and kms (which I was). This configuration will panic:

key_groups:
  - kms:
      - arn:aws:kms:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  - pgp:
      - YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

But this configuration will not:

key_groups:
  - kms:
      - arn:aws:kms:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    pgp: # <--- No dash before, so it's the same list item.
      - YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

@joshdk joshdk mentioned this issue Jul 22, 2021
Open
@acastle acastle mentioned this issue Feb 15, 2022
Open
@slewsys slewsys mentioned this issue Apr 25, 2022
Closed
slewsys added a commit to slewsys/sops that referenced this issue Nov 3, 2023
@slewsys
Serialize using mapstructure conversion instead of JSON marshalling.
cb1a94c 
These changes remove the JSON encoding/decoding steps that are
performed during the serialization of ini and dotenv files. This
roundtrip loses type information during the transformation which
causes values to be incorrectly converted to the JSON marshaller
defaults (int becomes float64, bool becomes string, etc, etc). In
place of this JSON encoding, the mapstructure library allows for a
direct conversion between the Metadata struct and
map[string]interface{} needed to leverage the stores.Flatten and
stores.Unflatten functions.

In addition this adds mapstructure tags to the metadata structures to
allow backwards compatibility with the JSON encoding.

Resolves getsops#879 & getsops#857

This is getsops#1009 by acastle with mapstructure updated to v1.5.0
applied to getsops/sops/main HEAD.
slewsys added a commit to slewsys/sops that referenced this issue Nov 3, 2023
@slewsys
Serialize using mapstructure conversion instead of JSON marshalling.
3c9301f 
These changes remove the JSON encoding/decoding steps that are
performed during the serialization of ini and dotenv files. This
roundtrip loses type information during the transformation which
causes values to be incorrectly converted to the JSON marshaller
defaults (int becomes float64, bool becomes string, etc, etc). In
place of this JSON encoding, the mapstructure library allows for a
direct conversion between the Metadata struct and
map[string]interface{} needed to leverage the stores.Flatten and
stores.Unflatten functions.

In addition this adds mapstructure tags to the metadata structures to
allow backwards compatibility with the JSON encoding.

Resolves getsops#879 & getsops#857

This is getsops#1009 by acastle with mapstructure updated to v1.5.0
and unused imports removed applied to getsops/sops/main HEAD.
@slewsys slewsys mentioned this issue Nov 3, 2023
Closed
slewsys added a commit to slewsys/sops that referenced this issue Nov 4, 2023
@slewsys
Serialize using mapstructure conversion instead of JSON marshalling.
21b951b 
These changes remove the JSON encoding/decoding steps that are
performed during the serialization of ini and dotenv files. This
roundtrip loses type information during the transformation which
causes values to be incorrectly converted to the JSON marshaller
defaults (int becomes float64, bool becomes string, etc, etc). In
place of this JSON encoding, the mapstructure library allows for a
direct conversion between the Metadata struct and
map[string]interface{} needed to leverage the stores.Flatten and
stores.Unflatten functions.

In addition this adds mapstructure tags to the metadata structures to
allow backwards compatibility with the JSON encoding.

Resolves getsops#879 & resolves getsops#857

This is a re-submission of @acastle's getsops#1009 with mapstructure updated
to v1.5.0 and unused imports removed applied to getsops/sops/main HEAD.
slewsys added a commit to slewsys/sops that referenced this issue Nov 4, 2023
@slewsys
Serialize using mapstructure conversion instead of JSON marshalling.
ed28fdd 
These changes remove the JSON encoding/decoding steps that are
performed during the serialization of ini and dotenv files. This
roundtrip loses type information during the transformation which
causes values to be incorrectly converted to the JSON marshaller
defaults (int becomes float64, bool becomes string, etc, etc). In
place of this JSON encoding, the mapstructure library allows for a
direct conversion between the Metadata struct and
map[string]interface{} needed to leverage the stores.Flatten and
stores.Unflatten functions.

In addition this adds mapstructure tags to the metadata structures to
allow backwards compatibility with the JSON encoding.

Resolves getsops#879 & resolves getsops#857

This is a re-submission of @acastle's getsops#1009 with mapstructure updated
to v1.5.0 and unused imports removed applied to getsops/sops/main HEAD.
@felixfontein felixfontein mentioned this issue Dec 28, 2023
Merged
@felixfontein
Copy link
Contributor

#1394 should fix this (shamir_threshold breaking INI/DotEnv files).

slewsys added a commit to slewsys/sops that referenced this issue Dec 30, 2023
@slewsys
Serialize using mapstructure conversion instead of JSON marshalling.
053c93d 
These changes remove the JSON encoding/decoding steps that are
performed during the serialization of ini and dotenv files. This
roundtrip loses type information during the transformation which
causes values to be incorrectly converted to the JSON marshaller
defaults (int becomes float64, bool becomes string, etc, etc). In
place of this JSON encoding, the mapstructure library allows for a
direct conversion between the Metadata struct and
map[string]interface{} needed to leverage the stores.Flatten and
stores.Unflatten functions.

In addition this adds mapstructure tags to the metadata structures to
allow backwards compatibility with the JSON encoding.

Resolves getsops#879 & resolves getsops#857

This is a re-submission of @acastle's getsops#1009 with mapstructure updated
to v1.5.0 and unused imports removed applied to getsops/sops/main HEAD.
@slewsys slewsys mentioned this issue Dec 30, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug priority/medium Medium priority issues (e.g. breaking changes that have a workaround)
Projects
None yet
Milestone
No milestone
4 participants
@joshdk @ajvb @felixfontein @cohadar