Skip to content
This repository has been archived by the owner on Feb 7, 2024. It is now read-only.

dgr 78 panics with invalid memory address at deserialization #221

Closed
mark-kubacki opened this issue Nov 29, 2016 · 7 comments
Closed

dgr 78 panics with invalid memory address at deserialization #221

mark-kubacki opened this issue Nov 29, 2016 · 7 comments

Comments

@mark-kubacki
Copy link
Contributor

mark-kubacki commented Nov 29, 2016 •
edited
Loading

Using dgr 78 from HEAD:

# clean Ubuntu 16.04 VM

sudo apt-get -q update
sudo apt-get -y --no-install-recommends install \
  golang-1.6-go bats upx \
  git sudo curl

sudo ln -s /usr/lib/go-1.6/bin/go /usr/bin/go
sudo ln -s /usr/lib/go-1.6/bin/gofmt /usr/bin/gofmt
export GOPATH=${HOME}/cleanenv
mkdir -p cleanenv && cd $_

go get -v -t github.com/Masterminds/glide
export PATH=${PATH}:${HOME}/cleanenv/bin
mkdir -p github.com/blablacar && cd $_
git clone -o blablacar https://github.com/blablacar/dgr.git
cd dgr
mv vendor /tmp/ && go get -d -t ./...
mv /tmp/vendor ./
# ~/cleanenv/bin/glide install

./gomake clean build test

… throws an error:

Running test file -> test.bats
+ bats -p ./test.bats
 ✓ should fail with no name
 ✓ should fail if not exists
 ✓ should be runnable with only name
 ✗ should be globally working
   (in test file test.bats, line 26)
     `[ "$status" -eq 0 ]' failed
     2016-11-29 11:41:57 INFO            dgr/dgr/aci-build.go:48  Building                                     aci=example.com/aci-test
   2016-11-29 11:41:57 INFO            dgr/dgr/aci-build.go:48  Building                                     aci=example.com/aci-test
   2016-11-29 11:41:58 INFO            dgr/dgr/aci-build.go:229 Importing build to rkt                       aci=example.com/aci-test path=/home/mark/cleanenv/src/github.com/blablacar/dgr/tests/filled_up/target/builder/image.aci
   2016-11-29 11:41:58 INFO            dgr/dgr/aci-build.go:68  Calling rkt to start build                   aci=example.com/aci-test
   2016-11-29 11:41:58 INFO          d/a/bin-run/builder.go:65  Building aci                                 aci=aci-test
   Timezone Europe/Berlin does not exist in container, not updating container timezone.
   Running script -> /dgr/aci-home/runlevels/builder/run.sh
   export ACI_EXEC=''"'"'/dgr/bin/busybox'"'"' '"'"'sh'"'"
   export ACI_HOME='/dgr/aci-home'
   export ACI_NAME='aci-test'
   export ACI_PATH='/home/mark/cleanenv/src/github.com/blablacar/dgr/tests/filled_up'
   export CATCH_ON_ERROR='false'
   export CATCH_ON_STEP='false'
   export DGR_VERSION='0'
   export HOME='/root'
   export LOGNAME='root'
   export LOG_LEVEL='INFO'
   export OLDPWD='/dgr/usr/bin'
   export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
   export PWD='/'
   export ROOTFS='/opt/stage2/aci-test/rootfs'
   export SHLVL='3'
   export SYSTEMD_LOG_LEVEL='err'
   export TERM='xterm'
   export USER='root'
   export container='systemd-nspawn'
   export container_uuid='c8efa1b6-afba-4779-ab72-47ed4ecf5497'
   export log_level='4'
   Running script -> /dgr/builder/runlevels/build/run.sh
   2016-11-29 11:41:59 INFO            dgr/dgr/aci-build.go:89  Finished building aci                        fullname=example.com/aci-test:1.0.0-1
   2016-11-29 11:41:59 WARN             dgr/dgr/aci-sign.go:33  Sign disabled for this aci's domain          aci=example.com/aci-test domain=example.com
   2016-11-29 11:41:59 INFO             dgr/dgr/aci-test.go:28  Testing                                      aci=example.com/aci-test
   2016-11-29 11:42:00 INFO             dgr/dgr/aci-test.go:32  Building test aci                            aci=example.com/aci-test
   panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
   [signal 0xb code=0x1 addr=0x0 pc=0x4bb5be]

   goroutine 1 [running]:
   panic(0xa08620, 0xc82000e100)
        /usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
   encoding/json.(*decodeState).unmarshal.func1(0xc820119378)
        /usr/lib/go-1.6/src/encoding/json/decode.go:154 +0x88
   panic(0xa08620, 0xc82000e100)
        /usr/lib/go-1.6/src/runtime/panic.go:443 +0x4e9
   github.com/blablacar/dgr/vendor/github.com/appc/spec/schema/types.(*Isolator).UnmarshalJSON(0xc8200a95e0, 0xc8200a5489, 0x6c, 0x1f2, 0x0, 0x0)
        /home/mark/cleanenv/src/github.com/blablacar/dgr/vendor/github.com/appc/spec/schema/types/isolator.go:175 +0x12e
   encoding/json.(*decodeState).object(0xc8201a8a50, 0xa22760, 0xc8200a95e0, 0x199)
        /usr/lib/go-1.6/src/encoding/json/decode.go:560 +0x143
   encoding/json.(*decodeState).value(0xc8201a8a50, 0xa22760, 0xc8200a95e0, 0x199)
        /usr/lib/go-1.6/src/encoding/json/decode.go:367 +0x3a1
   encoding/json.(*decodeState).array(0xc8201a8a50, 0x9ed260, 0xc8201cb450, 0x197)
        /usr/lib/go-1.6/src/encoding/json/decode.go:518 +0xa6b
   encoding/json.(*decodeState).value(0xc8201a8a50, 0x9ed260, 0xc8201cb450, 0x197)
        /usr/lib/go-1.6/src/encoding/json/decode.go:364 +0x3c1
   encoding/json.(*decodeState).object(0xc8201a8a50, 0xa78560, 0xc8201cb3a8, 0x199)
        /usr/lib/go-1.6/src/encoding/json/decode.go:684 +0x116a
   encoding/json.(*decodeState).value(0xc8201a8a50, 0xa78560, 0xc8201cb3a8, 0x199)
        /usr/lib/go-1.6/src/encoding/json/decode.go:367 +0x3a1
   encoding/json.(*decodeState).object(0xc8201a8a50, 0xa3e580, 0xc8201cb3a8, 0x199)
        /usr/lib/go-1.6/src/encoding/json/decode.go:684 +0x116a
   encoding/json.(*decodeState).value(0xc8201a8a50, 0xa3e580, 0xc8201cb3a8, 0x199)
        /usr/lib/go-1.6/src/encoding/json/decode.go:367 +0x3a1
   encoding/json.(*decodeState).object(0xc8201a8a50, 0xa4c8e0, 0xc8201cb340, 0x199)
        /usr/lib/go-1.6/src/encoding/json/decode.go:684 +0x116a
   encoding/json.(*decodeState).value(0xc8201a8a50, 0x8de320, 0xc8201cb340, 0x16)
        /usr/lib/go-1.6/src/encoding/json/decode.go:367 +0x3a1
   encoding/json.(*decodeState).unmarshal(0xc8201a8a50, 0x8de320, 0xc8201cb340, 0x0, 0x0)
        /usr/lib/go-1.6/src/encoding/json/decode.go:168 +0x196
   encoding/json.Unmarshal(0xc8200a5440, 0x1c3, 0x23b, 0x8de320, 0xc8201cb340, 0x0, 0x0)
        /usr/lib/go-1.6/src/encoding/json/decode.go:96 +0x12b
   github.com/blablacar/dgr/vendor/github.com/ghodss/yaml.Unmarshal(0xc82017d180, 0x25a, 0x25a, 0x8de320, 0xc8201cb340, 0x0, 0x0)
        /home/mark/cleanenv/src/github.com/blablacar/dgr/vendor/github.com/ghodss/yaml/yaml.go:37 +0x20c
   github.com/blablacar/dgr/dgr/common.ProcessManifestTemplate(0xc82017c780, 0x25b, 0x0, 0x0, 0x582400, 0xc8201cadc0, 0x0, 0x0)
        /home/mark/cleanenv/src/github.com/blablacar/dgr/dgr/common/dgr-manifest.go:124 +0xac7
   main.NewAciWithManifest(0xc8201b4500, 0x3e, 0x0, 0x0, 0x0, 0xc82017c780, 0x25b, 0xc8201ae060, 0x21, 0x0, ...)
        /home/mark/cleanenv/src/github.com/blablacar/dgr/dgr/aci.go:53 +0x63
   main.(*Aci).buildTestAci(0xc8201b03f0, 0x0, 0x0, 0x0, 0x0)
        /home/mark/cleanenv/src/github.com/blablacar/dgr/dgr/aci-test.go:164 +0xbaa
   main.(*Aci).Test(0xc8201b03f0, 0x0, 0x0)
        /home/mark/cleanenv/src/github.com/blablacar/dgr/dgr/aci-test.go:33 +0x165
   main.newTestCommand.func1(0xc82017aea0, 0xc82019a8e0, 0x0, 0x2)
        /home/mark/cleanenv/src/github.com/blablacar/dgr/dgr/command.go:260 +0xf0
   github.com/blablacar/dgr/vendor/github.com/spf13/cobra.(*Command).execute(0xc82017aea0, 0xc82019a700, 0x2, 0x2, 0x0, 0x0)
        /home/mark/cleanenv/src/github.com/blablacar/dgr/vendor/github.com/spf13/cobra/command.go:477 +0x3f4
   github.com/blablacar/dgr/vendor/github.com/spf13/cobra.(*Command).Execute(0xc82017bd40, 0x0, 0x0)
        /home/mark/cleanenv/src/github.com/blablacar/dgr/vendor/github.com/spf13/cobra/command.go:551 +0x46a
   main.Execute()
        /home/mark/cleanenv/src/github.com/blablacar/dgr/dgr/main.go:101 +0x83f
   main.main()
        /home/mark/cleanenv/src/github.com/blablacar/dgr/dgr/main.go:52 +0x1a2
 ✗ dgr init should create working aci
@n0rad
Copy link
Member

n0rad commented Nov 29, 2016

can you post the aci-manifest.yml, it's probably a use case around isolators that I did not included into my yesterday's tests about it.

I saw yesterday that latest version of appc is handling isolator marshalling correctly so everything around isolator is in appc code now.

@mark-kubacki
Copy link
Contributor Author

The error is thrown on Travis, too:
https://travis-ci.org/blablacar/dgr/builds/179754485#L638

@mark-kubacki
Copy link
Contributor Author

mark-kubacki commented Dec 7, 2016 •
edited
Loading

This is caused by isolators not being unmarshalled from YAML to the corresponding data structure. The APPC specification works with json.RawMessage for isolators (and subsequently a factory to map types), and our YAML converter converts via YAML→JSON→struct. This then breaks. See also go-yaml/yaml#13

This happens:

  1. The ACI gets built (as part of the tests, in this example).
  2. dgr/aci-test: buildTestAci adds isolators (or the user has specified them herself) in preparation for the test container.
  3. dgr/common: ProcessManifestTemplate calls yaml.Unmarshal and panics

A possible workaround:

type DgrIsolator types.Isolator

func (i *DgrIsolator) UnmarshalYAML(func unmarshal(…)…) error {
  var a struct {Name string; Value string `json:value` `yaml:valueraw`}
  var b struct {Name string; Value []string `json:value` `yaml:valueraw`}
  var c struct {Name string; Value uint64 `json:value` `yaml:valueraw`}
  // …

  if err := unmarshal(&a); err == nil {
    str := json.ToString(&a) // exemplary!
    i.(types.Isolator).UnmarshalJSON(str)
    return
  }
  if … // try every type
}

@n0rad
Copy link
Member

n0rad commented Dec 20, 2016

Thanks that was helpful. It looks like using interface{} is enough to work.

e0add66 > use interface{} as Value for isolators
af26514 > revert of #211 relying of appc objects

I have added a simple test to be sure resources & seccomp/capa both works

@mark-kubacki
Copy link
Contributor Author

Thanks!

(It's up to you to close this. ;-) )

@n0rad
Copy link
Member

n0rad commented Dec 20, 2016

I will when a new version will be released

@n0rad
Copy link
Member

n0rad commented Dec 29, 2016

fixed in v79

@n0rad n0rad closed this as completed Dec 29, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mark-kubacki @n0rad