partition,snap: add support for android boot #3324

Merged
merged 11 commits into from May 19, 2017

Conversation

Projects
None yet
5 participants
Contributor

alfonsosanchezbeato commented May 16, 2017

Support for android style bootloading

@zyga zyga changed the title from Androidboot to partition,snap: add support for android boot May 16, 2017

Simon Fels and others added some commits Nov 1, 2016

Rename fastboot bootloader to android-boot
Rename fastboot bootloader to android-boot, as this is a more accurate
description (fastboot is a protocol). android-boot is chosen because
what we support is booting from devices that are configured in the
usual way for Android devices, with recovery and boot partitions and
boot images with format as defined in
https://android.googlesource.com/platform/system/core/+/master/mkbootimg/bootimg.h

A few comments / early feedback.

partition/androidboot.go
+// -*- Mode: Go; indent-tabs-mode: t -*-
+
+/*
+ * Copyright (C) 2014-2017 Canonical Ltd
@zyga

zyga May 16, 2017

Contributor

Nitpick: was this written in 2014?

partition/androidboot.go
+}
+
+func (a *androidboot) GetBootVars(names ...string) (map[string]string, error) {
+ out := make(map[string]string)
@zyga

zyga May 16, 2017

Contributor

nitpick: move this closer to the loop that populates it.

@chipaca

chipaca May 16, 2017

Member

furthermore: you know how big it needs to be (len(names)), so make it with that size already

partition/androidboot_test.go
+ . "gopkg.in/check.v1"
+)
+
+func mockAndroidbootFile(c *C, newPath string, mode os.FileMode) {
@zyga

zyga May 16, 2017

Contributor

Should we not have a SetUpTest function that calls dirs.SetRootDir(c.MkDir())? This should be paired with TearDownTest that calls dirs.SetRootDir("")

partition/androidboot_test.go
+ c.Assert(err, IsNil)
+}
+
+func (s *PartitionTestSuite) makeFakeAndroidbootConfig(c *C) {
@zyga

zyga May 16, 2017

Contributor

If this happens to be done in each test feel free to do it in SetUpTest

partition/androidboot_test.go
+func (s *PartitionTestSuite) TestNewAndroidbootNoAndroidbootReturnsNil(c *C) {
+ s.makeFakeAndroidbootConfig(c)
+
+ dirs.GlobalRootDir = "/something/not/there"
@zyga

zyga May 16, 2017

Contributor

You don't want this, see my comment above

+ return err
+ }
+
+ rawEnv := bytes.Split(buf, []byte("\n"))
@zyga

zyga May 16, 2017

Contributor

I think this would be nicer with a bufio.Scanner

@chipaca

chipaca May 16, 2017

Member

yeah, this whole method needs restructuring around that. Please.

+ l := bytes.SplitN(env, []byte("="), 2)
+ // be liberal in what you accept
+ if len(l) < 2 {
+ continue
@zyga

zyga May 16, 2017

Contributor

Nitpick: please log weird stuff with logger.Noticef

@chipaca

chipaca May 16, 2017

Member

(or logger.Debugf if it's weird but harmless)

+ return err
+ }
+ }
+
@zyga

zyga May 16, 2017

Contributor

Is this the place to use our atomic write helpers?

@chipaca

chipaca May 16, 2017

Member

yep :-)

@chipaca

chipaca May 16, 2017

Member

also, one nice thing about bytes.Buffer is that you can just declare it and start using it; NewBuffer is only needed when you want to preload the buffer with something. Otherwise, just

var w bytes.Buffer

and you're done (although some things will need you to pass a pointer to this, e.g. things expecting an io.Writer).

Very nice, thank you!

A bunch of small things to fix, mostly idiomatic and not conceptual.

partition/androidboot.go
+}
+
+func (a *androidboot) GetBootVars(names ...string) (map[string]string, error) {
+ out := make(map[string]string)
@zyga

zyga May 16, 2017

Contributor

nitpick: move this closer to the loop that populates it.

@chipaca

chipaca May 16, 2017

Member

furthermore: you know how big it needs to be (len(names)), so make it with that size already

partition/androidboot_test.go
+ *
+ */
+
+package partition
@chipaca

chipaca May 16, 2017

Member

can you make this package partition_test, please? (maybe use export_test.go to get to the private bits you need for testing)

partition/androidboot_test.go
+func mockAndroidbootFile(c *C, newPath string, mode os.FileMode) {
+ newpath := filepath.Join(dirs.GlobalRootDir, "/boot/androidboot")
+ os.MkdirAll(newpath, os.ModePerm)
+ err := ioutil.WriteFile(newPath, nil, mode)
@chipaca

chipaca May 16, 2017

Member

what exactly is this supposed to be doing? I mean, what's the intention?
(because I'm pretty sure there's something wrong)

@alfonsosanchezbeato

alfonsosanchezbeato May 17, 2017

Contributor

This creates a folder and then a new file. I think it is confusing due to a variable named newpath and another one named newPath. I have changed that.

partition/androidboot_test.go
+
+ a := newAndroidboot()
+ c.Assert(a, NotNil)
+ c.Assert(a, FitsTypeOf, &androidboot{})
@chipaca

chipaca May 16, 2017

Member

in general please use Assert when something needs to be true for the rest of the test to make sense, but Check when otherwise (in this case, the second one would usually be a Check unless you were then doing a type assertion)

partition/androidboot_test.go
+
+ a := newAndroidboot()
+ bootVars := map[string]string{}
+ bootVars["snap_mode"] = "try"
@chipaca

chipaca May 16, 2017

Member

you could directly

bootVars := map[string]string{"snap_mode": "try"}
+ v, err := a.GetBootVars("snap_mode")
+ c.Assert(err, IsNil)
+ c.Check(v, HasLen, 1)
+ c.Check(v["snap_mode"], Equals, "try")
@chipaca

chipaca May 16, 2017

Member

(this would be the right way to use Assert vs Check; good :-) )

+ return err
+ }
+
+ rawEnv := bytes.Split(buf, []byte("\n"))
@zyga

zyga May 16, 2017

Contributor

I think this would be nicer with a bufio.Scanner

@chipaca

chipaca May 16, 2017

Member

yeah, this whole method needs restructuring around that. Please.

+ l := bytes.SplitN(env, []byte("="), 2)
+ // be liberal in what you accept
+ if len(l) < 2 {
+ continue
@zyga

zyga May 16, 2017

Contributor

Nitpick: please log weird stuff with logger.Noticef

@chipaca

chipaca May 16, 2017

Member

(or logger.Debugf if it's weird but harmless)

+ return err
+ }
+ }
+
@zyga

zyga May 16, 2017

Contributor

Is this the place to use our atomic write helpers?

@chipaca

chipaca May 16, 2017

Member

yep :-)

@chipaca

chipaca May 16, 2017

Member

also, one nice thing about bytes.Buffer is that you can just declare it and start using it; NewBuffer is only needed when you want to preload the buffer with something. Otherwise, just

var w bytes.Buffer

and you're done (although some things will need you to pass a pointer to this, e.g. things expecting an io.Writer).

+
+func (a *androidbootenvTestSuite) TestSet(c *C) {
+ env := androidbootenv.NewEnv(a.envPath)
+ c.Check(env, NotNil)
@chipaca

chipaca May 16, 2017

Member

... Assert :-)

+
+func (a *androidbootenvTestSuite) TestSaveAndLoad(c *C) {
+ env := androidbootenv.NewEnv(a.envPath)
+ c.Check(env, NotNil)
@chipaca

chipaca May 16, 2017

Member

here this should probably be an Assert (otherwise all the other tests will panic)

snap/gadget.go
foundBootloader = true
default:
- return nil, fmt.Errorf(errorFormat, "bootloader must be either grub or u-boot")
+ return nil, fmt.Errorf(errorFormat, "bootloader must be either grub, u-boot or android-boot")
@chipaca

chipaca May 16, 2017

Member

s/either/one of/

Let us know if there's anything we can do to help.

alfonsosanchezbeato added some commits May 17, 2017

Contributor

alfonsosanchezbeato commented May 17, 2017

PR resubmitted after addressing the requested changes. @zyga and @chipaca thanks for your thorough review and for being patient with a golang newbie like me :)

Looks very nice, thanks for doing this work. I have some comments inside that might be worth considering. But nothing major, it looks pretty nice.

+ file, err := os.Open(a.path)
+ if err != nil {
+ return err
+ }
@mvo5

mvo5 May 17, 2017

Collaborator

defer file.Close() is missing here.

@chipaca

chipaca May 17, 2017

Member

(fixed, below)

partition/export_test.go
+)
+
+// creates a new Androidboot bootloader object
+func MockNewAndroidboot() Bootloader {
@mvo5

mvo5 May 17, 2017

Collaborator

(nitpick) this can probably just be NewAndroidboot() nothing is mocked/faked here its just exported for the tests.

partition/export_test.go
+
+func MockAndroidbootFile(c *C, mode os.FileMode) {
+ f := &androidboot{}
+ newpath := filepath.Join(dirs.GlobalRootDir, "/boot/androidboot")
@mvo5

mvo5 May 17, 2017

Collaborator

(nitpick) might be easier to do: newpath := filepath.Dir(f.ConfigFile()) (or f.Dir()), then you don't need to encode the knowledge of the exact path into this test.

partition/export_test.go
+func MockAndroidbootFile(c *C, mode os.FileMode) {
+ f := &androidboot{}
+ newpath := filepath.Join(dirs.GlobalRootDir, "/boot/androidboot")
+ os.MkdirAll(newpath, os.ModePerm)
@mvo5

mvo5 May 17, 2017

Collaborator

(nitpick) I think it makes sense to get the err from os.MkdirAll() and c.Assert(err, IsNil).

@mvo5

mvo5 May 17, 2017

Collaborator

Also, os.ModePerm is 0777, even in its a test I would use 0755 here instead (just in case umask is funny etc).

Much nicer, thank you! A few more suggestions.

+
+ out := make(map[string]string, len(names))
+ for _, name := range names {
+ out[name] = env.Get(name)
@zyga

zyga May 17, 2017

Contributor

Can there be any duplicates in names?

@alfonsosanchezbeato

alfonsosanchezbeato May 17, 2017

Contributor

Not really, it would be a bug either in snapd or in the bootloader if that happens.

+ if err := env.Load(); err != nil && !os.IsNotExist(err) {
+ return err
+ }
+ for k, v := range values {
@zyga

zyga May 17, 2017

Contributor

Do we need to worry about overwriting values here?

@alfonsosanchezbeato

alfonsosanchezbeato May 17, 2017

Contributor

No, values should not be duplicated

partition/androidboot_test.go
+func (g *androidbootTestSuite) SetUpTest(c *C) {
+ dirs.SetRootDir(c.MkDir())
+
+ // the file needs to exist
@zyga

zyga May 17, 2017

Contributor

It would be more useful to explain why the file has to exist.

partition/androidboot_test.go
+}
+
+func (s *androidbootTestSuite) TestNewAndroidbootNoAndroidbootReturnsNil(c *C) {
+ dirs.SetRootDir("")
@zyga

zyga May 17, 2017

Contributor

Except when ran on android. I would suggest you to actually keep using the temporary random directory and populate it (or not populate it perhaps) so that it has the right data for the test. This will just run it on the real system which is not what unit tests are about.

partition/androidboot_test.go
+}
+
+func (s *androidbootTestSuite) TestSetGetBootVar(c *C) {
+ a := partition.MockNewAndroidboot()
@zyga

zyga May 17, 2017

Contributor

You can move this to the SetUpTest function as it seems to happen in all the tests.

@alfonsosanchezbeato

alfonsosanchezbeato May 17, 2017

Contributor

TestNewAndroidbootNoAndroidbootReturnsNil is unfortunately an exception.

+ "github.com/snapcore/snapd/osutil"
+)
+
+type Env struct {
@zyga

zyga May 17, 2017

Contributor

Some documentation (tip: run golint in the directory) would help here.

+ l := strings.SplitN(scanner.Text(), "=", 2)
+ // be liberal in what you accept
+ if len(l) < 2 {
+ logger.Noticef("WARNING: bad value while parsing %v", a.path)
@zyga

zyga May 17, 2017

Contributor

It might be invaluable to actually log scanner.Text() with %q here.

+}
+
+func (a *androidbootenvTestSuite) TestSet(c *C) {
+ env := androidbootenv.NewEnv(a.envPath)
@zyga

zyga May 17, 2017

Contributor

env := ... can be moved to SetUpTest

partition/export_test.go
+)
+
+// creates a new Androidboot bootloader object
+func MockNewAndroidboot() Bootloader {
@zyga

zyga May 17, 2017

Contributor

This is not a mock, it's doing the real thing.

Instead do this:

var NewAndroidBoot = newAndroidBoot

(Note that I also changed the capitalization of androidboot to AndroidBoot)

@alfonsosanchezbeato

alfonsosanchezbeato May 17, 2017

Contributor

Changed but to NewAndroidBoot instead

partition/export_test.go
+ return newAndroidboot()
+}
+
+func MockAndroidbootFile(c *C, mode os.FileMode) {
@zyga

zyga May 17, 2017

Contributor

Nitpick: spell it as androidBoot everywhere

nearly there! marking approved by me as my concerns are raised by zyga and mvo already :-)

Contributor

alfonsosanchezbeato commented May 17, 2017

Thanks again for the reviews, PR refreshed after addressing comments.

alfonsosanchezbeato added some commits May 17, 2017

mvo5 approved these changes May 18, 2017

One last set of nitpicks. LGTM then

+ }
+ a.env[l[0]] = l[1]
+ }
+
@zyga

zyga May 18, 2017

Contributor

I think you need to use the scanner.Err here. See https://golang.org/src/bufio/example_test.go for examples.

@alfonsosanchezbeato

alfonsosanchezbeato May 18, 2017

Contributor

Good catch, done

+func (a *Env) Save() error {
+ var w bytes.Buffer
+
+ for k, v := range a.env {
@zyga

zyga May 18, 2017

Contributor

Do we care about deterministic output here? If so please sort the keys, and write in order.

@alfonsosanchezbeato

alfonsosanchezbeato May 18, 2017

Contributor

Not really, neither the booloader or snapd cares about the order in which the variables are stored.

+ a.env.Set("key3", "value3")
+
+ err := a.env.Save()
+ c.Assert(err, IsNil)
@zyga

zyga May 18, 2017

Contributor

Can you add a trivial test that ensures the output is as you expect, as in the actual string is what we expect (deterministic)

@alfonsosanchezbeato

alfonsosanchezbeato May 18, 2017

Contributor

See previous comment

Contributor

alfonsosanchezbeato commented May 18, 2017

PR repushed after addressing comments

Codecov Report

Merging #3324 into master will decrease coverage by 0.01%.
The diff coverage is 73.77%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3324      +/-   ##
==========================================
- Coverage    77.6%   77.59%   -0.02%     
==========================================
  Files         364      366       +2     
  Lines       24956    25014      +58     
==========================================
+ Hits        19368    19409      +41     
- Misses       3865     3876      +11     
- Partials     1723     1729       +6
Impacted Files Coverage Δ
snap/gadget.go 82.35% <100%> (ø) ⬆️
partition/bootloader.go 80.55% <33.33%> (-4.74%) ⬇️
partition/androidbootenv/androidbootenv.go 66.66% <66.66%> (ø)
partition/androidboot.go 84.61% <84.61%> (ø)
interfaces/sorting.go 93.33% <0%> (-3.34%) ⬇️
overlord/ifacestate/helpers.go 62.86% <0%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eee977f...7824302. Read the comment docs.

zyga approved these changes May 19, 2017

LGTM, there are some tweaks we could do but I won't mind doing it in a follow-up.

@zyga zyga merged commit 004f307 into snapcore:master May 19, 2017

7 checks passed

artful-amd64 autopkgtest finished (success)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
xenial-amd64 autopkgtest finished (success)
Details
xenial-i386 autopkgtest finished (success)
Details
xenial-ppc64el autopkgtest finished (success)
Details
yakkety-amd64 autopkgtest finished (success)
Details
zesty-amd64 autopkgtest finished (success)
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment