errtracker: include bits of snap-confine apparmor profile #3421

Merged
merged 1 commit into from Jun 1, 2017

Conversation

Projects
None yet
4 participants
Contributor

zyga commented Jun 1, 2017

This patch extends the error report with information about the used
snap-confine apparmor profile (as expressed by the text of the file) as
well as hint of partial dpkg update.

Partial or failed updates can leave the old (previous) apparmor profile
of snap-confine around (since it is tagged as a conffile) and
subsequently cause any snap execution to fail with access to
"/run/snapd/lock" directory.

To test the theory we will now measure the hashes of the current and any
.dpkg-new versions of the apparmor profile of snap-confine and attach
them to the report.

Signed-off-by: Zygmunt Krynicki zygmunt.krynicki@canonical.com

@zyga zyga requested a review from mvo5 Jun 1, 2017

@zyga zyga added the Critical label Jun 1, 2017

chipaca approved these changes Jun 1, 2017

Does what it says on the tin.

I'm not sure if it does already, but if not, maybe also include the output of dpkg-query -W -f '${Status}' snapd?

errtracker/errtracker.go
+// https://forum.snapcraft.io/t/test-failures-with-cannot-create-lock-directory-run-snapd-lock/390/
+func detectPartialDpkgUpdate() string {
+ _, err := os.Stat(snapConfineProfileDpkgNew)
+ if err == nil {
@chipaca

chipaca Jun 1, 2017

Member

osutil.FileExists might be pithier

@zyga

zyga Jun 1, 2017

Contributor

Done (force pushed for easier cherry-picking)

Contributor

zyga commented Jun 1, 2017

Interesting, yes, I like both suggestions. @mvo5 how do you feel about the dpkg-query?

codecov-io commented Jun 1, 2017

Codecov Report

Merging #3421 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3421      +/-   ##
==========================================
+ Coverage   77.51%   77.52%   +0.01%     
==========================================
  Files         371      371              
  Lines       25537    25544       +7     
==========================================
+ Hits        19796    19804       +8     
  Misses       3986     3986              
+ Partials     1755     1754       -1
Impacted Files Coverage Δ
errtracker/errtracker.go 73.07% <100%> (+2.65%) ⬆️
interfaces/sorting.go 90% <0%> (-3.34%) ⬇️
overlord/snapstate/snapstate.go 81.5% <0%> (+0.23%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a8fe3e8...123542b. Read the comment docs.

Thanks for working on this! Added some small suggestions :)

errtracker/errtracker.go
+//
+// This probe is here to aid in resolving the following issue:
+// https://forum.snapcraft.io/t/test-failures-with-cannot-create-lock-directory-run-snapd-lock/390/
+func detectPartialDpkgUpdate() string {
@mvo5

mvo5 Jun 1, 2017

Collaborator

(nitpick) the name is not ideal, as far as dpkg is concerned its not a partial update, it did what it is supposed to do. Maybe detectStaleSnapConfineApparmorConffile or something?

@zyga

zyga Jun 1, 2017

Contributor

AFAIR this is also happening when a partial update (broken mid way, before snapd is configured) happens so this is why I called it like that. I can rename it to something neutral like detectSnapConfineDpkgNew

errtracker/errtracker.go
+// https://forum.snapcraft.io/t/test-failures-with-cannot-create-lock-directory-run-snapd-lock/390/
+func detectPartialDpkgUpdate() string {
+ if osutil.FileExists(snapConfineProfileDpkgNew) {
+ return "dpkg-new file present"
@mvo5

mvo5 Jun 1, 2017

Collaborator

Maybe the md5 hash of the file instead of this string? This way we might get a clue if all files are modified in the same way and we can compare with the hash of previous deb packages etc.

@zyga

zyga Jun 1, 2017

Contributor

aha, good idea

errtracker/errtracker.go
@@ -119,6 +135,7 @@ func Report(snap, errMsg, dupSig string, extra map[string]string) (string, error
"CoreSnapdBuildID": coreBuildID,
"Date": timeNow().Format(time.ANSIC),
"Snap": snap,
+ "PartialDpkgUpdate": detectPartialDpkgUpdate(),
@mvo5

mvo5 Jun 1, 2017

Collaborator

StaleSnapdApparmorConffile maybe? Also we should not add it if its empty I think.

@zyga

zyga Jun 1, 2017

Contributor

I think we can do two things:

  • show the flag that .dpkg-new is present
  • carry md5 of the actual profile

WDYT?

@zyga zyga changed the title from errtracker: include hints of partial dpkg update in error reports to errtracker: include bits of snap-confine apparmor profile Jun 1, 2017

Very nice! two tiny suggestions and then we should squash it and cherry pick in 2.26

errtracker/errtracker.go
+ if err != nil {
+ return ""
+ }
+ return fmt.Sprintf("%x", sha1.Sum(profileText))
@mvo5

mvo5 Jun 1, 2017

Collaborator

If we use md5 here we can more easily compare with the dpkg metadata. This way we can look at e.g. dpkg -s snapd and immediately see the md5sums of the conffiles without having to unpack the debs.

@zyga

zyga Jun 1, 2017

Contributor

Done

errtracker/errtracker.go
@@ -119,6 +130,8 @@ func Report(snap, errMsg, dupSig string, extra map[string]string) (string, error
"CoreSnapdBuildID": coreBuildID,
"Date": timeNow().Format(time.ANSIC),
"Snap": snap,
+ "SnapConfineCurrent": snapConfineProfileDigest(""),
@mvo5

mvo5 Jun 1, 2017

Collaborator

Maybe SnapConfineApparmorProfileCurrent to avoid possible confusion with the hash of the actual snap-confine binary? Same below.

@zyga

zyga Jun 1, 2017

Contributor

Done, I kept it brief for golang reflow reasons but I called it this way before :)

@zyga

zyga Jun 1, 2017

Contributor

Done, I kept it brief for golang reflow reasons but I called it this way before :)

errtracker: include bits of snap-confine apparmor profile
This patch extends the error report with information about the used
snap-confine apparmor profile (as expressed by the text of the file) as
well as hint of partial dpkg update.

Partial or failed updates can leave the old (previous) apparmor profile
of snap-confine around (since it is tagged as a conffile) and
subsequently cause any snap execution to fail with access to
"/run/snapd/lock" directory.

To test the theory we will now measure the hashes of the current and any
.dpkg-new versions of the apparmor profile of snap-confine and attach
them to the report.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

mvo5 approved these changes Jun 1, 2017

@mvo5 mvo5 merged commit 8911fd2 into snapcore:master Jun 1, 2017

1 of 7 checks passed

artful-amd64 autopkgtest running
Details
xenial-amd64 autopkgtest running
Details
xenial-i386 autopkgtest running
Details
xenial-ppc64el autopkgtest running
Details
yakkety-amd64 autopkgtest running
Details
zesty-amd64 autopkgtest running
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

zyga added a commit to zyga/snapd that referenced this pull request Jun 1, 2017

errtracker: include bits of snap-confine apparmor profile (#3421)
This patch extends the error report with information about the used
snap-confine apparmor profile (as expressed by the text of the file) as
well as hint of partial dpkg update.

Partial or failed updates can leave the old (previous) apparmor profile
of snap-confine around (since it is tagged as a conffile) and
subsequently cause any snap execution to fail with access to
"/run/snapd/lock" directory.

To test the theory we will now measure the hashes of the current and any
.dpkg-new versions of the apparmor profile of snap-confine and attach
them to the report.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

mvo5 added a commit that referenced this pull request Jun 1, 2017

errtracker: include bits of snap-confine apparmor profile (#3421) (#3424
)

This patch extends the error report with information about the used
snap-confine apparmor profile (as expressed by the text of the file) as
well as hint of partial dpkg update.

Partial or failed updates can leave the old (previous) apparmor profile
of snap-confine around (since it is tagged as a conffile) and
subsequently cause any snap execution to fail with access to
"/run/snapd/lock" directory.

To test the theory we will now measure the hashes of the current and any
.dpkg-new versions of the apparmor profile of snap-confine and attach
them to the report.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

@zyga zyga deleted the zyga:feature/detect-partial-updates branch Jun 5, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment