overlord,daemon,cmd: re-map snap names around the edges of snapd #5491

zyga · Jul 10, 2018

This branch provides the support code for allowing snapd to re-map snap names in interface interactions around the edges of the system, namely the state and the API.

The branch contains three main segments:

the remapping support code in the interface manager (two patches)
re-mapping of connection state ({get,set}Conns) (one patch)
re-mapping of API requests and responses (one patch)
discarding commented-out stale test (one patch)

Updates over weekend:

reduce the code to just snap name re-mapping
unify with and replace the nickname system
enable the core-core-system mapper (system in the API, core in state and memory)

codecov-io · Jul 10, 2018

Codecov Report

Merging #5491 into master will increase coverage by 0.12%.
The diff coverage is 95.09%.

@@            Coverage Diff             @@
##           master    #5491      +/-   ##
==========================================
+ Coverage   78.95%   79.07%   +0.12%     
==========================================
  Files         514      514              
  Lines       38854    38918      +64     
==========================================
+ Hits        30678    30776      +98     
+ Misses       5710     5673      -37     
- Partials     2466     2469       +3

Impacted Files	Coverage Δ
snap/info.go	`83.47% <ø> (-0.38%)`	⬇️
overlord/configstate/configstate.go	`87.5% <100%> (+2.08%)`	⬆️
daemon/api.go	`77.33% <100%> (+2.49%)`	⬆️
cmd/snap/cmd_interfaces.go	`85.45% <100%> (ø)`	⬆️
cmd/snap/cmd_interface.go	`79.54% <75%> (ø)`	⬆️
overlord/ifacestate/helpers.go	`70.91% <93.65%> (+4.46%)`	⬆️
overlord/ifacestate/handlers.go	`64.88% <0%> (-0.4%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 999c6bd...e969d39. Read the comment docs.

pedronis · Jul 13, 2018

pedronis reviewed Jul 13, 2018

View changes

the code looks good, still have some high level questions/concerns

pedronis · Jul 13, 2018

overlord/ifacestate/helpers.go

+// Data coming from the state and from API requests is changed so that slots on "core"
+// become slots on "snapd" (but only when "snapd" snap itself is being used). When
+// data is about to hit the state again it is re-mapped back.
+func RemapIncomingConnRef(cref *interfaces.ConnRef) (changed bool) {


I think it would be safer if these Remap* functions didn't do changes in place but where functional (the internal interface can keep being in place)

pedronis · Jul 13, 2018

daemon/api.go

 	for _, info := range infos {
 		// Convert interfaces.Info into interfaceJSON
 		plugs := make([]*plugJSON, 0, len(info.Plugs))
 		for _, plug := range info.Plugs {
+			// Re-map outgoing plug information.


at this point in API if understand correctly we will possibly return "core" in some situations even if the snap doesn't exist, right?

wondering if we should go all the way and as we discussed skip this intermediate state and return "system" instead, it means that the mapping to state vs the mapping for the outgoing API world need to be different, or that we need another layer of mapping just here (core => system)

Because the mapping is puggable we can easily return system here. It is all defined by the CoreSnapdMapper (which should be renamed). I can change that to return system for both snapd and core instead.

I am looking at what it would take to return "system" to the API and "core" to to the state now.

pedronis · Jul 13, 2018

overlord/ifacestate/helpers.go

+	return changedPlug || changedSlot
+}
+
+// RemapOutgoingPlugRef potentially re-maps the snap name of a plug reference.


given that snapd will have only implicit slots afawk, remapping of plugs is only for completeness but is not stricly needed right?

mvo5 · Jul 16, 2018

mvo5 reviewed Jul 16, 2018

View changes

Looks nice! Thanks for working on this.

mvo5 · Jul 16, 2018

daemon/api_test.go

+	ifacestate.IdentityMapper // Embed the identity mapper to reuse empty state mapping functions.
+}
+
+func (m *inverseCaseMapper) RemapSnapFromRequest(snapName string) string {


(nitpick^2) I wonder if we should have something like from-req-$snapName (and vice for the RemapSnapToResponse).

mvo5 · Jul 16, 2018

mvo5 reviewed Jul 16, 2018

View changes

mvo5 · Jul 16, 2018

daemon/api.go

@@ -1647,7 +1647,7 @@ func splitQS(qs string) []string {

 func getSnapConf(c *Command, r *http.Request, user *auth.UserState) Response {
 	vars := muxVars(r)
-	snapName := snap.DropNick(vars["name"])
+	snapName := ifacestate.RemapSnapFromRequest(vars["name"])


For configuration we may need our own set of of remap helpers. Last time we discussed this the idea was that we map all configuration from "system" -> "core" even on a core18 system. If this has changed thats fine but we probably still need to tweak things a bit because e.g. "canConfigure()" allows setting "core" even if core is not installed. If we change where the "system" configuration is stored we probably need to update "canConfigure"

it feels also strange to use a helper from ifacestate in configuration related code

One simple fix would be to have similar remap helpers in configstate.

This is incorrect application of IMO correct logic. This simple tweak ought to fix this:

snapName := ifacestate.RemapSnapToStore(ifacestate.RemapSnapFromRequest(vars["name"]))

pedronis · Jul 16, 2018

pedronis requested changes Jul 16, 2018

View changes

using the helpers from ifacestate for the config bits feels confusing and seems also wrong, we need to map system to core in any case there or do some other changes

pedronis · Jul 16, 2018

daemon/api.go

@@ -1647,7 +1647,7 @@ func splitQS(qs string) []string {

 func getSnapConf(c *Command, r *http.Request, user *auth.UserState) Response {
 	vars := muxVars(r)
-	snapName := snap.DropNick(vars["name"])
+	snapName := ifacestate.RemapSnapFromRequest(vars["name"])


it feels also strange to use a helper from ifacestate in configuration related code

pedronis · Jul 16, 2018

overlord/ifacestate/helpers.go

+// even if the request used "core".
+func (m *CoreSnapdSystemMapper) RemapSnapFromRequest(snapName string) string {
+	if snapName == "system" || snapName == "core" {
+		return "snapd"


this is wrong for the use we do for config!

pedronis · Jul 16, 2018

pedronis reviewed Jul 16, 2018

View changes

pedronis · Jul 16, 2018

snap/mapper.go

+	return snapName
+}
+
+func (m *CoreSnapdSystemMapper) ConfigIfaceRemapSnapToResponse(snapName string) string {


is ConfigIface a typo? is there no tests failing?

Yeah, sorry for this.

pedronis · Jul 16, 2018

snap/mapper.go

+ *
+ */
+
+package snap


I'm not sure snap is the best place for this, a mapper makes sense and can be chosen only inside snapd

I pushed a simpler version that just adds a remap helper into configstate now.

pedronis · Jul 17, 2018

pedronis approved these changes Jul 17, 2018

View changes

+1, with some small comments

pedronis · Jul 17, 2018

overlord/configstate/configstate.go

@@ -120,3 +120,19 @@ func Configure(st *state.State, snapName string, patch map[string]interface{}, f
 	task := hookstate.HookTask(st, summary, hooksup, contextData)
 	return state.NewTaskSet(task)
 }
+
+// RemapSnapFromRequest renames a snap as received from an API request
+func RemapSnapFromRequest(snapName string) string {


these need unit tests?

Yes, I just wrote those.

pedronis · Jul 17, 2018

overlord/ifacestate/helpers.go

+// exactly the same as with the "snapd" mapper. This can be used to make any
+// necessary adjustments the test suite.
+type CoreCoreSystemMapper struct {
+	IdentityMapper // Embedding the nil mapper allows us to cut on boilerplate.


nil mapper seems the wrong name now? same below

Fixed here and below.

pedronis · Jul 17, 2018

overlord/configstate/configstate.go

+}
+
+// RemapSnapToResponse renames a snap as about to be sent from an API response
+func RemapSnapToResponse(snapName string) string {


is this ever used?

No it is not. I'm looking if it should be used now.

pedronis · Jul 17, 2018

pedronis reviewed Jul 17, 2018

View changes

pedronis · Jul 17, 2018

daemon/api.go

@@ -1714,7 +1714,7 @@ func setSnapConf(c *Command, r *http.Request, user *auth.UserState) Response {
 	taskset, err := configstate.ConfigureInstalled(st, snapName, patchValues, 0)
 	if err != nil {
 		if _, ok := err.(*snap.NotInstalledError); ok {
-			return SnapNotFound(snapName, err)
+			return SnapNotFound(configstate.RemapSnapToResponse(snapName), err)


we will not hit this case because core/system can be configured before anything is installed

Removed now

mvo5 · Jul 17, 2018

mvo5 approved these changes Jul 17, 2018

View changes

zyga added the ⛔ Blocked label Jul 10, 2018

zyga force-pushed the zyga:feature/plug-slot-remapping branch from 679b1de to 8056bc0 Jul 10, 2018

zyga referenced this pull request Jul 10, 2018
Closed
overlord/ifacestate: add re-mapping function for plugs and slots #5492

zyga force-pushed the zyga:feature/plug-slot-remapping branch from 8056bc0 to 72f358b Jul 12, 2018

zyga added Core18 and removed ⛔ Blocked labels Jul 12, 2018

pedronis requested a review from niemeyer Jul 13, 2018

zyga changed the title from overlord,daemon: re-map interface plugs and slots around the edges of snapd to overlord,daemon,cmd: re-map snap names around the edges of snapd Jul 13, 2018

configstate: use config specific mapper

Loading status checks…

292c38b

mvo5 force-pushed the zyga:feature/plug-slot-remapping branch from ff338c7 to 292c38b Jul 16, 2018

zyga force-pushed the zyga:feature/plug-slot-remapping branch from e969d39 to 339a091 Jul 17, 2018

mvo5 merged commit 1544ced into snapcore:master Jul 17, 2018
1 check passed

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

zyga deleted the zyga:feature/plug-slot-remapping branch Jul 17, 2018

snapcore/snapd

Join GitHub today

overlord,daemon,cmd: re-map snap names around the edges of snapd #5491

Conversation

zyga commented Jul 10, 2018 • edited

zyga added the ⛔ Blocked label Jul 10, 2018

zyga force-pushed the zyga:feature/plug-slot-remapping branch from 679b1de to 8056bc0 Jul 10, 2018

zyga referenced this pull request Jul 10, 2018

This comment has been minimized.

codecov-io commented Jul 10, 2018 • edited

Codecov Report

zyga added some commits Jul 9, 2018

zyga force-pushed the zyga:feature/plug-slot-remapping branch from 8056bc0 to 72f358b Jul 12, 2018

zyga added Core18 and removed ⛔ Blocked labels Jul 12, 2018

pedronis reviewed Jul 13, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

pedronis requested a review from niemeyer Jul 13, 2018

zyga added some commits Jul 13, 2018

zyga changed the title from overlord,daemon: re-map interface plugs and slots around the edges of snapd to overlord,daemon,cmd: re-map snap names around the edges of snapd Jul 13, 2018

mvo5 reviewed Jul 16, 2018 View changes

This comment has been minimized.

mvo5 reviewed Jul 16, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

pedronis requested changes Jul 16, 2018 View changes

This comment has been minimized.

This comment has been minimized.

pedronis reviewed Jul 16, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

mvo5 force-pushed the zyga:feature/plug-slot-remapping branch from ff338c7 to 292c38b Jul 16, 2018

pedronis approved these changes Jul 17, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

zyga added some commits Jul 17, 2018

pedronis reviewed Jul 17, 2018 View changes

This comment has been minimized.

This comment has been minimized.

mvo5 approved these changes Jul 17, 2018 View changes

zyga force-pushed the zyga:feature/plug-slot-remapping branch from e969d39 to 339a091 Jul 17, 2018

Hide details View details mvo5 merged commit 1544ced into snapcore:master Jul 17, 2018 1 check passed

1 check passed

zyga deleted the zyga:feature/plug-slot-remapping branch Jul 17, 2018

zyga commented Jul 10, 2018 •

edited

zyga force-pushed the zyga:feature/plug-slot-remapping branch from `679b1de` to `8056bc0` Jul 10, 2018

codecov-io commented Jul 10, 2018 •

edited

zyga force-pushed the zyga:feature/plug-slot-remapping branch from `8056bc0` to `72f358b` Jul 12, 2018

pedronis reviewed Jul 13, 2018

View changes

mvo5 reviewed Jul 16, 2018

View changes

mvo5 reviewed Jul 16, 2018

View changes

pedronis requested changes Jul 16, 2018

View changes

pedronis reviewed Jul 16, 2018

View changes

mvo5 force-pushed the zyga:feature/plug-slot-remapping branch from `ff338c7` to `292c38b` Jul 16, 2018

pedronis approved these changes Jul 17, 2018

View changes

pedronis reviewed Jul 17, 2018

View changes

mvo5 approved these changes Jul 17, 2018

View changes

zyga force-pushed the zyga:feature/plug-slot-remapping branch from `e969d39` to `339a091` Jul 17, 2018

mvo5 merged commit `1544ced` into snapcore:master Jul 17, 2018
1 check passed