lifecycle: clean after deleting container #1587

kalikiana · 2017-10-04T10:57:37Z

Have you followed the guidelines for contributing?
Have you signed the CLA?
If this is a bugfix. Have you checked that there is a bug report open for the issue you are trying to fix on bug reports?
If this is a new feature. Have you discussed the design on the forum?
Have you successfully run ./runtests.sh static?
Have you successfully run ./runtests.sh unit?

come-maiz · 2017-10-07T06:11:34Z

snapcraft/cli/lifecycle.py

    project_options = get_project_options(**kwargs)
    container_config = env.get_container_config()
-    if container_config.use_container:
-        step = step or 'pull'
+    if container_config.use_container and step == 'pull' and not parts:


I don't get this @kalikiana. I might be doing something wrong while testing it, but it seems to me that this still just deletes the container.
The execute method in lxd checks if we are cleaning, and if so, it just deletes the container.
In this pr I see nothing to modify that behaviour.

Sorry about that. It looks like the second commit was lost... I re-did the missing piece. So: clean should be called with the container to delete it, in all other cases "normal" lifecycle clean is used so that it works even if the container is broken or already deleted.

I'd like to rectify the poor separation of concerns, here. We're going to great lengths to ensure we only ever call lifecycle.containerbuild('clean') if step == 'pull' and not parts, with no apparent reason. We then proceed to pass the step as well as the parts (that we just ensured was empty) to the very generic-sounding lifecycle.containerbuild() method. That method then calls the very generic-sounding lxd.Project().execute() method. Finally, that method checks to see if we're cleaning the pull step with no parts, in which case it blows the container away.

I'd like to see lxd.Project().execute() be written more generically. If it was asked to clean with no parts, fine, blow the container away. If it was asked to clean with parts, do nothing. If something else, call super(). Then we don't have to let the conditions where the container needs to be removed leak to this line, where we're checking those conditions again. We can simply call lifecycle.containerbuild('clean') just like we call lifecycle.clean() and expect it to do the right thing.

kalikiana · 2017-10-10T12:08:50Z

Note: All but the snap test have passed

kyrofa

Good progress here, thank you! However, there's a concerns leakage that looks like a bug waiting to happen. I suggest we fix that.

kyrofa · 2017-10-10T16:44:50Z

snapcraft/cli/lifecycle.py

    project_options = get_project_options(**kwargs)
    container_config = env.get_container_config()
-    if container_config.use_container:
-        step = step or 'pull'
+    if container_config.use_container and step == 'pull' and not parts:


I'd like to rectify the poor separation of concerns, here. We're going to great lengths to ensure we only ever call lifecycle.containerbuild('clean') if step == 'pull' and not parts, with no apparent reason. We then proceed to pass the step as well as the parts (that we just ensured was empty) to the very generic-sounding lifecycle.containerbuild() method. That method then calls the very generic-sounding lxd.Project().execute() method. Finally, that method checks to see if we're cleaning the pull step with no parts, in which case it blows the container away.

I'd like to see lxd.Project().execute() be written more generically. If it was asked to clean with no parts, fine, blow the container away. If it was asked to clean with parts, do nothing. If something else, call super(). Then we don't have to let the conditions where the container needs to be removed leak to this line, where we're checking those conditions again. We can simply call lifecycle.containerbuild('clean') just like we call lifecycle.clean() and expect it to do the right thing.

kalikiana · 2017-10-11T14:13:25Z

@kyrofa Somehow I can reply... something's off with GitHub since today I think. In any case, I changed it as per your suggestion and the logic is only in lxd.py now.

kyrofa · 2017-10-11T22:18:29Z

snapcraft/internal/lxd.py

+        if step == 'clean':
+            # we don't execute clean here but rely on CLI to do it
+            # clean with no parts deletes the container
+            if self._get_container_status() and args == ['--step', 'pull']:


I think re-purposing this function to clean is both confusing (clean is not a lifecycle step, and then one sees --step pull and gets more confused) and is causing the diff of this PR to be larger than it needs to be (changing the step from None to 'pull' everywhere, which seems unnecessary). How difficult would it be to simply add a clean function here that accepts a step (the step being cleaned) and args?

Eh, I changed my mind on defaulting step to pull at a higher level, otherwise you'll need to treat None and 'pull' as equivalent here. However, I would still like to see a clean function here instead of execute.

I reverted the default change now. It's not essential to the PR, it just seemed like a simplification.

And I replaced the subclass' execute logic with a delete method - now lifecycle remains in charge and reading lxd.py doesn't require any knowledge of what lifecycle does with it.

kyrofa · 2017-10-11T22:23:55Z

snapcraft/internal/lifecycle.py

-    if not step:
-        step = 'pull'
-
+def clean(project_options, parts, step):


Why not just default step='pull' so it continues working the way it was?

kyrofa · 2017-10-11T22:24:25Z

snapcraft/tests/test_lifecycle.py

@@ -653,7 +653,7 @@ def test_clean_removes_global_state(self):
    plugin: nil
 """)
        lifecycle.execute('pull', self.project_options)
-        lifecycle.clean(self.project_options, parts=None)
+        lifecycle.clean(self.project_options, parts=None, step='pull')


If you use pull as the default, this is unnecessary.

kalikiana · 2017-10-13T17:45:22Z

Tests are green again!

kyrofa

This seems fine to me, now. We'll need to refactor it if the container ever needs to actually do something when individual parts or steps are cleaned, but this solves the problem for now.

come-maiz

If I understand correctly, this calls the clean commands outside of the container. I can see no issue right now, but I think this has the potential to break. I think that if the build is done in the container, the clean should execute in there too. But for now, I think this is a good improvement. Maybe we care about this when it becomes an issue, not now.

sergiusens · 2017-10-16T20:25:27Z

the business logic seems to have spread a bit too much into the cli components, hasn't it?

kyrofa · 2017-10-16T20:37:05Z

Yes indeed.

sergiusens

Sorry to say, but it would be good to prevent leaking too much into this.

It is acceptable if you want to surface UI (click.echo, echo.warn or prompts) but not for usage of moving common logic.

I would personally rather have it repeated.

kalikiana · 2017-10-20T13:01:03Z

@sergiusens I moved all logic back into the container, in the clean() method. Lifecycle knows nothing again like it did before.

come-maiz

To me, this looks more readable than before. Reapproving.

Significant change occurred.

kalikiana · 2017-10-25T10:26:15Z

I updated the branch against master (no change in code, but it now lives in _project.py).

Moved the logic back into the container, out of lifecycle. Needs re-checking.

kyrofa · 2017-10-30T14:21:14Z

snapcraft/cli/lifecycle.py

@@ -175,11 +177,12 @@ def clean(parts, step, **kwargs):
    """
    project_options = get_project_options(**kwargs)
    container_config = env.get_container_config()
+    step = step or 'pull'


You can fix LP: #1727445 right here by not doing this and handling both cases within Project.clean().

lifecycle: clean after deleting container

8756942

kalikiana self-assigned this Oct 6, 2017

come-maiz reviewed Oct 7, 2017

View reviewed changes

Christian added 2 commits October 8, 2017 22:34

Always clean outside of container

af278ac

Fix unit test failures

034a77b

kyrofa suggested changes Oct 10, 2017

View reviewed changes

Only define container clean behavior in one place

52f8680

kyrofa reviewed Oct 11, 2017

View reviewed changes

Christian added 2 commits October 12, 2017 14:17

Revert consolidation of step default in CLI

eef9a8e

Move deleting into a dedicated method

4e137b0

kalikiana force-pushed the lxd-clean-clean branch from 7dce61d to 4e137b0 Compare October 12, 2017 12:37

Merge branch 'master' into lxd-clean-clean

202421f

kyrofa previously approved these changes Oct 13, 2017

View reviewed changes

come-maiz approved these changes Oct 13, 2017

View reviewed changes

sergiusens previously requested changes Oct 16, 2017

View reviewed changes

Christian added 3 commits October 20, 2017 14:27

Merge branch 'master' into lxd-clean-clean

76862d5

Move all clean logic back into lxd

727d68c

Assert that lifecycle was called outside container

dcf3cad

come-maiz approved these changes Oct 24, 2017

View reviewed changes

Merge branch 'master' into lxd-clean-clean

3691bc9

sergiusens added this to the 2.35 milestone Oct 26, 2017

kyrofa reviewed Oct 30, 2017

View reviewed changes

sergiusens added the Bug Actual bad behavior that don't fall into maintenance or documentation label Oct 30, 2017

sergiusens merged commit daca461 into canonical:master Oct 30, 2017

sergiusens mentioned this pull request Nov 1, 2017

Release changelog for 2.35 #1650

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lifecycle: clean after deleting container #1587

lifecycle: clean after deleting container #1587

kalikiana commented Oct 4, 2017 •

edited

come-maiz Oct 7, 2017

kalikiana Oct 8, 2017

kyrofa Oct 10, 2017

kalikiana commented Oct 10, 2017

kyrofa left a comment

kyrofa Oct 10, 2017

kalikiana commented Oct 11, 2017

kyrofa Oct 11, 2017 •

edited

kyrofa Oct 11, 2017

kalikiana Oct 12, 2017

kyrofa Oct 11, 2017

kyrofa Oct 11, 2017

kalikiana commented Oct 13, 2017

kyrofa left a comment

come-maiz left a comment

sergiusens commented Oct 16, 2017

kyrofa commented Oct 16, 2017

sergiusens left a comment

kalikiana commented Oct 20, 2017

come-maiz left a comment

kalikiana commented Oct 25, 2017

kyrofa Oct 30, 2017 •

edited

lifecycle: clean after deleting container #1587

lifecycle: clean after deleting container #1587

Conversation

kalikiana commented Oct 4, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kalikiana commented Oct 10, 2017

kyrofa left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kalikiana commented Oct 11, 2017

kyrofa Oct 11, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kalikiana commented Oct 13, 2017

kyrofa left a comment

Choose a reason for hiding this comment

come-maiz left a comment

Choose a reason for hiding this comment

sergiusens commented Oct 16, 2017

kyrofa commented Oct 16, 2017

sergiusens left a comment

Choose a reason for hiding this comment

kalikiana commented Oct 20, 2017

come-maiz left a comment

Choose a reason for hiding this comment

kalikiana commented Oct 25, 2017

kyrofa Oct 30, 2017 • edited

Choose a reason for hiding this comment

kalikiana commented Oct 4, 2017 •

edited

kyrofa Oct 11, 2017 •

edited

kyrofa Oct 30, 2017 •

edited