Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for snapd package manager #13

Merged
merged 2 commits into from
Mar 14, 2018

Conversation

noelmcloughlin
Copy link
Member

@noelmcloughlin noelmcloughlin commented Mar 13, 2018

This PR adds formula support for snapd package manager.

Verified on Fedora:

$ sudo salt-call state.highstate --local
[ERROR   ] Command '['dnf', 'versionlock', 'list']' failed with return code: 1
[ERROR   ] output: No such command: versionlock. Please use /usr/bin/dnf --help
It could be a DNF plugin command, try: "dnf install 'dnf-command(versionlock)'"
[ERROR   ] Command '['dnf', 'versionlock', 'list']' failed with return code: 1
[ERROR   ] output: No such command: versionlock. Please use /usr/bin/dnf --help
It could be a DNF plugin command, try: "dnf install 'dnf-command(versionlock)'"
local:
----------
          ID: pkg_req_pkgs
    Function: pkg.installed
      Result: True
     Comment: The following packages were installed/updated: snapd
     Started: 10:15:09.328165
    Duration: 81800.251 ms
     Changes:   
              ----------
              snap-confine:
                  ----------
                  new:
                      2.30-1.fc27
                  old:
              snapd:
                  ----------
                  new:
                      2.30-1.fc27
                  old:
              snapd-selinux:
                  ----------
                  new:
                      2.30-1.fc27
                  old:
              squashfs-tools:
                  ----------
                  new:
                      4.3-15.fc27
                  old:
----------
          ID: wanted_pkgs
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed.
              Package iptables is not being held..
              Package vim-minimal is not being held..
              Package lxde-common is not being held..
              Package redhat-lsb-core is not being held..
              Package python2-pip is not being held..
              Package ca-certificates is not being held.
     Started: 10:16:31.145721
    Duration: 326.031 ms
     Changes:   
----------
          ID: unwanted_pkgs
    Function: pkg.removed
      Result: True
     Comment: All specified packages are already absent
     Started: 10:16:31.472044
    Duration: 0.887 ms
     Changes:   
----------
          ID: pip_req_pkgs
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 10:16:31.473074
    Duration: 0.696 ms
     Changes:   
----------
          ID: gem_req_pkgs
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 10:16:31.473930
    Duration: 0.583 ms
     Changes:   
----------
          ID: packages-snapd.service-service
    Function: service.running
        Name: snapd.service
      Result: True
     Comment: Service snapd.service has been enabled, and is running
     Started: 10:16:31.980996
    Duration: 332.52 ms
     Changes:   
              ----------
              snapd.service:
                  True
----------
          ID: packages-snapd.socket-service
    Function: service.running
        Name: snapd.socket
      Result: True
     Comment: The service snapd.socket is already running
     Started: 10:16:32.314073
    Duration: 40.829 ms
     Changes:   
----------
          ID: packages-snapd-hello-world-wanted
    Function: cmd.run
        Name: snap install hello-world
      Result: True
     Comment: Command "snap install hello-world" run
     Started: 10:16:32.356819
    Duration: 15161.719 ms
     Changes:   
              ----------
              pid:
                  58238
              retcode:
                  0
              stderr:
              stdout:
                  
                  E?[7mnsure prerequisites for "hello-world" are availa?[0mble                                                                                          
                  En?[7msure prerequisites for "hello-world" are availab?[0mle                                                                                          
                  Fetc?[7mh and check assertions for snap "core" (4206)   ?[0m                                                                                          
                  Fetch?[7m and check assertions for snap "core" (4206)    ?[0m                                                                                         
                  Fetch a?[7mnd check assertions for snap "core" (4206)     ?[0m                                                                                        
                  Fetch an?[7md check assertions for snap "core" (4206)       ?[0m                                                                                      
                  Fetch and?[7m check assertions for snap "core" (4206)        ?[0m                                                                                     
                  Fetch and c?[7mheck assertions for snap "core" (4206)          ?[0m                                                                                   
                  Fetch and ch?[7meck assertions for snap "core" (4206)           ?[0m                                                                                  
                  Mount snap "co?[7mre" (4206)                                      ?[0m                                                                                
                  Mount snap "cor?[7me" (4206)                                       ?[0m                                                                               
                  Mount snap "core"?[7m (4206)                                        ?[0m                                                                              
                  Mount snap "core" ?[7m(4206)                                          ?[0m                                                                            
                  Setup snap "core" (?[7m4206) security profiles                         ?[0m                                                                           
                  Setup snap "core" (42?[7m06) security profiles (phase 2)                 ?[0m                                                                         
                  Setup snap "core" (420?[7m6) security profiles (phase 2)                  ?[0m                                                                        
                  ?[0m?[K2018-03-13T10:16:38-06:00 INFO Waiting for restart...
                  
                  Setup snap "core" (4206)?[7m security profiles (phase 2)                    ?[0m                                                                      
                  Setup snap "core" (4206) ?[7msecurity profiles (phase 2)                     ?[0m                                                                     
                  Setup snap "core" (4206) s?[7mecurity profiles (phase 2)                      ?[0m                                                                    
                  Setup snap "core" (4206) sec?[7murity profiles (phase 2)                        ?[0m                                                                  
                  Setup snap "core" (4206) secu?[7mrity profiles (phase 2)                         ?[0m                                                                 
                  Setup snap "core" (4206) securi?[7mty profiles (phase 2)                           ?[0m                                                               
                  Setup snap "core" (4206) securit?[7my profiles (phase 2)                            ?[0m                                                              
                  Setup snap "core" (4206) security ?[7mprofiles (phase 2)                             ?[0m                                                             
                  Setup snap "core" (4206) security p?[7mrofiles (phase 2)                               ?[0m                                                           
                  Setup snap "core" (4206) security pr?[7mofiles (phase 2)                                ?[0m                                                          
                  Setup snap "core" (4206) security prof?[7miles (phase 2)                                  ?[0m                                                        
                  Setup snap "core" (4206) security profi?[7mles (phase 2)                                   ?[0m                                                       
                  Setup snap "core" (4206) security profile?[7ms (phase 2)                                     ?[0m                                                     
                  Setup snap "core" (4206) security profiles?[7m (phase 2)                                      ?[0m                                                    
                  Setup snap "core" (4206) security profiles (?[7mphase 2)                                       ?[0m                                                   
                  Setup snap "core" (4206) security profiles (p?[7mhase 2)                                         ?[0m                                                 
                  Setup snap "core" (4206) security profiles (ph?[7mase 2)                                          ?[0m                                                
                  Setup snap "core" (4206) security profiles (phas?[7me 2)                                            ?[0m                                              
                  Setup snap "core" (4206) security profiles (phase?[7m 2)                                             ?[0m                                             
                  Setup snap "core" (4206) security profiles (phase 2?[7m)                                              ?[0m                                            
                  Setup snap "core" (4206) security profiles (phase 2)?[7m                                                ?[0m                                          
                  Setup snap "core" (4206) security profiles (phase 2) ?[7m                                                ?[0m                                         
                  Setup snap "core" (4206) security profiles (phase 2)   ?[7m                                                ?[0m                                       
                  Setup snap "core" (4206) security profiles (phase 2)    ?[7m                                                ?[0m                                      
                  Setup snap "core" (4206) security profiles (phase 2)      ?[7m                                                ?[0m                                    
                  Setup snap "core" (4206) security profiles (phase 2)       ?[7m                                                ?[0m                                   
                  Setup snap "core" (4206) security profiles (phase 2)         ?[7m                                               ?[0m                                  
                  Setup snap "core" (4206) security profiles (phase 2)          ?[7m                                                ?[0m                                
                  Setup snap "core" (4206) security profiles (phase 2)           ?[7m                                                ?[0m                               
                  Setup snap "core" (4206) security profiles (phase 2)             ?[7m                                                ?[0m                             
                  Setup snap "core" (4206) security profiles (phase 2)              ?[7m                                                ?[0m                            
                  Setup snap "core" (4206) security profiles (phase 2)                ?[7m                                                ?[0m                          
                  Setup snap "core" (4206) security profiles (phase 2)                 ?[7m                                                ?[0m                         
                  Setup snap "core" (4206) security profiles (phase 2)                   ?[7m                                               ?[0m                        
                  Setup snap "core" (4206) security profiles (phase 2)                    ?[7m                                                ?[0m                      
                  Setup snap "core" (4206) security profiles (phase 2)                     ?[7m                                                ?[0m                     
                  Setup snap "core" (4206) security profiles (phase 2)                       ?[7m                                                ?[0m                   
                  Setup snap "core" (4206) security profiles (phase 2)                        ?[7m                                                ?[0m                  
                  Setup snap "core" (4206) security profiles (phase 2)                          ?[7m                                               ?[0m                 
                  Setup snap "core" (4206) security profiles (phase 2)                           ?[7m                                                ?[0m               
                  Setup snap "core" (4206) security profiles (phase 2)                            ?[7m                                                ?[0m              
                  Setup snap "core" (4206) security profiles (phase 2)                              ?[7m                                                ?[0m            
                  Setup snap "core" (4206) security profiles (phase 2)                               ?[7m                                                ?[0m           
                  Setup snap "core" (4206) security profiles (phase 2)                                 ?[7m                                                ?[0m         
                  Setup snap "core" (4206) security profiles (phase 2)                                  ?[7m                                                ?[0m        
                  Setup snap "core" (4206) security profiles (phase 2)                                    ?[7m                                               ?[0m       
                  Setup snap "core" (4206) security profiles (phase 2)                                     ?[7m                                                ?[0m     
                  Setup snap "core" (4206) security profiles (phase 2)                                      ?[7m                                                ?[0m    
                  Setup snap "core" (4206) security profiles (phase 2)                                        ?[7m                                                ?[0m  
                  Setup snap "core" (4206) security profiles (phase 2)                                         ?[7m                                                ?[0m 
                  Setup snap "core" (4206) security profiles (phase 2)                                         ?[7m                                                ?[0m 
                  Setup snap "core" (4206) security profiles (phase 2)                                        ?[7m                                                ?[0m  
                  Setup snap "core" (4206) security profiles (phase 2)                                      ?[7m                                                ?[0m    
                  Set automatic aliases for snap "core"                                                    ?[7m                                                ?[0m     
                  Fetch and check assertions for snap "hello-world" (27)                                  ?[7m                                               ?[0m       
                  Fetch and check assertions for snap "hello-world" (27)                                ?[7m                                                ?[0m        
                  Fetch and check assertions for snap "hello-world" (27)                               ?[7m                                                ?[0m         
                  Fetch and check assertions for snap "hello-world" (27)                             ?[7m                                                ?[0m           
                  Fetch and check assertions for snap "hello-world" (27)                            ?[7m                                                ?[0m            
                  Fetch and check assertions for snap "hello-world" (27)                          ?[7m                                                ?[0m              
                  Fetch and check assertions for snap "hello-world" (27)                         ?[7m                                                ?[0m               
                  Fetch and check assertions for snap "hello-world" (27)                        ?[7m                                               ?[0m                 
                  Fetch and check assertions for snap "hello-world" (27)                      ?[7m                                                ?[0m                  
                  Mount snap "hello-world" (27)                                              ?[7m                                                ?[0m                   
                  Mount snap "hello-world" (27)                                            ?[7m                                                ?[0m                     
                  Mount snap "hello-world" (27)                                           ?[7m                                                ?[0m                      
                  Setup snap "hello-world" (27) security profiles                        ?[7m                                               ?[0m                        
                  Setup snap "hello-world" (27) security profiles                      ?[7m                                                ?[0m                         
                  Setup snap "hello-world" (27) security profiles                     ?[7m                                                ?[0m                          
                  Setup snap "hello-world" (27) security profiles                   ?[7m                                                ?[0m                            
                  Set automatic aliases for snap "hello-world"                     ?[7m                                                ?[0m                             
                  ?[0m?[?12;25h?[Khello-world 6.3 from 'canonical' installed
----------
          ID: packages-snapd-goodbye-world-unwanted
    Function: cmd.run
        Name: snap remove goodbye-world
      Result: True
     Comment: onlyif execution failed
     Started: 10:16:47.519558
    Duration: 38.403 ms
     Changes:   

Summary for local
------------
Succeeded: 9 (changed=3)
Failed:    0
------------
Total states run:     9
Total run time:  97.702 s

@noelmcloughlin noelmcloughlin added the enhancement New feature or request label Mar 13, 2018
@noelmcloughlin noelmcloughlin force-pushed the snapd_support branch 6 times, most recently from cdb2434 to 5b4bcfe Compare March 13, 2018 15:58
@javierbertoli
Copy link
Member

Hey, @noelmcloughlin. Thanks for this PR.

A question/issue:

The idea with this formula is to be able to install/manage packages in a very basic way, so installing and maintaning a service (like snapd) seems not in the scope of the formula at first. OTOH, installing and running the service is pretty much the same as making sure and installing python-pip before installing PIP packages, just that, in this case, it runs as a daemon.

I have never used snapd myself, so that's why I'd rather ask you 😄 : Am I right that this is like the latter? If that's the case, I'm OK with adding it here. If it requires/accept some sort of complex configuration (a la apt or yum), don't you agree that maybe it deserves its own formula?

@noelmcloughlin
Copy link
Member Author

noelmcloughlin commented Mar 13, 2018

I never used snapd before today, but need this for kubernetes dev stuff. After reviewing the technology, and considering a separate formula briefly, it seemed more logical to add here. As separate formula, it would still work best as clone of packages-formula afaics. But reading the documentation, and using the tool, it feels like pip type solution. So it fits IMHO - separate formula feels like overkill.

I'm trying to fix kitchen with systemd support.

I don't think snaps SLS is optimal (i.e. depends on pkgs.sls) but future PR could fix?

@@ -21,3 +21,12 @@ packages:
required:
states: []
pkgs: []
snaps:
package: snapd
collides: ['snap',]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Debian (and probably Ubuntu) is a gene manipulation software, not related to snap packages? Is this correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Fedora, snapd was conflicting with snap package? https://admin.fedoraproject.org/pkgdb/package/rpms/snap/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see. Debian's https://packages.debian.org/stretch/snap 😄

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha Ha. I thought your reference to gene manipulation was an allegory to something ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hehe, no, it's a 'real thing' 😄

packages.snaps.collides seems a candidate for osfamilymap.yml or osmap.yml, with an empty default, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Default get matched in osmap.yaml? I was grepping inside a bunch of salt formulas trying to find example of usage but didn't and not sure. But if it is supported, absolutely yes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the map.jinja file: the order is to set defaults from defaults.yml (so 'yes', to your question), override with osfamilymap, then with osmap, then with pillar. Unless something's wrong with the map.jinja file, which is always possible 😄

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but I was wondering if following stanza works in osmap.yml? Will this match any OS not already listed?

...

Default:
   snaps:
      collides:  ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, not Default, but Fedora. It should be another part of the existing Fedora entry in osmap.yml, just like the existing one for pips. I think that will do.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested on Ubuntu and no collision. Snap is not in OpenSUSE repo. snapd is not available for Centos yet.

Looks like Fedora only issue. I have updated the commit accordingly.

extend:
unwanted_pkgs:
pkg.removed:
- pkgs: {{ packages.snaps.collides }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As packages.snaps.collides is always set in defaults.yml, some packages will be always removed even if you don't want to manage snap packages. This will happen, at least with snap (see above)

Perhaps we can keep all these packages removals conditioned to packages.snaps.wanted != [] or packages.snaps.unwanted != []?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to change this - if you clearly see a nice solution - pls suggest.

So should I wrap extend inside jinja do you think? or maybe onlyif/unless.

(BTW: Normally extends goes to init.sls where includes are centralized but one thing at a time).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

onlyif/unless are 'cleaner' to me, but in this case, as there are 2 or'ed conditions and a few places where to use them, perhaps an if block surrounding all these will do for now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@javierbertoli Thanks for the review.

I agree. I've wrapped block with Jinja2 and I think the comparison operators are not needed - Lets see if Travis CI works and I test again on Fedora shortly.

Copy link
Member Author

@noelmcloughlin noelmcloughlin Mar 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested packages and packages.pkgs, gems, pips (i.e. without snaps SLS) on Fedora and both high-states were successful.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Travis CI was successful - Issue #14 looks like noise on Debian.

{% endfor %}
{% endif %}

{% for snap in packages.snaps.service %}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, perhaps we can install and manage the service conditioned to packages.snaps.wanted != [] or packages.snaps.unwanted != []?

@noelmcloughlin
Copy link
Member Author

Debian is working now in kitchen test. This PR closes #14

�[36m       Summary for local�[0m
�[36m       -------------�[0;0m�[0m
�[36m       �[0;32mSucceeded: 21�[0;0m (�[0;32mchanged=12�[0;0m)�[0m
�[36m       �[0;36mFailed:     0�[0;0m�[0m

@noelmcloughlin noelmcloughlin force-pushed the snapd_support branch 3 times, most recently from e611c7c to 1b5dae8 Compare March 13, 2018 19:46
@noelmcloughlin
Copy link
Member Author

While using snap I found following issue on Fedora:

"classic confinement requires snaps under /snap or symlink from /snap to /var/lib/snapd/snap"

I updated PR to resolve this issue on Fedora only.

          ID: packages-snap-classic-symlink
    Function: file.symlink
        Name: /snap
      Result: True
     Comment: Created new symlink /snap -> /var/lib/snapd/snap
     Started: 13:48:41.172376
    Duration: 6028.969 ms
     Changes:   
              ----------
              new:
                  /snap

@javierbertoli
Copy link
Member

javierbertoli commented Mar 13, 2018

BTW, reading this:

"classic confinement requires snaps under /snap or symlink from /snap to /var/lib/snapd/snap"

I have a silly sls to manage symlinks based on pillar data, pretty much the same way this formula does. Do you think it's worth to turn it into a formula?

@noelmcloughlin
Copy link
Member Author

noelmcloughlin commented Mar 13, 2018

I'm not sure, maybe. To be honest I rarely need to create symlinks (except file.alternatives), but ask around to see if others are interested.

BTW, I forked this repo while back, with intention of checking to see for potential formulas. He's handling some symlinks too ...


{% for dst, src in pillar['dots'].items() %}

{{ home }}/{{ dst }}:
  file.symlink:
    - target: {{ dots }}/{{ src }}
    - user: {{ user }}
    - makedirs: True
    - force: True
    - require:
      - dots-repo

{% endfor %}

@noelmcloughlin
Copy link
Member Author

Snap is working for me now on Fedora.

 $ sudo snap install conjure-up --classic
conjure-up 2.5.2-20180113.1501 from 'canonical' installed

Travis was stuck - I rebooted the job.

@noelmcloughlin noelmcloughlin force-pushed the snapd_support branch 2 times, most recently from 7579f5f to a402ccc Compare March 14, 2018 11:56
@noelmcloughlin
Copy link
Member Author

noelmcloughlin commented Mar 14, 2018

Hi @javierbertoli I fixed kitchen.yml for Debian and Ubuntu so udev package gets installed. Fedora also works. There is some problem with Centos travis CI job which is unrelated to this PR. Could this be merged and centos kitchen issue be fixed separately?

�[35m           Rendering SLS 'base:epel' failed: Jinja variable 'dict object' has no attribute 'testing'�[0;0m�[0m
�[31m>>>>>> ------Exception-------�[0m
�[31m>>>>>> Class: Kitchen::ActionFailed�[0m
�[31m>>>>>> Message: 1 actions failed.
>>>>>>     Converge failed on instance <centos-centos-7>.  Please see .kitchen/logs/centos-centos-7.log for more details�[0m
�[31m>>>>>> ----------------------�[0m
�[31m>>>>>> Please see .kitchen/logs/kitchen.log for more details�[0m
�[31m>>>>>> Also try running `kitchen diagnose --all` for configuration

Copy link
Member

@javierbertoli javierbertoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@noelmcloughlin , would you agree to one last change and we can merge?

package: snapd
collides: []
symlink: False
service: ['snapd.service', 'snapd.socket',]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind you just change this line to

service: ['snapd']

so the salt uses whatever the OS has (systemd, upstart, etc) and that's it? I checked and snapd.socket is a dependency of snapd.service, so there's no need to include it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for suggestion, fixing that now.

@noelmcloughlin
Copy link
Member Author

Hi @javierbertoli just one non-optimal implementation detail.

Today, we use following pattern in pip / gems SLS.

{% set req_states = packages.pips.required.states %}
{% set req_pkgs = packages.pips.required.pkgs %}
{% set wanted_pips = packages.pips.wanted %}
{% set unwanted_pips = packages.pips.unwanted %}

### REQ PKGS (without these, some of the WANTED PIPS will fail to install)
pip_req_pkgs:
  pkg.installed:
    - pkgs: {{ req_pkgs }}

My PR follows the pip/gems pattern but introduces extends/include in snap SLS.

{% set req_states = packages.snaps.required.states %}
{% set wanted_snaps = packages.snaps.wanted %}
{% set unwanted_snaps = packages.snaps.unwanted %}

{% if packages.snaps.wanted or packages.snaps.unwanted %}

### REQ PKGS (without this, SNAPS can fail to install/uninstall)
include:
  - packages.pkgs

extend: 
... etc ...

I think this is fine for now. but overall not optimal. Other formulae seem to place extends/includes inside init.SLS. Just flagging this as maybe improvement needed?

@javierbertoli
Copy link
Member

@noelmcloughlin submitted a PR to fix the CentOS issue.

@noelmcloughlin
Copy link
Member Author

noelmcloughlin commented Mar 14, 2018

Thanks, starting Travis CI job again. Correction - seems to be running already.

I created #15 for extends question.

@javierbertoli javierbertoli merged commit 0bea34b into saltstack-formulas:master Mar 14, 2018
@noelmcloughlin noelmcloughlin deleted the snapd_support branch March 14, 2018 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants