Restart salt-minion *after* end of run

[tlemarchand]

Hi all,

Is there a reason why something like : http://docs.saltstack.com/en/latest/faq.html#what-is-the-best-way-to-restart-a-salt-daemon-using-salt is not used in this formula ?

I changed salt-formula (locally), it's a huge win : I can now spawn a new salt minion (with salt-cloud) in one highstate run, without interruption.

salt/minion.sls :

  cmd.wait:
    - name: echo salt-call --local service.restart salt-minion | at now + 1 minute
    - python_shell: True
    - order: last
    - watch:
{% if salt_settings.install_packages %}
      - pkg: salt-minion
{% endif %}
      - file: salt-minion
      - file: remove-old-minion-conf-file

I didn't manage "atd" since it's on by default on Debian, but it's definitely doable.

[gravyboat]

Not as far as I'm aware. Might be a good thing to check for in pillar whether someone has reset set to true/false and act accordingly.

[iggy]

at is definitely not on by default in Debian (at least every Debian box I have access to doesn't have it). I'd prefer not to add it as a further dependency if we don't absolutely have to (i.e. it actually fixes a real bug not just a possibly annoyance).

[puneetk]

Its easier to use listen_in and a service.running with a restart. and it translates to all Os’es.

On May 19, 2015, at 12:35 PM, Brian Jackson notifications@github.com wrote:

at is definitely not on by default in Debian (at least every Debian box I have access to doesn't have it). I'd prefer not to add it as a further dependency if we don't absolutely have to (i.e. it actually fixes a real bug not just a possibly annoyance).

—
Reply to this email directly or view it on GitHub #136 (comment).

[tlemarchand]

I understand. At least on Debian Jessie, I have a bug : at the end of highstate run if salt-minion is restarted, salt-minion does not return any information to salt-master, so I only get a blank result.
About atd : All my debian are installed from netinstall, with only "base system" and "ssh" selected.

[aboe76]

@puneetk I have tested the listen_in part, and strangely on my arch box, it works

arch.example.com:
----------
          ID: salt-minion
    Function: file.recurse
        Name: /etc/salt/minion.d
      Result: True
     Comment: Recursively updated /etc/salt/minion.d
     Started: 21:28:14.689541
    Duration: 333.038 ms
     Changes:   
              ----------
              /etc/salt/minion.d/f_defaults.conf:
                  ----------
                  diff:
                      --- 
                      +++ 
                      @@ -213,7 +213,7 @@
                       # print verbose changes
                       #state_verbose: True
                       # multi line output
                      -state_output: changes
                      +state_output: terse
                       # output diff
                       #state_output_diff: False

  Name: /etc/salt/minion.d/_defaults.conf - Function: file.absent - Result: Clean
----------
          ID: salt-minion
    Function: service.running
      Result: True
     Comment: Service restarted
     Started: 21:28:15.861776
    Duration: 348.888 ms
     Changes:   
              ----------
              salt-minion:
                  True
----------
          ID: listener_salt-minion
    Function: service.mod_watch
        Name: salt-minion
      Result: True
     Comment: Service restarted
     Started: 21:28:21.153770
    Duration: 309.12 ms
     Changes:   
              ----------
              salt-minion:
                  True

Summary for arch.example.com
-------------
Succeeded: 93 (changed=4)
Failed:     0
-------------
Total states run:     93

But on my debian jessie box, it doesn't...

debian.example.com:
    Minion did not return. [No response]

This is the change in salt/minion.sls:

salt-minion:
{% if salt_settings.install_packages %}
  pkg.installed:
    - name: {{ salt_settings.salt_minion }}
{% endif %}
  file.recurse:
    - name: {{ salt_settings.config_path }}/minion.d
    - template: jinja
    - source: salt://salt/files/minion.d
    - clean: {{ salt_settings.clean_config_d_dir }}
    - exclude_pat: _*
    - context:
        standalone: False
    - listen_in:
      - service: salt-minion
  service.running:
    - enable: True
    - name: {{ salt_settings.minion_service }}
    - watch:
{% if salt_settings.install_packages %}
      - pkg: salt-minion
{% endif %}
      - file: salt-minion
      - file: remove-old-minion-conf-file

[puneetk]

Yup listen_in works on ubuntu, debian too :)

On Jun 5, 2015, at 12:31 PM, Niels Abspoel notifications@github.com wrote:

@puneetk https://github.com/puneetk I have tested the listen_in part, and strangely on my arch box, it works

arch.example.com:

      ID: salt-minion
Function: file.recurse
    Name: /etc/salt/minion.d
  Result: True
 Comment: Recursively updated /etc/salt/minion.d
 Started: 21:28:14.689541
Duration: 333.038 ms
 Changes:   
          ----------
          /etc/salt/minion.d/f_defaults.conf:
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -213,7 +213,7 @@
                   # print verbose changes
                   #state_verbose: True
                   # multi line output
                  -state_output: changes
                  +state_output: terse
                   # output diff
                   #state_output_diff: False

Name: /etc/salt/minion.d/_defaults.conf - Function: file.absent - Result: Clean

      ID: salt-minion
Function: service.running
  Result: True
 Comment: Service restarted
 Started: 21:28:15.861776
Duration: 348.888 ms
 Changes:   
          ----------
          salt-minion:
              True

---

      ID: listener_salt-minion
Function: service.mod_watch
    Name: salt-minion
  Result: True
 Comment: Service restarted
 Started: 21:28:21.153770
Duration: 309.12 ms
 Changes:   
          ----------
          salt-minion:
              True

Summary for arch.example.com

Succeeded: 93 (changed=4)

Failed: 0

Total states run: 93
But on my debian jessie box, it doesn't...

debian.example.com:
Minion did not return. [No response]
This is the change in salt/minion.sls:

salt-minion:
{% if salt_settings.install_packages %}
pkg.installed:
- name: {{ salt_settings.salt_minion }}
{% endif %}
file.recurse:
- name: {{ salt_settings.config_path }}/minion.d
- template: jinja
- source: salt://salt/files/minion.d
- clean: {{ salt_settings.clean_config_d_dir }}
- exclude_pat: _*
- context:
standalone: False
- listen_in:
- service: salt-minion
service.running:
- enable: True
- name: {{ salt_settings.minion_service }}
- watch:
{% if salt_settings.install_packages %}
- pkg: salt-minion
{% endif %}
- file: salt-minion
- file: remove-old-minion-conf-file
—
Reply to this email directly or view it on GitHub #136 (comment).

[tlemarchand]

@aboe76 This is exactly why I had to use "at" in this formula.
Maybe it's a Salt bug, or a Jessie bug, I don't know but I think it should be fixed.

[genuss]

We have additional issue with simple minion restart via watch or listen_to.
If a state.apply is run from a master which is Debian Jessie to minion which is Debian Wheezy than a minion on restart will spawn additional salt process. This is a definitely another issue, but can be worked around by a restart with at.