Allow orchestrate to call orchestration files

[ealphonse]

Description of Issue/Question

I have a use-case that requires calling orchestration files within state.orchestrate.

I am developing a continuous deployment pipeline in salt: I have the same
orchestration file used to deploy at various stages + rollback if needed. It is
like a procedure which args are pillar data.

The stages are dealt with another orchestration file, and if deployment
fails (onfail) I rollback to the lastest working version. Salt orchestration
is powerful to script setups like that.

The documentation hints toward using module.run: - name: saltutil.runner - _fun: state.orchestrate or salt.runner: - name: state.orchestrate but it is not working as expected.

I filled in issue #33388 but a bigger problem is that this nested call does not
pass the return value. The root cause seems to be that saltstack does not
support calling orchestrate from orchestrate.

As discussed with @whiteinge, allowing it could bring more expressive power to
salt, and by gathering new data between each orchestration file call, one
could deal with the limitation that the whole tree must be compiled at the top of
the run. Note that the include directive does not help here.

With this feature, it is very easy to visualize and implement a continuous
deployment pipeline in salt (maybe with the handling of issue #28682 too).

I guess it is general enough to bring more power to other use-cases as well.

To summarize, calling orchestration files with orchestrate would allow to implement procedures in salt and get their benefits:

• reusing code
• gathering new data during execution to act on (new context)
• bringing more expressive power and less relying on external tools.

If you think it would be a helpful addition, my 2 cents would be to define a new saltmod function to directly call an orchestration file. This function would accept pillar and saltenv args. In doing so, it would be easy to understand from the doc, and also would allow to output a readable execution trace.

Thank you in advance for your thoughts on this.

[whiteinge]

Good stuff. Thanks for typing that up.

[whiteinge]

Thinking this over a bit: I think the 'salt' state module would be a good place for this addition to live. It could be called from Orchestrate or also from minion-side state runs (which would work for masterless as well).

Thoughts on other benefits of this proposal:

• With States now the logic for entire state run must be compiled at the top of the run. This would allow for incremental compiles that can adapt to system changes during the run.
  • I would be very interested to explore how pass the running dictionary for the parent run to the child so the sub-state run could make decisions based on the results from other states.
• Using requisites you could conditionally kick off sub-state runs.

[Karunamon]

I think I just ran into this in an attempt to reduce code duplication in my orchestration jobs. A number of high-level tasks (think bringing down a NAS or reconfiguring core networking) mean that some applications in my infrastructure need to be restarted, and I wanted to have one single source of truth for the startup/shutdown process.

Example sls:

# /srv/salt/orchestrate/application/test.sls

some_random_state:
  salt.function:
    - name: cmd.run
    # Just a bogus command to demonstrate a requirement inside orchestrate
    - target: 'saltmstr*'
    - args:
      - uname -a
    - require:
      - salt: restart_master

restart_master:
  salt.runner:
    - name: state.orchestrate
    - mods:
      - deployment.appliction.master_restart


# /srv/salt/orchestrate/application/master_restart.sls
# Start/stop commands replaced with status commands for testing

stop_app_master:
  salt.function:
    - tgt: 'appsrv01'
    - name: cmd.run
    - arg:
      - . /etc/profile && /etc/init.d/appdaemon status


start_app_master:
  salt.function:
    - tgt: 'appsrv01'
    - name: cmd.run
    - arg:
      - . /etc/profile && /etc/init.d/appdaemon status

Executing this state with salt-run state.orchestrate orchestrate.application.test --out=yaml results in the output linked here

The calling orchestration run doesn't see the true result from the required run, and so comes back requisite failed.

[whiteinge]

Take a look at the addition in #39670. It is a good first step into making this happen because it is highstate-aware and can parse the return from an inner Orchestrate run.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.