Target IG by hosts

[zoullou]

ISSUE TYPE

• Feature Idea

SUMMARY

Hello,

I have hundred of server on one inventory and i want to launch a job template on all theses servers. But all server can't be joined by the same IG due to DMZ. So actually, i have to split my inventory into several inventory and associating specific IG (with severs in right DMZ location) to all inventory and i have to launch the same template several times on all inventory.

Could it be possible to specify the IG to use at host level in the inventory by adding custom var for host and fail back to default IG in case if this var is not set.

all:
children:
custom_group:
hosts:
server1:
instance_group: dmz1_ig
server2:
instance_group: dmz2_ig
server3:

Thanks

[AlexSCorey]

This might be something for us to consider in the future. In the meantime I think you could:

• On the Job Template that you have to launch you could select Prompt on Launch for the inventory. Then, create a workflow job template that uses that job template repeatedly, only with the different inventories specified when you create each workflow node and then launch the workflow.

I hope this is helpful

[barzog]

This might be something for us to consider in the future. In the meantime I think you could:

• On the Job Template that you have to launch you could select Prompt on Launch for the inventory. Then, create a workflow job template that uses that job template repeatedly, only with the different inventories specified when you create each workflow node and then launch the workflow.

So, instead of running job in multiple instance groups where IG is based on host inventory hostvar we must manually create workflow for each job template and in each workflow we must add same job template but with different inventory value (IG is set on inventory level).
This kind of work is not error prone and disable single source of truth in git for inventory files (workflows are not getted from git)...

May be other solutions exists?
One that comes to my mind (but only as a workaround): for each job template create supporting job template: where we will use awx.awx collection and create temporary inventory groups with necessary instancegroup based on hostvars and then spawn 'original' job template using temporary inventory.
So complexity will be n+n (original job + support job) (and fully git based) instead of n+nm (original job+workflownumber of instance group)

[AlanCoding]

I see some overlap with #2941, in particular

Select multiple inventories, and slice by inventory

I'm open to discussing solutions where the framing is that workflow (or other feature) parameterization is insufficient for the purpose. Right now, I'm also very interested in re-working the design of smart inventory and sliced job features to be something more flexible, general, and coherent.

Sliced Jobs

We have 1 input to JTs to make them into a sliced job, which is the job_slice_count, and when launched it will automatically create a workflow. This has always seemed a little narrow to me. It would make more sense as a generic "auto workflow creator". Instead of the single integer input, we could turn this feature into a superset of the current feature by allowing templating to define the workflow node properties based on host values. In your case, you would seek to define the node's instance group based on a hostvar (if I read it correctly).

Smart Inventory

This feature produces 1 subset of hosts based on a pool of hosts. You have the problem of breaking up the pool of hosts from 1 inventory into multiple slices, and running with different related prompts or parameters.

This process of (pool of hosts) --> (subset) also seems narrow to me. In Ansible core, a "host pattern" defines a subset of hosts to target, but that's too limited and not what most users want to get out of the feature. The ideal version of this feature might be a better analog to Ansible's group_by, where it goes (pool of hosts) --> (multiple groupings). As I say group_by, that starts to make smart inventory look similar to sliced jobs... but in the sliced jobs case, there is no resultant inventory tracked in the database for the subsets we run across. I think that's a flawed pattern, because the targeted inventory is hidden from the user. If possible, I would like to changed sliced jobs to force creation of the subset inventories for the user's reference. I envision some kind of powerfully-general multi-inventory (to replace smart inventory), which can be pared with a automatic workflow-spawning JT (to replace sliced jobs). Then when combining the two, the templates operate together to slice the inventory into sub-inventories and add node properties in one loop over the full source host set.

[barzog]

As for me - looks great. Regarding smart inventory - we use that only for simplest job templates, because smart invs does not support group_vars (and groups from main inventory as well) as per issue: #1999. If the proposed solutions will cover following workflow:
JT run with Smart Inv as host source (i.e. run job on all Ubuntu hosts) and when some of that hosts have host_var or group_var stating that it should run on specific instance_group - do it - this would be really nice.

[barzog]

With recent introduction of constructed inventory it seems that things are moving in the right direction. I.e. now we have host/group vars from constructed (thats a huge improvement over smart inventory) plus custom groups with own vars (which can be set from git). But still instance group setting from host vars (or from static inventory) feature is seems missing (or unclear how to configure). May be @AlanCoding cad shed some light on it?

Tested with following:
No Instance Set on Organization Level. We have two instance groups: default and other one.
Created two static inventories with different instance group set (1st with default instance group, 2nd with other one).
Created job template without setting instance group. Checked job template on both static inventory - correct instance group is set from inventory, job execution is successful.
Created constructed inventory based on two static inventories without setting instance group with following config:
plugin: ansible.builtin.constructed
strict: true
use_vars_plugins: true
keyed_groups:
- prefix: distro
key: ansible_distribution
Synced ok (no errors shown)
Checked job template with host from first instance group (default) - OK.
Checked job template with host from 2nd instance group (non-default) - not OK.