Fixes #27937 - API for Ansible Inventory template scheduling #299

ofedoren · 2019-09-10T15:43:16Z

To use the endpoint there should exist the Ansible Inventory report template (part of this PR)

sjha4 · 2019-09-16T15:15:31Z

We would probably also need the below fields ( or decide which ones we need from the below response) from the earlier setup:

"foreman": {
          "architecture_id": 1,
          "architecture_name": "x86_64",
          "build": "False",
          "capabilities": [
            "build"
          ],
          "certname": "samir",
          "comment": "None",
          "compute_profile_id": "None",
          "compute_profile_name": "None",
          "compute_resource_id": "None",
          "compute_resource_name": "None",
          "content_facet_attributes": {
            "applicable_module_stream_count": 0,
            "applicable_package_count": 0,
            "content_source": "None",
            "content_source_id": "None",
            "content_source_name": "None",
            "content_view": {
              "id": 1,
              "name": "Default Organization View"
            },
            "content_view_id": 1,
            "content_view_name": "Default Organization View",
            "errata_counts": {
              "bugfix": 0,
              "enhancement": 0,
              "security": 0,
              "total": 0
            },
            "id": 1,
            "kickstart_repository": "None",
            "kickstart_repository_id": "None",
            "kickstart_repository_name": "None",
            "lifecycle_environment": {
              "id": 1,
              "name": "Library"
            },
            "lifecycle_environment_id": 1,
            "lifecycle_environment_name": "Library",
            "upgradable_module_stream_count": 0,
            "upgradable_package_count": 0,
            "uuid": "None"
          },
          "created_at": "2019-09-10 18:56:49 UTC",
          "disk": "None",
          "domain_id": "None",
          "domain_name": "None",
          "enabled": "True",
          "environment_id": "None",
          "environment_name": "None",
          "global_status": 1,
          "global_status_label": "OK",
          "hostgroup_id": "None",
          "hostgroup_name": "None",
          "hostgroup_title": "None",
          "id": 1,
          "image_file": "",
          "image_id": "None",
          "image_name": "None",
          "installed_at": "None",
          "ip": "10.11.216.233",
          "ip6": "None",
          "last_compile": "2019-09-10 18:56:51 UTC",
          "last_report": "None",
          "location_id": 2,
          "location_name": "Default Location",
          "mac": "e4:70:b8:22:1d:66",
          "managed": "False",
          "medium_id": "None",
          "medium_name": "None",
          "model_id": "None",
          "model_name": "None",
          "name": "samir",
          "operatingsystem_id": 1,
          "operatingsystem_name": "CentOS 7",
          "organization_id": 1,
          "organization_name": "Default Organization",
          "owner_id": 4,
          "owner_name": "Admin User",
          "owner_type": "User",
          "provision_method": "build",
          "ptable_id": "None",
          "ptable_name": "None",
          "puppet_ca_proxy": "None",
          "puppet_ca_proxy_id": "None",
          "puppet_ca_proxy_name": "None",
          "puppet_proxy": "None",
          "puppet_proxy_id": "None",
          "puppet_proxy_name": "None",
          "puppet_status": 0,
          "pxe_loader": "None",
          "realm_id": "None",
          "realm_name": "None",
          "sp_ip": "None",
          "sp_mac": "None",
          "sp_name": "None",
          "sp_subnet_id": "None",
          "subnet6_id": "None",
          "subnet6_name": "None",
          "subnet_id": "None",
          "subnet_name": "None",
          "updated_at": "2019-09-10 18:56:54 UTC",
          "uptime_seconds": "None",
          "use_image": "None",
          "uuid": "None"
        }

sjha4 · 2019-09-16T19:23:44Z

@ofedoren : I am not sure if I am using the required param correctly, but with something like this: {'required':'all'} I get an error. Do you have an example curl with what the hash expects?

sjha4 · 2019-09-16T20:39:53Z

app/views/foreman_ansible/job_templates/ansible_-_inventory.erb

+    required = input('required').split(',').map { |r| r.strip.downcase }
+    inventory_data.merge!('Organization': host.organization) if (['all', 'organization'] & required).any?
+    inventory_data.merge!('Location': host.location) if (['all', 'location'] & required).any?
+    inventory_data.merge!('Hostgroup': host.hostgroup.name) if (['all', 'hostgroup'] & required).any?


Suggested change

inventory_data.merge!('Hostgroup': host.hostgroup.name) if (['all', 'hostgroup'] & required).any?

inventory_data.merge!('Hostgroup': host.hostgroup.name) if (['all', 'hostgroup'] & required).any? && host.hostgroup

to avoid nil reference.

So based on the new fields we want to add, I tried this:
['architecture_id','architecture_name'].each do |field| inventory_data.merge!(field=>host[field])

nor can I do something like inventory_data.merge!(field=>host.send(field))

It seems the Host::Managed::Jail does not like it.

ares

few comments, please let me know if I should provide more details

app/views/foreman_ansible/job_templates/ansible_-_inventory.erb

ares · 2019-09-17T14:08:38Z

app/views/foreman_ansible/job_templates/ansible_-_inventory.erb

+- name: required
+  required: false
+  input_type: user
+  description: Comma-separated values that should be returned.


I think it would be nicer user experience if we define separate input for every optional part, it would have restricted values yes no and would be required. That in UI should generate select boxes to chose from yes/no.

We could also use this chance to add default values to inputs (that would be PR to Foreman core)

Done: theforeman/foreman#7050

app/views/foreman_ansible/job_templates/ansible_-_inventory.erb

sjha4 · 2019-09-17T17:41:13Z

Below is the list of fields available on the host model so they can be pulled into the report:
https://gist.github.com/sjha4/eee846297dcf051e9a4f14198b444309
I am looking at the Content Facet Attributes data that is needed to create the content view and lifecycle groups for hosts.

ofedoren · 2019-09-19T17:01:57Z

The third commit requires Katello/katello#8356.

ofedoren · 2019-09-19T17:36:21Z

The fourth commit requires theforeman/foreman#7052.

ofedoren · 2019-09-19T17:39:03Z

Below is the list of fields available on the host model so they can be pulled into the report:
https://gist.github.com/sjha4/eee846297dcf051e9a4f14198b444309
I am looking at the Content Facet Attributes data that is needed to create the content view and lifecycle groups for hosts.

@sjha4, that's quite a big list and I'm not sure if we should add a separate template input for each field.

@ares, how can we choose what should be in the report (or at least as template inputs) and what should be not?

sjha4 · 2019-09-19T18:38:34Z

@ofedoren : Right..It's >70 fields in the intial API call I was testng with. I am not sure we need all the details on the report.
We do need the content_facet_attributes. I have a PR in katello Katello/katello#8357 and a dummy report here: http://pastebin.test.redhat.com/798999 with content_facety_attributes that we would need to add to this report.

app/controllers/api/v2/ansible_inventories_controller.rb

ofedoren · 2019-09-23T18:24:48Z

@sjha4, thanks for your input! I've added Content Facet Attributes to the template.

ares · 2019-09-25T12:30:00Z

You can use https://projects.theforeman.org/issues/27937 for tracking this

ares

Nice work so far, leaving few comments

app/controllers/api/v2/ansible_inventories_controller.rb

app/views/foreman_ansible/job_templates/ansible_-_inventory.erb

ofedoren

Thanks for review! To simplify (I hope for testing purposes as well) I squashed the commits to one and to correspond with the issue.

app/controllers/api/v2/ansible_inventories_controller.rb

app/views/foreman_ansible/job_templates/ansible_-_inventory.erb

app/controllers/api/v2/ansible_inventories_controller.rb

ares · 2019-09-30T07:24:27Z

looks well now, safe mode PRs seems to be in, this is now blocked on default values, description and format PRs in core

ofedoren · 2019-10-01T17:12:09Z

The template has been updated:

By default the following data are in the report: Host name, Organization, Location, Host Group, IPv4. Other can be configured via template inputs.
Host group is shown with nesting included (as in UI, e.g. Base/Katello CentOS 7). This requires Fixes #27891 - Allow host group titles in safe mode foreman#7075

mmoll · 2019-10-11T01:28:16Z

please rebase

ofedoren · 2019-10-11T13:04:35Z

@mmoll, rebased.

mmoll · 2019-10-11T14:34:28Z

now it fails on rubocop

ofedoren · 2019-10-11T15:07:50Z

@mmoll, thanks :) fixed.

mmoll · 2019-10-11T17:50:01Z

almost there, permission missing :)

ares

This should probably resuse existing core permission for generating reports

ares · 2019-10-14T09:35:26Z

lib/foreman_ansible/register.rb

@@ -82,7 +82,7 @@
        :edit_ansible_variables, :destroy_ansible_variables]

  role 'Ansible Tower Inventory Reader',
-       [:view_hosts, :view_hostgroups, :view_facts],
+       [:view_hosts, :view_hostgroups, :view_facts, :view_schedule],


Do we need separate view_schedule permission? I think this should be the same permission as for genertimg regular report

I've reused generate_report_tempaltes permission if it makes sense.

yes, that's exactly what I had in mind

ares · 2019-10-16T13:10:30Z

[test foreman_ansible]

ares

Tested, ack pending jenkins

ares · 2019-10-17T07:39:52Z

Thanks @ofedoren, merging!

theforeman-bot added Needs testing Not yet reviewed labels Sep 10, 2019

ofedoren force-pushed the feat-inventory-rep-eng branch from 163075a to f7e803c Compare September 13, 2019 13:11

sjha4 reviewed Sep 16, 2019

View reviewed changes

ares requested changes Sep 17, 2019

View reviewed changes

theforeman-bot added Waiting on contributor and removed Not yet reviewed labels Sep 17, 2019

This was referenced Sep 17, 2019

[WIP] Update foreman inventory to use foreman's inventory report ansible/awx#4758

Closed

Update foreman inventory to use foreman's inventory report ansible/ansible#62438

Merged

ofedoren force-pushed the feat-inventory-rep-eng branch from f7e803c to b0ee88e Compare September 18, 2019 18:30

theforeman-bot added Needs re-review and removed Waiting on contributor labels Sep 18, 2019

ezr-ondrej reviewed Sep 20, 2019

View reviewed changes

app/controllers/api/v2/ansible_inventories_controller.rb Outdated Show resolved Hide resolved

ares requested changes Sep 26, 2019

View reviewed changes

theforeman-bot added Waiting on contributor Needs re-review and removed Needs re-review Waiting on contributor labels Sep 26, 2019

ofedoren force-pushed the feat-inventory-rep-eng branch from 61bef82 to eb0d67e Compare September 27, 2019 17:59

ofedoren commented Sep 27, 2019

View reviewed changes

ofedoren force-pushed the feat-inventory-rep-eng branch from eb0d67e to a037cc3 Compare October 1, 2019 17:07

ofedoren force-pushed the feat-inventory-rep-eng branch from a037cc3 to 7588d58 Compare October 8, 2019 15:56

ofedoren changed the title ~~[WIP] API for Ansible Inventory template scheduling~~ Fixes #27937 - API for Ansible Inventory template scheduling Oct 8, 2019

ofedoren force-pushed the feat-inventory-rep-eng branch from 7588d58 to 3e077fb Compare October 8, 2019 15:57

sjha4 mentioned this pull request Oct 9, 2019

Update foreman inventory to use foreman's inventory report ansible/awx#4988

Merged

ofedoren force-pushed the feat-inventory-rep-eng branch from 3e077fb to 2060412 Compare October 11, 2019 13:04

ofedoren force-pushed the feat-inventory-rep-eng branch from 2060412 to 9bb59b4 Compare October 11, 2019 15:06

ofedoren force-pushed the feat-inventory-rep-eng branch from 9bb59b4 to ae55223 Compare October 11, 2019 17:55

ares requested changes Oct 14, 2019

View reviewed changes

theforeman-bot added Waiting on contributor and removed Needs re-review labels Oct 14, 2019

ofedoren force-pushed the feat-inventory-rep-eng branch from ae55223 to 74c6206 Compare October 14, 2019 15:24

theforeman-bot added Needs re-review and removed Waiting on contributor labels Oct 14, 2019

Fixes #27937 - API for Ansible Inventory template scheduling

7267312

ofedoren force-pushed the feat-inventory-rep-eng branch from 74c6206 to 7267312 Compare October 14, 2019 15:27

ares approved these changes Oct 16, 2019

View reviewed changes

theforeman-bot removed the Needs re-review label Oct 16, 2019

ares added the 4.0 label Oct 16, 2019

ares merged commit 20a5a44 into theforeman:master Oct 17, 2019

ezr-ondrej mentioned this pull request Aug 25, 2020

Improving Inventory Foreman theforeman/foreman-ansible-modules#851

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes #27937 - API for Ansible Inventory template scheduling #299

Fixes #27937 - API for Ansible Inventory template scheduling #299

ofedoren commented Sep 10, 2019

sjha4 commented Sep 16, 2019

sjha4 commented Sep 16, 2019

sjha4 Sep 16, 2019

sjha4 Sep 17, 2019

ares left a comment

ares Sep 17, 2019

ofedoren Sep 19, 2019

sjha4 commented Sep 17, 2019

ofedoren commented Sep 19, 2019 •

edited

ofedoren commented Sep 19, 2019

ofedoren commented Sep 19, 2019

sjha4 commented Sep 19, 2019

ofedoren commented Sep 23, 2019

ares commented Sep 25, 2019

ares left a comment

ofedoren left a comment

ares commented Sep 30, 2019

ofedoren commented Oct 1, 2019

mmoll commented Oct 11, 2019

ofedoren commented Oct 11, 2019

mmoll commented Oct 11, 2019

ofedoren commented Oct 11, 2019

mmoll commented Oct 11, 2019

ares left a comment

ares Oct 14, 2019

ofedoren Oct 14, 2019

ares Oct 16, 2019

ares commented Oct 16, 2019

ares left a comment

ares commented Oct 17, 2019

	inventory_data.merge!('Hostgroup': host.hostgroup.name) if (['all', 'hostgroup'] & required).any?
	inventory_data.merge!('Hostgroup': host.hostgroup.name) if (['all', 'hostgroup'] & required).any? && host.hostgroup

Fixes #27937 - API for Ansible Inventory template scheduling #299

Fixes #27937 - API for Ansible Inventory template scheduling #299

Conversation

ofedoren commented Sep 10, 2019

sjha4 commented Sep 16, 2019

sjha4 commented Sep 16, 2019

sjha4 Sep 16, 2019

Choose a reason for hiding this comment

sjha4 Sep 17, 2019

Choose a reason for hiding this comment

ares left a comment

Choose a reason for hiding this comment

ares Sep 17, 2019

Choose a reason for hiding this comment

ofedoren Sep 19, 2019

Choose a reason for hiding this comment

sjha4 commented Sep 17, 2019

ofedoren commented Sep 19, 2019 • edited

ofedoren commented Sep 19, 2019

ofedoren commented Sep 19, 2019

sjha4 commented Sep 19, 2019

ofedoren commented Sep 23, 2019

ares commented Sep 25, 2019

ares left a comment

Choose a reason for hiding this comment

ofedoren left a comment

Choose a reason for hiding this comment

ares commented Sep 30, 2019

ofedoren commented Oct 1, 2019

mmoll commented Oct 11, 2019

ofedoren commented Oct 11, 2019

mmoll commented Oct 11, 2019

ofedoren commented Oct 11, 2019

mmoll commented Oct 11, 2019

ares left a comment

Choose a reason for hiding this comment

ares Oct 14, 2019

Choose a reason for hiding this comment

ofedoren Oct 14, 2019

Choose a reason for hiding this comment

ares Oct 16, 2019

Choose a reason for hiding this comment

ares commented Oct 16, 2019

ares left a comment

Choose a reason for hiding this comment

ares commented Oct 17, 2019

ofedoren commented Sep 19, 2019 •

edited