Cannot read hostgroups without content_source, content_view or lifecycle_environment

[ifireball]

When trying to run read() on a hostgroup that doesn't have the $subject properties set (Such hostgroup could easily be created from the UI), the following exception is yielded:

.../python2.7/site-packages/nailgun/entities.pyc in read(self, entity, attrs, ignore)
   1964                          'content_view_id',
   1965                          'lifecycle_environment_id'):
-> 1966                 attrs[attr] = attrs2[attr]
   1967         return super(HostGroup, self).read(entity, attrs, ignore)
   1968 

KeyError: 'content_source_id'

[abalakh]

@ifireball The issue here is your Satellite is not returning content_source_id field via PUT request. It's basically application bug, but we can workaround it on nailgun side. The thing is I can't reproduce it.
Could you please specify your Satellite version? For Sat 6.2 it's not reproducible as mentioned in traceback code snipped will be skipped. Tested manually just in case - not reproducible.

I've tried it on Sat 6.1.8 and still no success.
At first i've tried to create and read hostgroup via nailgun:

>>> entities.HostGroup().create()
2016-04-26 16:03:56 - nailgun.client - DEBUG - Making HTTP POST request to https://example.com/api/v2/hostgroups with options {'verify': False, 'auth': ('admin', 'secret_password_here'), 'headers': {'content-type': 'application/json'}} and data {"hostgroup": {"name": "\ud867\udd3e\ud849\udef6\ud80c\udd15\ud853\udd88\ud850\udd46\ud861\udfc0\u4ac1\u5b90"}}.
2016-04-26 16:03:57 - nailgun.client - DEBUG - Received HTTP 200 response: {"id":1,"name":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","title":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","subnet_id":null,"subnet_name":null,"operatingsystem_id":null,"operatingsystem_name":null,"domain_id":null,"domain_name":null,"environment_id":null,"environment_name":null,"compute_profile_id":null,"compute_profile_name":null,"ancestry":null,"puppet_proxy_id":null,"puppet_ca_proxy_id":null,"ptable_id":null,"ptable_name":null,"medium_id":null,"medium_name":null,"architecture_id":null,"architecture_name":null,"realm_id":null,"realm_name":null,"created_at":"2016-04-26T13:03:57Z","updated_at":"2016-04-26T13:03:57Z","all_puppetclasses":[],"locations":[],"organizations":[],"parameters":[],"template_combinations":[],"puppetclasses":[],"config_groups":[]}
2016-04-26 16:03:57 - nailgun.client - DEBUG - Making HTTP GET request to https://example.com/api/v2/hostgroups/1 with options {'verify': False, 'auth': ('admin', 'secret_password_here'), 'headers': {'content-type': 'application/json'}} and no data.
2016-04-26 16:03:58 - nailgun.client - DEBUG - Received HTTP 200 response: {"id":1,"name":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","title":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","subnet_id":null,"subnet_name":null,"operatingsystem_id":null,"operatingsystem_name":null,"domain_id":null,"domain_name":null,"environment_id":null,"environment_name":null,"compute_profile_id":null,"compute_profile_name":null,"ancestry":null,"puppet_proxy_id":null,"puppet_ca_proxy_id":null,"ptable_id":null,"ptable_name":null,"medium_id":null,"medium_name":null,"architecture_id":null,"architecture_name":null,"realm_id":null,"realm_name":null,"created_at":"2016-04-26T13:03:57Z","updated_at":"2016-04-26T13:03:57Z","all_puppetclasses":[],"locations":[],"organizations":[],"parameters":[],"template_combinations":[],"puppetclasses":[],"config_groups":[]}
2016-04-26 16:03:58 - nailgun.client - DEBUG - Making HTTP PUT request to https://example.com/api/v2/hostgroups/1 with options {'verify': False, 'auth': ('admin', 'secret_password_here'), 'headers': {'content-type': 'application/json'}} and data {"hostgroup": {}}.
2016-04-26 16:03:59 - nailgun.client - DEBUG - Received HTTP 200 response: {"ancestry":null,"architecture_id":null,"compute_profile_id":null,"content_source_id":null,"content_view_id":null,"created_at":"2016-04-26T13:03:57Z","domain_id":null,"environment_id":null,"grub_pass":"","id":1,"image_file":"","lifecycle_environment_id":null,"medium_id":null,"name":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","operatingsystem_id":null,"ptable_id":null,"puppet_ca_proxy_id":null,"puppet_proxy_id":null,"realm_id":null,"root_pass":null,"subnet_id":null,"title":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","updated_at":"2016-04-26T13:03:57Z","use_image":null,"vm_defaults":null}
nailgun.entities.HostGroup(nailgun.config.ServerConfig(url='https://example.com', verify=False, version='6.1.8', auth=('admin', 'secret_password_here')), domain=None, medium=None, parent=None, ptable=None, content_source=None, lifecycle_environment=None, id=1, name=u'\U00029d3e\U000226f6\U00013115\U00024d88\U00024146\U000287c0\u4ac1\u5b90', subnet=None, puppet_proxy=None, realm=None, content_view=None, operatingsystem=None, environment=None, location=[], organization=[], puppet_ca_proxy=None, architecture=None)
>>> entities.HostGroup(id=1).read()
2016-04-26 16:04:16 - nailgun.client - DEBUG - Making HTTP GET request to https://example.com/api/v2/hostgroups/1 with options {'verify': False, 'auth': ('admin', 'secret_password_here'), 'headers': {'content-type': 'application/json'}} and no data.
2016-04-26 16:04:16 - nailgun.client - DEBUG - Received HTTP 200 response: {"id":1,"name":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","title":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","subnet_id":null,"subnet_name":null,"operatingsystem_id":null,"operatingsystem_name":null,"domain_id":null,"domain_name":null,"environment_id":null,"environment_name":null,"compute_profile_id":null,"compute_profile_name":null,"ancestry":null,"puppet_proxy_id":null,"puppet_ca_proxy_id":null,"ptable_id":null,"ptable_name":null,"medium_id":null,"medium_name":null,"architecture_id":null,"architecture_name":null,"realm_id":null,"realm_name":null,"created_at":"2016-04-26T13:03:57Z","updated_at":"2016-04-26T13:03:57Z","all_puppetclasses":[],"locations":[],"organizations":[],"parameters":[],"template_combinations":[],"puppetclasses":[],"config_groups":[]}
2016-04-26 16:04:16 - nailgun.client - DEBUG - Making HTTP PUT request to https://example.com/api/v2/hostgroups/1 with options {'verify': False, 'auth': ('admin', 'secret_password_here'), 'headers': {'content-type': 'application/json'}} and data {"hostgroup": {}}.
2016-04-26 16:04:17 - nailgun.client - DEBUG - Received HTTP 200 response: {"ancestry":null,"architecture_id":null,"compute_profile_id":null,"content_source_id":null,"content_view_id":null,"created_at":"2016-04-26T13:03:57Z","domain_id":null,"environment_id":null,"grub_pass":"","id":1,"image_file":"","lifecycle_environment_id":null,"medium_id":null,"name":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","operatingsystem_id":null,"ptable_id":null,"puppet_ca_proxy_id":null,"puppet_proxy_id":null,"realm_id":null,"root_pass":null,"subnet_id":null,"title":"𩴾𢛶𓄕𤶈𤅆𨟀䫁宐","updated_at":"2016-04-26T13:03:57Z","use_image":null,"vm_defaults":null}
nailgun.entities.HostGroup(nailgun.config.ServerConfig(url='https://example.com', verify=False, version='6.1.8', auth=('admin', 'secret_password_here')), domain=None, medium=None, parent=None, ptable=None, content_source=None, lifecycle_environment=None, id=1, name=u'\U00029d3e\U000226f6\U00013115\U00024d88\U00024146\U000287c0\u4ac1\u5b90', subnet=None, puppet_proxy=None, realm=None, content_view=None, operatingsystem=None, environment=None, location=[], organization=[], puppet_ca_proxy=None, architecture=None)

Then I tried to create a hostgroup via UI ('New Hostgroup' > Specified name > Clicked 'Submit', no additional fields) and read it:

>>> entities.HostGroup(id=2).read()
2016-04-26 16:05:16 - nailgun.client - DEBUG - Making HTTP GET request to https://example.com/api/v2/hostgroups/2 with options {'verify': False, 'auth': ('admin', 'secret_password_here'), 'headers': {'content-type': 'application/json'}} and no data.
2016-04-26 16:05:16 - nailgun.client - DEBUG - Received HTTP 200 response: {"id":2,"name":"testgroup","title":"testgroup","subnet_id":null,"subnet_name":null,"operatingsystem_id":null,"operatingsystem_name":null,"domain_id":null,"domain_name":null,"environment_id":null,"environment_name":null,"compute_profile_id":null,"compute_profile_name":null,"ancestry":null,"puppet_proxy_id":null,"puppet_ca_proxy_id":null,"ptable_id":null,"ptable_name":null,"medium_id":null,"medium_name":null,"architecture_id":null,"architecture_name":null,"realm_id":null,"realm_name":null,"created_at":"2016-04-26T13:05:06Z","updated_at":"2016-04-26T13:05:06Z","all_puppetclasses":[],"locations":[],"organizations":[{"id":1,"name":"Default Organization","title":"Default Organization"}],"parameters":[],"template_combinations":[],"puppetclasses":[],"config_groups":[]}
2016-04-26 16:05:16 - nailgun.client - DEBUG - Making HTTP PUT request to https://example.com/api/v2/hostgroups/2 with options {'verify': False, 'auth': ('admin', 'secret_password_here'), 'headers': {'content-type': 'application/json'}} and data {"hostgroup": {}}.
2016-04-26 16:05:17 - nailgun.client - DEBUG - Received HTTP 200 response: {"ancestry":null,"architecture_id":null,"compute_profile_id":null,"content_source_id":null,"content_view_id":null,"created_at":"2016-04-26T13:05:06Z","domain_id":null,"environment_id":null,"grub_pass":"","id":2,"image_file":"","lifecycle_environment_id":null,"medium_id":null,"name":"testgroup","operatingsystem_id":null,"ptable_id":null,"puppet_ca_proxy_id":null,"puppet_proxy_id":null,"realm_id":null,"root_pass":null,"subnet_id":null,"title":"testgroup","updated_at":"2016-04-26T13:05:06Z","use_image":null,"vm_defaults":null}
nailgun.entities.HostGroup(nailgun.config.ServerConfig(url='https://example.com', verify=False, version='6.1.8', auth=('admin', 'secret_password_here')), domain=None, medium=None, parent=None, ptable=None, content_source=None, lifecycle_environment=None, id=2, name=u'testgroup', subnet=None, puppet_proxy=None, realm=None, content_view=None, operatingsystem=None, environment=None, location=[], organization=[nailgun.entities.Organization(nailgun.config.ServerConfig(url='https://example.com', verify=False, version='6.1.8', auth=('admin', 'secret_password_here')), id=1)], puppet_ca_proxy=None, architecture=None)

And, as you can see here, it was read successfully and content_source_id field was correctly returned in my case. Any help in reproducing the issue?

[ifireball]

I actually made this happen with upstream Katello - you can get my exact configuration by using the Vagrantfile here: https://github.com/ifireball/python-satellite-dsl/blob/master/Vagrantfile

The important part is that I did not change the default configuration, I just created one simple hostgroup. Most importantly, I did not create any content management configuration.

[abalakh]

As far as you have all the environment set and ready... Could you please update this one line in your path/to/nailgun/nailgun/entities.py and tell me if it resolves the issue?

diff --git a/nailgun/entities.py b/nailgun/entities.py
index 4579c05..46b69ad 100644
--- a/nailgun/entities.py
+++ b/nailgun/entities.py
@@ -2006,7 +2006,7 @@ class HostGroup(
             for attr in ('content_source_id',
                          'content_view_id',
                          'lifecycle_environment_id'):
-                attrs[attr] = attrs2[attr]
+                attrs[attr] = attrs2.get(attr)
         return super(HostGroup, self).read(entity, attrs, ignore)

     def update(self, fields=None):

I believe the fix should be as simple as that.