Added support of modulemd to combined profile; ENT-834 #1927

jirihnidek · 2018-09-25T16:03:41Z

Added support for reporting enabled modules, when DNF is used
and Modularity is supported by DNF Python API
Added unit test for modulemd profile
Fixed and refactored unit tests of enabled repositories profile

nikosmoum

For now I have mostly some questions around how this works, before requesting any changes.

nikosmoum · 2018-09-26T16:07:26Z

src/rhsm/profile.py

 except ImportError:
    dnf = None
+    libdnf = None


You should probably split this into 2 try/except blocks (one for dnf, one for libdnf), because when I installed dnf (but not libdnf), the except block set dnf to None (even though dnf was successfully imported). Unless this is by design, in which case, ignore this.

Also, how can I import the libdnf python module? (pip search doesn't find it)

@nikosmoum I'm sorry that I didn't provide more instruction for testing this PR. It could be possible to install libdnf from Fedora repository, but you will need master branch of libdnf.

This PR can be tested on Fedora 28 (support for modularity). Look at this two articles:

https://fedoramagazine.org/modularity-fedora-28-server-edition/

https://fedoramagazine.org/working-modules-fedora-28/

and add install some modules to the system. You will also need master branch of libdnf and master branch of DNF using master branch of libdnf. It is not necessary to install libdnf to the system. Instruction for our feature branch related to libdnf plugins can help you:

https://github.com/candlepin/subscription-manager/tree/feature/libdnf_plugins/src/dnf-plugins/test_plugin#not-installing-to-the-root

When you will test this PR, then you will have to add path of DNF source code to PYTHONPATH.

And you are right. I should split importing of two modules into two try/except blocks.

nikosmoum · 2018-09-26T16:15:07Z

src/rhsm/profile.py

+            except AttributeError:
+                log.info("DNF does not provide modulemd functionality")
+                return []
+            all_module_list = modules.getModulePackages()


When I run the code, I always get the AttributeError exception, because the _moduleContainer field doesn't exist. Is this always going to be the case? How can I get to the next block, where the _moduleContainer actually is there?

Let's look at my reply to your previous comment. Master branch of libdnf provides this attribute.

nikosmoum · 2018-09-26T16:18:07Z

test/test_cache.py

@@ -135,6 +151,7 @@ def test_combined_profile_update_check_has_changed(self):
        uuid = 'FAKEUUID'
        uep = Mock()
        uep.has_capability = Mock(return_value=True)
+        print(self.current_profile)


Do we need this line? Or is it forgotten debug output?

Good point.

parthaa · 2018-10-01T20:43:08Z

@jirihnidek I know this is a stretch (just asking as extra credit) but can you provide information on whether a module is "locked" or "unlocked" .

jirihnidek · 2018-10-08T10:23:38Z

@jirihnidek I know this is a stretch (just asking as extra credit) but can you provide information on whether a module is "locked" or "unlocked" .

Hi @parthaa, It seems that support for locking of modules was dropped from libdnf and dnf (master branch). See:

PYTHONPATH=`readlink -f .` bin/dnf-3 module list --enabled
Last metadata expiration check: 2:03:32 ago on Mon Oct  8 10:14:52 2018.
Fedora Modular 28 - x86_64
Name                                        Stream                                 Profiles                                                        Summary                                                         
django                                      1.6 [e]                                python2_development, default [d]                                A high-level Python Web framework                               

Fedora Modular 28 - x86_64 - Updates
Name                                        Stream                                 Profiles                                                        Summary                                                         
django                                      1.6 [e]                                python2_development, default [d]                                A high-level Python Web framework                               
reviewboard                                 3.0 [e]                                server, default [i]                                             A web-based code review tool                                    

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled

* Added support for reporting enabled modules, when DNF is used and Modularity is supported by DNF Python API * Added unit test for modulemd profile * Fixed and refactored unit tests of enabled repositories profile

This commit adds routing and api bindings to import the combined report from subscription manager Background: Subscription Manager is going to consolidate * Package Profiles * Module MD Data * Enabled Repository report all in a single payload to a new end point (candlepin/subscription-manager#1927) As a user I would like Katello to accept the new payload from Subscription Manager This commit does the following * Add a "profiles" endpoint where subman sends reports * Make sense of the package profile and enabled reports * Make the event queue raise the appropriate events.

Adds code to import module inventory from subscription manager This commit adds routing and api bindings to import the combined report from subscription manager Background: Subscription Manager is going to consolidate * Package Profiles * Module MD Data * Enabled Repository report all in a single payload to a new end point (candlepin/subscription-manager#1927) As a user I would like Katello to accept the new payload from Subscription Manager This commit does the following * Implements the module md inventory endpoint * Adds the models and controller for Available and HostAvailable module streams * Adds basic UI to show enabled/disabled/installed module streams

Adds code to import module inventory from subscription manager This commit adds routing and api bindings to import the combined report from subscription manager Background: Subscription Manager is going to consolidate * Package Profiles * Module MD Data * Enabled Repository report all in a single payload to a new end point (candlepin/subscription-manager#1927) As a user I would like Katello to accept the new payload from Subscription Manager This commit does the following * Implements the module md inventory endpoint * Adds the models and controller for Available and HostAvailable module streams * Adds basic UI to show enabled/disabled/installed module streams more